Brad Fitzpatrick ([info]bradfitz) wrote in [info]lj_backend,
@ 2004-04-27 15:27:00
Previous Entry  Add to memories!  Tell a Friend!  Next Entry
Fun stuff lately in server land...
Presentation I did at MySQL conference in Orlando:

http://www.danga.com/words/2004_mysqlcon/

Building a distributed filesystem for Fotobilder/LiveJournal (will be open source):

http://www.livejournal.com/~brad/2009886.html
http://www.livejournal.com/~brad/2010534.html
http://www.livejournal.com/~brad/2010997.html

We just bought 2 machines with 16 250GB disks, so we'll soon have 8TB of storage. I imagine we'll get about 6TB of real storage out of that after redundancy. (thumbnails and scaled versions will only be on disk once, probably, since they can be recreated easily....)

Building a new load balancer for FotoBilder/LiveJournal, with special support for mixing efficient buffer of mod_perl requests and for efficiently serving large files (using sendfile(2)) from disk, so mod_perl doesn't have to do it:

http://www.livejournal.com/~brad/2007943.html

The proxy works already w/ FotoBilder. Haven't put it into production yet, but we rebooted all our LiveJournal proxies into Debian testing w/ epoll.h headers so we could build IO::Epoll (which is a requirement for Perlbal). They were already running Linux 2.6 (for epoll)

(Post a new comment)


[info]ydna
2004-04-27 04:42 pm UTC (link)
Are you still getting machines from Silicon Mechanics? Still jazzed with them?

(Reply to this) (Thread)

[info]lisa
2004-04-27 05:08 pm UTC (link)
Yes and yes!

(Reply to this) (Parent)

[info]jamesd
2004-04-28 12:29 am UTC (link)
After a recommendation from Brad, Wikipedia switched to Silicon Mechanics. Happy experiences so far. People who know what they are talking about when you call their 800 number at 5PM Pacific on Sunday and equipment which so far hasn't given hardware trouble.

One experience which prompted the change was a Penguin Computer database machine which is, will be or already was on it's second (or is it third?) trip back to them, this time for a refund. Failed to stay up for more than a few days. Failed memory tests after the first trip there and back.

Have been some issues with the SM machines over the last week, two Squid cache machines going down, but both look to be software issues.

I'm not the person who decides where the orders go but SM looks like the ongoing primary choice for a site which wants reliability out of the box (but tests anyway, of course).

(Reply to this) (Parent)(Thread)

[info]bradfitz
2004-04-28 11:07 pm UTC (link)
Feedster just ordered a bunch of machines on our recommendation as well.

(Reply to this) (Parent)
Presentation
[info]geekpixie
2004-04-27 06:07 pm UTC (link)
That presentation is really great, I work with a php/mysql DB every day, and we've had an explosion in traffic and users in the past year that's caused us to evolve and proceed much the way LJ did, it was really interesting to see some of the same thought processes played out elsewhere :) That and the sense of humor used in the presentation is very similar to our company culture. A company of 12+ people, all working mainly on the web (sound familiar?)

Thanks for sharing!

(Reply to this)


[info]monomyth
2004-04-27 06:39 pm UTC (link)
Why not xraids :)?

(Reply to this) (Thread)

[info]bradfitz
2004-04-28 11:06 pm UTC (link)
Because they're frickin' expensive.

(Reply to this) (Parent)

[info]s1m
2004-04-28 09:28 am UTC (link)
It's very interesting presentation, not too many people have such a load. We handle about the same number of requests (little bit less, actually, about 600 rps) over here. Every request takes about 8 database reads and system produces a complicated report (50+ kb of logic) in response. Right now we have 2 Oracle servers (RAC) and 2 frontend servers running win2003 + our own application server software.

P.S. Well, it might be a good idea to buy a SAN box for such amount of data. It's quite expensive but it's worth it.

(Reply to this)

Create an Account
Forgot your login or password?
Login w/ OpenID
English • Español • Deutsch • Русский…