A high write-volume MySQL user
Spinn3r crawls and indexes blogs. It says it covers 1 million blogs and 25K posts/hour, doing thousands of write transactions per second. And it does this into federated MySQL — but with a lot of software built on top. To wit:
We’ve built out a number of technologies on top of MySQL that help enable us to easily scale operations.
We’ve implemented an Open Source load balancing JDBC driver named lbpool. Lbpool allows us to loosely couple our MySQL slaves which allow us to gracefully handle system failures. It also supports load balancing, reprovisioning, slave lag, and other advanced features not available in the stock MySQL JDBC driver.
We’ve also built out a sharded database similar to infrastructure built at other companies such as Google (Adwords) and Yahoo (Flickr). Our sharded DB has a number of interesting properties including ultra high throughput requirements (we process 52TB per month), distributed sequence generation, and distributed query execution.
Please subscribe to our feed!
Comments
One Response to “A high write-volume MySQL user”
Leave a Reply
As you noted we’re going to be speaking at the MySQL users conference.
Stop by and say hi 🙂
I think we have a booth in the OSS pavilion to discuss lbpool.
Kevin