{"id":240,"date":"2008-08-12T10:38:58","date_gmt":"2008-08-12T18:38:58","guid":{"rendered":"http:\/\/www.khaitan.org\/blog\/?p=240"},"modified":"2008-08-12T10:38:58","modified_gmt":"2008-08-12T18:38:58","slug":"mysql-data-sharding-using-spock-proxy","status":"publish","type":"post","link":"https:\/\/www.khaitan.org\/blog\/2008\/08\/mysql-data-sharding-using-spock-proxy\/","title":{"rendered":"MySQL data sharding using Spock Proxy"},"content":{"rendered":"<p>Yesterday at the Silicon valley MySQL Meetup, Frank of Spock.com talked about <a href=\"http:\/\/spockproxy.sourceforge.net\/\">Spock Proxy<\/a>. Spock Proxy is a fork of MySQL proxy which has been built to meet the data sharding needs of <a href=\"http:\/\/spock.com\/\">Spock.com<\/a>, the people search engine.<\/p>\n<p>Here are some highlights:<\/p>\n<ul>\n<li>Spock.com&#8217;s web interface is built on Rails and they use ActiveRecords as their O-R layer for MySQL data access<\/li>\n<li>Spock has around 1,000 web servers using Rails and they connect to MySQL slaves and masters using Spock Proxy<\/li>\n<li>Spock Proxy acts like\u00a0a normal MySQL engine, except that it transparently talks to other MySQL servers. At spock they use 4 master and 4 slaves each having their own Spock Proxy.<\/li>\n<li>The Web servers each have one connection open to the Spock Proxy while the proxy may have 100s of pooled connections<\/li>\n<li>The Proxy tokenizes a SQL statement and figures out the target shard for the query. The query must have a shard_key. The shard_key is stored in a Universal DB which stores the dictionary of the partitioned tables, shard hostname\/user\/password, ranges and range for auto_incremented columns<\/li>\n<li>It currently supports only range based partitioning &#8212; while a lot of partitioning is done based on hashing, but should not be a big deal to change<\/li>\n<li>The current alpha version is very much suited to meet Spock&#8217;s internal needs, but I&#8217;m sure people will take this up to generalize<\/li>\n<li>Unsupported query constructs (like inner queries, group by, multi-table joins) may not throw exceptions. DDLs are also not supported<\/li>\n<\/ul>\n<p>\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Yesterday at the Silicon valley MySQL Meetup, Frank of Spock.com talked about Spock Proxy. Spock Proxy is a fork of MySQL proxy which has been built to meet the data sharding needs of Spock.com, the people search engine. Here are some highlights: Spock.com&#8217;s web interface is built on Rails and they use ActiveRecords as their [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[101],"tags":[328,136,137],"_links":{"self":[{"href":"https:\/\/www.khaitan.org\/blog\/wp-json\/wp\/v2\/posts\/240"}],"collection":[{"href":"https:\/\/www.khaitan.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.khaitan.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.khaitan.org\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.khaitan.org\/blog\/wp-json\/wp\/v2\/comments?post=240"}],"version-history":[{"count":0,"href":"https:\/\/www.khaitan.org\/blog\/wp-json\/wp\/v2\/posts\/240\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.khaitan.org\/blog\/wp-json\/wp\/v2\/media?parent=240"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.khaitan.org\/blog\/wp-json\/wp\/v2\/categories?post=240"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.khaitan.org\/blog\/wp-json\/wp\/v2\/tags?post=240"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}