[Thrift] Handling failover and high availability with thrift services (oh, and load balancing too)

[Chris Lamprecht]

(This is a repost of a message I put on the facebook thrift group earlier,
but I'm moving over to this mailing list instead..easier to keep track of)

Hi,

I've been messing with thrift this week, it looks like it might be exactly
what we've been looking for -- an easy way for our developers to use or
write backend services without having to write networking code and without
the overblown heavyweight wsdl stuff available.

The only thing we need that thrift doesn't address is failover/high
availability and load balancing. I'm not suggesting that thrift add this in
-- it's so ingeniously simple, I'd hate to mess that up :) But a layer on
top of thrift services is what I'm thinking about. I have a few ideas that
I'll explore, but I am curious how others (including facebook) have handled
HA/failover/load balancing.

If I understand the TProcessor interface, it looks like one could easily
write a TProcessor that is basically a proxy, taking requests and
dispatching them to a pool of servers, keeping track of load, etc.

Another approach, which I've started already, is to write a Java servlet
bridge to thrift services (it's about 5 lines of code in a servlet), and
have the clients use HTTP. Then we can leverage apache mod_jk, which does a
decent job at load balancing/failover for Java servlets. I know HTTP isn't
as efficient as a pure streaming protocol, but with connection keep-alives
it's probably fast enough for us. A quick ping() benchmark over localhost
got about 1000 synchronous requests per second. I'll be happy to contribute
this in a patch if anyone is interested.

-chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.pub.facebook.com/pipermail/thrift/attachments/20070809/0538787b/attachment.html