Asynchronous Distributed Key-Value DB

I have developed a simple-beta version of an synchronous distributed key-value database. It is simple and quite robust and I plan to make it not so-simple and open to other languages (apart from JVM based ones) in the future :). You can download it and start using  from github.

In short (Refer wiki for more info): Suraj Bhirani

  1. It is Asynchronous
  2. You can have a single primary node with any number of secondary nodes which can join or leave arbitrarily. 
  3. It is truly distributed. You just need the address of “Arbiter” using which any node running on any instance can start acting as a node.
  4. Insert (Key-Value pair) can only happen at primary node. Search can be done on any node.
  5. Order and consistency is maintained at all nodes. i.e. given enough time, all replicas settle on the same view.
  6. Each node has a Persistence service which takes backup of every insertion/removal. This is client dependent and can be done as desired using any SQL/NoSQL database or file for that matter.
  7. Tested vigorously. I believe more effort has gone in testing it than building it.

The following are the current limitations. I hope to remove them by the next version

  1. Currently updates are only possible on a dedicated node, the primary replica.
  2. The primary (leader) node cannot fail during the uptime of the system.
  3. Membership is handled reliably by a provided subsystem (see section “The Arbiter”). New Replicas which wish to Join should send aJoin call to Arbiter. But for a replica to leave, the current process is quite tedious by sending a manual Replcias(Set[ActorRef])call to Primary Replica.
  4. The update rate is low, meaning that no incoming requests are rejected due to system overload. In case of rejecting an update the store is left in a possibly inconsistent state which may require a subsequent succeeding write to the same key to repair it.
  5. Clients are expected not to reuse request IDs before the request has been fully processed and responded to.
  6. There is an utter small chance for the database to be in in-consistent state. Please refer the section “Consistency in the case of failed replication or persistence” in the overview.

But in short “It Works!!”

Technologies:

  1. Scala
  2. Akka
  3. SBT as build tool

I highly recommend everyone to have a look at Akka. There are very frameworks that scale as beautifully as Akka does.

Special Thanks to the course “Principles of Reactive Programming” at coursera to help me get started.