I have developed a simple-beta version of an synchronous distributed key-value database. It is simple and quite robust and I plan to make it not so-simple and open to other languages (apart from JVM based ones) in the future :). You can download it and start using from github.
In short (Refer wiki for more info): Suraj Bhirani
- It is Asynchronous
- You can have a single primary node with any number of secondary nodes which can join or leave arbitrarily.
- It is truly distributed. You just need the address of “Arbiter” using which any node running on any instance can start acting as a node.
- Insert (Key-Value pair) can only happen at primary node. Search can be done on any node.
- Order and consistency is maintained at all nodes. i.e. given enough time, all replicas settle on the same view.
- Each node has a Persistence service which takes backup of every insertion/removal. This is client dependent and can be done as desired using any SQL/NoSQL database or file for that matter.
- Tested vigorously. I believe more effort has gone in testing it than building it.
The following are the current limitations. I hope to remove them by the next version
- Currently updates are only possible on a dedicated node, the primary replica.
- The primary (leader) node cannot fail during the uptime of the system.
- Membership is handled reliably by a provided subsystem (see section “The Arbiter”). New Replicas which wish to Join should send a
Arbiter. But for a replica to leave, the current process is quite tedious by sending a manual
Replcias(Set[ActorRef])call to Primary Replica.
- The update rate is low, meaning that no incoming requests are rejected due to system overload. In case of rejecting an update the store is left in a possibly inconsistent state which may require a subsequent succeeding write to the same key to repair it.
- Clients are expected not to reuse request IDs before the request has been fully processed and responded to.
- There is an utter small chance for the database to be in in-consistent state. Please refer the section “Consistency in the case of failed replication or persistence” in the overview.
But in short “It Works!!”
- SBT as build tool
I highly recommend everyone to have a look at Akka. There are very frameworks that scale as beautifully as Akka does.
Special Thanks to the course “Principles of Reactive Programming” at coursera to help me get started.