Reinventing the wheel - raft based KV store using Go & gRPC (protobuf)
A distributed key value store based on Raft consensus algorithm. It supports:
- Raft election
- Replication & log shipping
- Log compaction & snapshots
- Consensus based writes, with batching to improve throughput
- Auto follower to leader proxy for write operations (set/del)
- gRPC based client/server & node/node communication
- Abstracted raft layer which can potentially be used with other statemachines, or different communication protocols
cd cmd/rkv
go build .
cd cmd/rkvclient
go build .This builds two executables, one in each folder: rkv (rkv server), rkvclient (rkv client)
Below cmds start a 3 node cluster on loopback. To watch logs easily, it's better to start each node in a different terminal.
./rkv -nodeid 0 -addresses localhost:27015,localhost:27016,localhost:27017
./rkv -nodeid 1 -addresses localhost:27015,localhost:27016,localhost:27017
./rkv -nodeid 2 -addresses localhost:27015,localhost:27016,localhost:27017./rkvclient set -address localhost:27015 -key somekey0 -value v0
./rkvclient set -address localhost:27016 -key sk1 -value v1
./rkvclient set -address localhost:27016 -key sk2 -value v2
./rkvclient get -address localhost:27016 -key somekey0
./rkvclient get -address localhost:27017 -key sk1
./rkvclient del -address localhost:27015 -key sk2Below benchmark was run against the leader node directly:
./rkvclient benchmark -address localhost:27016 -times 20000
=====================================================
Benchmark with 1 connection, 100 concurrent clients:
SET
Total set clients : 100
Total set calls : 20000
Total set success : 20000
Total time taken : 989 ms
Average set latency : 4 ms
GET
Total get calls : 20000
Total get success : 20000
Total time taken : 511ms
Average get latency : 2msMIT © sidecus