lsfr.io

Distributed Key-Value Store Challenge #

Welcome to the distributed key-value store challenge!

In this challenge, you’ll build a distributed key-value store from scratch. You’ll start with a simple HTTP API and progressively add persistence, crash recovery, clustering, replication, and consensus mechanisms.

By the end, you’ll have built a system that can handle node failures, network partitions, and scale across multiple nodes while maintaining data consistency.

Stages #

  1. HTTP API

Build a basic in-memory key-value store with GET/PUT/DELETE operations over HTTP.

  1. Persistence

Add durability to your store. Data should survive clean shutdowns (SIGTERM).

  1. Crash Recovery

Handle unclean shutdowns (SIGKILL). Ensure data consistency after crashes.

  1. Clustering

Discover and connect to other nodes in a leader-follower arrangement.

  1. Read Replicas

Add read replicas that follow the leader. Handle eventual consistency across the cluster.

  1. Fault Tolerance

Handle leader failures and network partitions while maintaining cluster consistency.

  1. Strong Consistency

Implement strong consistency guarantees for read and write operations.

  1. Sharding

Distribute data across multiple shards for horizontal scaling.

Getting Started #

If you haven’t already, read this overview on how lsfr works and then start with stage 1 (HTTP API).

Resources #

Books #

Videos #