Image credit: https://unsplash.com/photos/ZiQkhI7417A

Embarking on the Redis Odyssey: An In-Depth Codecrafters Journey with Rust— Part 1

In an era where technology’s evolution continues to outpace our ability to fully grasp its intricacies, there’s one platform that has been faithfully guiding us on our coding journey: Codecrafters. This esteemed platform, known for fostering professional development among senior programmers, has been instrumental in unearthing the layers of complexity that lie within the realm of coding. Today, we venture together on a new, transformative journey as we delve into the foundations of Redis, an in-memory data structure store, used as a database, cache, and message broker.

Redis has revolutionized the way we understand data storage, processing, and retrieval. Its lightning-fast performance, rich set of features, and high scalability make it a popular choice among programmers worldwide. However, building and working with Redis can often seem like trying to navigate a labyrinth without a compass. With the guidance of Codecrafters and our seasoned seniors, this series aims to dissect the intricacies of Redis from its very foundation.

Join us as we embark on this odyssey, using Codecrafters as our beacon, guiding us through the complexities and nuances of Redis. Together, we’ll decipher its architectural brilliance, understand its design principles, and get hands-on with its practical applications. In doing so, we hope to not only demystify Redis but also inspire you to explore new horizons in your coding journey.

Disclaimer: you can follow along with this series without a subscription, but for the best experience, I suggest subscribing for a minimal package as the development experience the company provides is pretty awesome! This content is not promoted it’s simply my experience with the platform.

Starting the journey- passing stage 1

The Software Pro's Best Kept Secret.

As the course suggests, we first need to implement a simple TCP server that for now only listens for connections and accepts them.

// src/main.rs
use std::net::TcpListener;

fn main() {
    let listener = TcpListener::bind("127.0.0.1:6379").unwrap();

    for stream in listener.incoming() {
        match stream {
            Ok(_stream) => {
                println!("accepted new connection");
            }
            Err(e) => {
                println!("error: {}", e);
            }
        }
    }
}

The snippet of code gives us a starting point. Our server accepts a connection and logs to stdout with an appropriate message.

With Codecrafters CLI installed, we can initiate tests and ensure we pass stage one.

If you want to test it yourself, we can utilize telnet to make sure the server works. ( Every example assumes you start the server by running either cargo run or using the provided binary from the project ./spawn_redis_server.sh which simply wraps cargo run )

$ ./spawn_redis_server.sh

In another terminal tab / window we can run using the nc utility:

$ nc localhost 6379

In our server output we can see:

$ ./spawn_redis_server.sh
accepted new connection

Follow along with what we have so far Stage-1 tag

The “PING-PONG” — Stage 2

For the second stage, we now get into actually responding to the socket. To do so, we need to use RESP protocol. We’ll get more in-depth about how this works later, but to respond with a simple string, we denote it with the +character.

fn main() -> std::io::Result<()> { // new line
  // ...
  match stream {
            Ok(mut stream) => {
                println!("accepted new connection");
                stream.write(b"+PONG\r\n")?; // new line
            }
}

To test this change, we can now utilize the official `redis-cli` to connect to our barebones server and see if it works as expected:

$ cat <(echo command) - | nc localhost 6379
+PONG

Or, using codecrafters cli:

$ codecrafters test

This completed stage-2 you can see it here.

Connection loop — stage 3

On stage 3, we get to the fun stuff. We need our server to run in a loop to handle multiple messages and not close. Rust allows us to achieve this using the loop language construct. To make it more readable, we will be making a function for it:

fn connection_loop(mut stream: TcpStream) -> io::Result<()> {
    let mut buffer = [0; 256];

    while let Ok(n) = stream.read(&mut buffer) {
        if n == 0 {
            break; // connection was closed
        }

        stream.write(b"+PONG\r\n")?;
    }

    Ok(())
}

By looping over .read, we ensure the socket keeps receiving messages and responds with PONG every time. Additionally:

• Ok(n) = stream.read() — shows us how many bytes we have read and if it’s 0 we close the connection (break out of the loop)
• io::Result<()> — helps us error handle any exceptions since stream.write() can fail.
• buffer[0; 256] —we chose 256 bytes arbitrarily, it pre-allocated 256 bytes for a message read, but we use the first n only later.

We can now add this to our main.rs file:

fn main() -> io::Result<()> {
    let listener = TcpListener::bind("127.0.0.1:6379").unwrap();

    for stream in listener.incoming() {
        match stream {
            Ok(stream) => {
                println!("accepted new connection");
                connection_loop(stream)?;
            }
            Err(e) => {
                println!("error: {}", e);
            }
        }
    }

    Ok(())
}

To test this step, we can now utilize redis-cli officially.

$ echo -e "ping\r\nping" | redis-cli
PONG
PONG

Summary

This insightful article takes us on an in-depth journey to the foundations of Redis using Rust, aided by the mentoring platform Codecrafters. We start by setting up a simple TCP server that listens and accepts connections, before progressing to responding to the socket using the RESP protocol. The article then guides us on handling multiple messages without closing by creating a connection loop. Practical examples and code snippets are given at each step, with suggestions for testing the changes made. The piece concludes by teasing the upcoming part 2, promising exploration of concurrent connections and the ECHO command. Don’t miss it!

If you found my article as stimulating as a double espresso, feel free to caffeinate my coding fingers by buying me a coffee on my page!