Rebuilding Redis in Go, Part 4: Transactions and a Big Refactor

Section 4 is done! It was all about Transactions. INCR, MULTI, EXEC, and DISCARD. This one started with a pile of easy lessons and then demanded a big refactor. Also, I'm up to #837 on the CodeCrafters Go leaderboard, top 1000 and climbing!

The Easy Wins

INCR came in three lessons and they were all ez pz: increment an existing key, init a missing key to 1, and send an error if the value isn't an integer. Nothing to hard, which is fine. Not every command needs to be crazy.

MULTI and EXEC started easy too. No queueing, no real execution yet, just setting up a transaction and erroring if EXEC shows up before MULTI. Another ez 2.

Then came the "empty transaction" lesson, which was marked hard, and I honestly don't get why. Nothing's queued yet, so you just return an empty array. Done.

The Refactor

Executing a transaction was a different story. Huge refactor, but not a hard one.

The problem was that every command in my codebase wrote its response straight to the connection. That works fine when each command runs immediately, but a transaction queues up multiple commands and runs them all at EXEC time. Their outputs have to go somewhere in the meantime.

So I changed their returns. Commands don't write to the connection anymore. They return a helpers.Value, and the dispatcher is the one who sends it:

type CommandFunc func(helpers.Connection, []string) (helpers.Value, error)

That one signature change rippled through every command, but each individual change was simple. This is the kind of refactor I like, makes a big difference, but pretty minimal code changes per file.

One Client's Transaction Is Not Another's

The trickier part was state. A transaction can be active for one client and not another, so I needed per-client queues. I keyed them by the client's remote address:

var (
	clientsMu sync.Mutex
	clients   = make(map[net.Addr]*clientTransaction)
)

And since I could already see multiple concurrent transactions coming, the map got a mutex from day one. Same pattern I've been using since the blpop race condition, lock before touching the shared map, unlock when done.

func AddToQueue(c helpers.Connection, execute CommandFunc, args []string) {
	clientsMu.Lock()
	defer clientsMu.Unlock()
	transaction := clients[c.RemoteAddr()]
	if transaction == nil {
		return
	}
	transaction.commands = append(transaction.commands, queuedCommand{
		execute: execute,
		args:    append([]string(nil), args...),
	})
}

DISCARD was the cleanser after all that, transaction exists? delete it, return OK. No transaction? return an error. Done.

The Last Two Lessons

Here's the part I'm happiest about, the final two lessons. EXEC's return format and multiple concurrent transactions, I finished without even looking at them. I knew EXEC would return an array of the queued results, and the concurrency approach just worked. Copied the mutex pattern from my earlier code, careful about lock and unlock around the transaction table, and it passed.

Compare that to section 2, where a map race sent me off to study mutexes for an evening. This time the same class of problem was just easy. No blocking, no condition variables, just the pattern. That's progress!!

What's Next

Next up is optimistic locking. onwards!