Event sourcing: introducing rabotnik

This article develops an event sourced example application in small steps as a Ruby gem.

The last article with the promise of looking at a concrete example of how to implement an event sourced application in Ruby. In this article we'll start developing a little web application to get a first-hand look at how the concepts underlying event sourcing translate into code.

All of the code listed here is available on GitHub: https://github.com/dhamidi/rabotnik

The big picture

Event Sourcing / CQRS architecture
Event Sourcing / CQRS architecture

That's an overview about the different moving pieces in an event sourced system. We'll use this diagram throughout the article to track the progress we make through implementing the system.

Our test subject

We want to get our hands dirty coding as quickly as possible, so we'll go with the poster child for example applications, a todo/work tracker [1]. Ours will be called Rabotnik — работник means "worker" in Russian and sounds cool — and start out with very little features. In fact, it'll only have the most basic features in order to call itself a todo tracker. The user can:

In order to keep things simple for the time being "the user" is whoever visits our application's webpage. Proper user and session management will come later.

On CRUD

CRUD is not always the solution
CRUD is not always the solution

Note that the use cases outlined above can be viewed CRUD operations, but we don't call them "Read todos", "Create todo", "Update todo". Why? Well, there are two reasons:

When you build a CRUD application, you don't have any business logic to implement. You only get one piece of "business logic" for each letter of the acronym, and that "business logic" is the same in every CRUD application. This is ok [2] if you have an existing data source (an existing relational database, an Excel sheet, interfaces to a service the business uses, etc). We don't have an existing data source however, so CRUD is not a good fit.

In an event sourced application, the application keeps a log of all the state changes that have ever happened. Humans will look at this log: developers do so when working on the application, business people when they want to know how many people signed up last month but never did anything in the application. Which event log would your rather look at?

The event log for the CRUD domain:

2016-08-03 01:18:38+03:00 updated todo
2016-08-03 01:19:11+03:00 updated todo
2016-08-03 01:19:20+03:00 updated todo

or the one for the todo domain:

2016-08-03 01:20:20+03:00 captured todo
2016-08-03 01:21:21+03:00 prioritized todo
2016-08-03 01:21:38+03:00 started working on todo

The latter gives you a good overview about what has happened whereas the former is about as useful as giving all your variables single-letter names [3]

Using names from the problem domain is important to keep track of user intent and makes communication with domain experts (users!) easier.

The first test

After all this talking it's finally time to start with the coding. Because rabotnik is going to be a web application and Rack is the de-facto standard for building web applications in Ruby, it's easy to integrate it with a web server later. That's great because it means we can focus on our application code now, and take care of the HTTP plumbing later.

The "capture a todo" use case is a good starting point, because it doesn't depend on anything else existing and once we have a bunch of todos when can move on the "list todos" to usecase.

Here's a straightforward translation of that use case into a Minitest test [4]:

# test/rabotnik_test.rb
require 'test_helper'

class RabotnikTest < Minitest::Test
  def test_rabotnik_captures_a_todo_with_some_text
      app = Rabotnik::App.new
      capture_todo = Rabotnik::CaptureTodo.new(text: 'write tests')
      result = app.handle_command(capture_todo)
      assert_nil result.errors
      assert_equal(:todo_captured, result.events.first.event_name)
  end
end

There's a quite a few things going on here, the gist is that we're sending a command ("Capture a todo") to our application instance and expect the result to return no errors and a todo_captured event.

Running that test fails, as expected:

  1) Error:
  RabotnikTest#test_rabotnik_captures_a_todo_with_some_text:
  NameError: uninitialized constant Rabotnik::App
      /home/dhamidi/code/ruby/rabotnik/test/rabotnik_test.rb:5:in `test_rabotnik_captures_a_todo_with_some_text'

1 runs, 0 assertions, 0 failures, 1 errors, 0 skips

Let's add the missing constant:

# lib/rabotnik.rb
require "rabotnik/version"

module Rabotnik
  class App; end
end

Repeating this cycle a couple of times nets us this minimal implementation to make the test pass:

require "rabotnik/version"

module Rabotnik
  class App
    def handle_command(command)
      Result.new([TodoCaptured.new], nil)
    end
  end

  Result = Struct.new(:events, :errors)

  class CaptureTodo
    def initialize(text:)
    end
  end

  class TodoCaptured
    def event_name; :todo_captured; end
  end
end

This gave us our first command class — CaptureTodo — and our first event class — TodoCaptured. The event_name method on TodoCaptured seems unnecessary at the moment, as it doesn't convey any information the class name doesn't convey already. However since events are persistent, but our Ruby code can change, we avoid trouble further down the road should the Ruby class name change for any reason by not tangling up those two pieces of information. The event_name method defines the name of the event independent of the Ruby class name.

The implementation we got by satisfying the test conditions in the simplest way possible obviously doesn't work for the general case. Let's add a few more tests to make the behavior a bit more "real".

Getting closer to real behavior

I've omitted further tests so as to not blow up the size of this article too much. The bits of behavior that were missing to make this command/event complete were:

Looking at the architecture diagram again, we can see that we are halfway through introducing all the necessary components already:

Implemented parts marked green
Implemented parts marked green

Our next step will be implementing the "list all todos" use case. Here's the test:

def test_rabotnik_query_todos_lists_all_todos
  app = Rabotnik::App.new
  app.handle_command(Rabotnik::CaptureTodo.new(text: 'a todo'))
  app.handle_command(Rabotnik::CaptureTodo.new(text: 'another todo'))
  todos = app.query(:todos)

  refute_nil(todos, "No todos returned")
  assert_equal(['a todo', 'another todo'], todos.all.map(&:text))
end

Take a second to acknowledge the fact that there are zero things in this test that don't deal with the problem at hand. We don't need to worry about HTTP, the network, a database, rendering templates, etc to test our business logic. This is great, because it means our unit tests for the business logic will be fast, even when we have one thousand of them. Another advantage is that since we don't need to manage a database for running tests against our business logic, we can just focus on the actual behavior we want to test.

In order to make that test pass, we had to change the App class a bit:

class App
  def initialize
    @todos = Views::Todos.new
  end

  def handle_command(command)
    event = TodoCaptured.new(todo_id: SecureRandom.uuid, text: command.text)
    @todos.handle_event event
    Result.new([event], nil)
  end

  def query(view)
    @todos
  end
end

Our application instance now keeps track of a view of all todos. When handling commands, we pass the event resulting from the action on to the todos view. The todos view itself is pretty simple as well:

module Views
  class Todos
    Todo = Struct.new(:text)

    attr_reader :all

    def initialize
      @all = []
    end

    def handle_event(event)
      @all << Todo.new(event.text)
    end
  end
end

For every event we get, we create a new instance of a Views::Todos::Todo and add it to a list. This crude implementation has a couple of issues of course:

Since we're still at the do-the-simplest-thing-that-works stage, we'll worry about these issues later when we have pinned down these requirements with tests.

What we've skipped now it the event log: any events resulting from a command are directly fed to the view and then disappear forever.

Building an event store

Luckily an event store is not a complex thing compared to a relational database. In this early stage of development we'll just go with an in-memory implementation of an event store until we need something more robust. Persistence will be covered in a future article.

Since we will have multiple implementations of an event store and all of them share the same event store, we'll create a reusable test for the event store behavior. Luckily this is very easy with Minitest: a module containing the necessary test_ methods is enough. When testing a concrete implementation, the module is included and Minitest will run all of the included methods starting with test_.

Here's the test:

# test/rabotnik/event_store_test.rb
require 'test_helper'

module EventStoreTest
  TestEvent = Struct.new(:event_name)

  def test_version_returns_1_for_an_empty_event_store
    assert_equal(1, event_store.version)
  end

  def test_append_increments_version_by_1
    subject = event_store
    subject.append TestEvent.new(:test)
    assert_equal(2, subject.version)
  end

  def test_each_yields_to_block_for_every_event_that_has_been_appended
    subject = event_store
    subject.append TestEvent.new(:a)
    subject.append TestEvent.new(:b)

    seen = []
    subject.each { |event| seen << event }
    assert_equal([:a, :b], seen.map(&:event_name))
  end
end

So right now we can do three things with an event store:

  1. #version tells us how far the event store has advanced already. We start counting at 1 because it will make later checks for getting a subset of events easier.
  2. #append(event) adds a new event to the end of the event log.
  3. #each(&block) yields events to block in the order they have been appended.

That's all we need for a working event store. Later we'll make a few optimizations for handling subsets of events, but right now we don't need them.

Using this module to test an in-memory implementation of an event store is equally straightforward:

# test/rabotnik/in_memory_event_store_test.rb
require 'test_helper'
require 'event_store_test'

class Rabotnik::InMemoryEventStoreTest < Minitest::Test
  def event_store
    Rabotnik::InMemoryEventStore.new
  end

  include EventStoreTest
end

The implementation does the obvious:

# lib/rabotnik/in_memory_event_store.rb
module Rabotnik
  class InMemoryEventStore
    attr_reader :version
    def initialize
      @version = 1
      @events = []
    end

    def append(event)
      @events << event
      @version = @version + 1

      self
    end

    def each(&block)
      @events.each(&block)
    end
  end
end

Plugging it in

Now that we have a working event store, we need to plug it into our application so that the events are actually saved. By now you've probably guessed already that we're starting with a test for this:

def test_rabotnik_handle_command_stores_events_in_event_store
  event_store = Rabotnik::InMemoryEventStore.new
  app = Rabotnik::App.new(event_store: event_store)

  app.handle_command(Rabotnik::CaptureTodo.new(text: 'a todo'))

  seen = []
  event_store.each {|event| seen << event }
  assert_equal(['a todo'], seen.map(&:text))
end

Nothing fancy, we're injecting an event store instance into the application so that we can check the contents later, fire off a new command to the application and then check the contents of the event store.

The necessary change to the application is quite small:

# lib/rabotnik.rb
module Rabotnik
  class App
    def initialize(event_store: InMemoryEventStore.new) # added argument
      @todos = Views::Todos.new
      @event_store = event_store # added
    end

    def handle_command(command)
      event = TodoCaptured.new(todo_id: SecureRandom.uuid, text: command.text)
      @event_store.append event # added
      @todos.handle_event event
      Result.new([event], nil)
    end

The application now holds a reference to an event store and appends event to it when handling commands.

With the event store in place, we've covered all components of the architecture diagram:

Everything has been implemented
Everything has been implemented

Conclusion and outlook

In this installment we've covered all the basic architectural components of an event sourced system, using a test-driven approach to development. While all the components are in place, they are not generalized enough yet to work with more than type of one command.

We've implemented two out of the three use cases outlined at the beginning of this article. Implementing the "mark a todo as completed" use case will lead us to a couple of interesting requirements:

Once all of that is done, our application is feature complete, at least as far as the business logic is concerned. So far we did a good job at glossing over operational concerns: persistence across restarts hasn't been addressed and nobody can use our application yet, because it is not exposed via an HTTP server or CLI. These issues will be addressed in a later article as well.


1: The TodoMVC project page gives a glimpse at how often this idea can be elaborated for the purpose of showcasing code.

2: "Ok" does not mean "great". Even if Ruby on Rails (or some other framework) makes building CRUD apps very convenient.

3: Some people believe this to be viable

4: Yes, a test, not a spec, because tests can easily be reused across implementations by putting them into a module. A spec requires using define_method because it "does foo" actually translates into a method :"it does foo".