Elixir Testing Data with Blacksmith

Home / elixir / Elixir Testing Data with Blacksmith

Elixir Testing Data with Blacksmith

In the past few weeks, you’ve seen how Elixir syntax makes a big difference when it’s time to write tests. In this article, we’re going to continue on that theme.

Business applications need business data, and tests for those applications need fake data. Make that data plausible and you’ll find debugging much easier. Creating plausible testing data is relatively simple, but often leads to repetitive, tedious code. That’s exactly the type of programming that my team works to avoid.

The Typical Approaches

Most typical test cases use one of a handful of approaches.

  • Explicit. An application can simply explicitly generate test data, leading to code that looks like this so:
Person.create first_name: "Paul", last_name: "Revere"

The downside is that such code will get awkward once you are creating enough entities with enough attributes. In short order, the data overwhelms the test.

  • Through fixtures. An application can create test data in tabular form, and then load all of that data into the application.
# person.csv
first_name, last_name
Paul, Revere

This strategy works ok, but leads to code that is hard to maintain as applications grow because foreign keys are difficult to manage.

  • Through templates. An application can use some kind of template to create fake data. This is the approach we chose. We’ll show you many examples of this type of code as we go. This approach adds complexity, and the tradeoff is not worth it for simple applications. But once your application grows in complexity, the gains you can make in managing foreign keys and simplifying the creation of test data can be huge!

 Beautiful Data for Beautiful Tests

If you’ve been following this series, you’ve seen first hand our strong desire to make the repetitive tasks we do beautiful. If I’m going to express a concept once or twice, I might be willing to live with something ugly or repetitive, but tests and the data that make them go will be expressed hundreds of times. I’m going to do everything I can to strip away the layers of tedium.

With Blacksmith, we want to be able to express these concepts with the best possible syntax:

  • Templates. The central premise of Blacksmith is that we express a template in a module called Forge. That template describes an entity. This foundation will feed the many different data forms that we need. Users can then override any of the map’s keys as needed.
  • Flexible back ends. An entity may be a map, a struct, or even a JSON map. A persistent store might be a CSV file, a relational database, or some other type of database.
  • Prototypes. Sometimes, one entity is based on another. For example, an employee may be a user with an employee number.
  • Saved and unsaved data. Sometimes, our data is database backed, and sometimes not.
  • Single items and collections. We want to be able to create a list just as easily as a single record.
  • Composite collections. Databases have primary and foreign keys, and keeping those aligned with test data is sometimes tedious.
  • Sequences to keep elements unique. Sometimes, a test scenario may need a unique email or identifier. Blacksmith makes it easy.

Let’s dig into each one of these elements in detail.

Registering templates and prototypes with Forge

You’ll register each data template in a module usually called Forge. This is a template for a Person. Notice the faker framework that we use to fake individual elements:

We’re registering each type of entity in our Forge module. The first entry registers users. The Faker library will supply plausible values for first_name and last_name. We’ll use a sequence to create an email so that each email for a new user will be unique. Blacksmith will register three different functions for creating users:

  • Forge.user( overrides \\ %{} ) will create a user, overriding any keys
  • Forge.user_list( n, overrides \\ %{}) will create a list of n users
  • Forge.saved_user( repo, overrides \\ %{}) will create a user, saved to repo
  • Forge.saved_user_list( n, repo, overrides \\ %{} ) will create a list of n users, saved to repo. 

By default, calling user and user_list creates a map, though you can change this behavior. You’ll need to customize Blacksmith to use the saved_ versions.  We’ll show you how to override that behavior later, as well as how to use the override values.

Also in the above listing, you can see an admin registration. This is defined as a :blacksmith entry, which means the entry will take all of the default values from the specified :prototype and either overriding or copying the supplied values. For admins, Blacksmith will create a new user with a roles field set to [“admin”].

Instantiating Template Entries

Now, you can create new users or admins. Just use your Forge module and the macros that Blacksmith creates for you, like this:

 user = Forge.user

Or, you can create a list of five users, like this:

user = Forge.user_list 5

You can override any of the registered attributes, like this:

user = Forge.user first_name: "Will", last_name: "Override"

Simple and powerful. Now, your tests can quickly specify create new entities with many elements, but overriding or creating new attributes as needed.

Mutual Attributes and Having

You can also create several entities together with a mutual list of attributes. Let’s say we want to create a few users, all sharing the same country_id attribute. We could use the having macro, like this:

Forge.having country_id: usa_id do
  user = Forge.user
  admin = Forge.admin

Now, both user and admin will have a country_id equal to usa_id.  This advantage is small here, but will grow significantly with the number of attributes and entities that you need to manage. For example, in our tests, we have many questions and choices, all which have the same survey_id and company_id.

What if we need to create some other type of entity, like say an Elixir struct?

Creating Structs

With a little knowledge of Elixir, you can easily have Blacksmith create structs for you, too. Since an Elixir struct is just a map with the extra __struct__ key, just adding the key

__struct__: User

to the existing User record would create users that satisfy a struct called User.

Creating Custom Entities

You can create something other than maps or structs. To do so, you just need to tell Blacksmith how. Elixir has a JSON library called Poison. Let’s create a forge that tells Blacksmith how to create a new JSON hash from an Elixir map, like this:

You can see a Forge with two specific enhancements. The first is the attribute variable, @new_function. Blacksmith creates maps, and that function tells Blacksmith how to create a JSON hash from a map.

Second, you can see the function new_json defined in Blacksmith.Config.  That function simply takes the attributes (the ones generated by Blacksmith), merges in the overrides, and then calls Poison.Encoder.Encode on the result. The result is that each new entity will create a new Json entity.

Customizing Persistence

If you want to create persistent entities, you need to show Blacksmith how to save single entities, and lists of entities. For example, you might want to save records using Ecto, a persistence engine for Elixir. Here’s how you’d go about it. Let’s say you’re starting with an Ecto schema, like this:

We’re creating a simple database-backed entity. The database schema and model will both have corresponding fields.

Next, we’d create a forge. Our forge would specify the User struct for Ecto, as well as a couple of functions that Ecto could use to save one or many entities, like this:

Now, all that remains is to create a couple of functions to show Blacksmith how to persist Ecto data:

Now, if you have a repo called UserRepo, you can create one or many rows with

user = Forge.saved_user UserRepo
party = Forge.saved_user_list 5, UserRepo

It’s that easy.

Wrapping Up

If we had to create one or two database records for our tests, or if all of our entities had a handful of fields, we would probably opt for a different approach. We’d simply create them explicitly by hand. Similarly, if we had to create one database with complex relationships, we’d just use our software to create the database and save that data as fixtures.

Unfortunately, when you’re testing business software, neither of these conditions are true. We have to create plausible data for each one of the thousands of test cases we run. With this simple little strategy for creating flat entities and lists, saved or not, we can take the tedium out of running tests.


Bruce Tate
Bruce Tate
Bruce Tate is the CTO of icanmakeitbetter.com, and author of more than 10 books, including the best-selling Seven Languages in Seven Weeks.
Recent Posts
Showing 3 comments
  • Peter Marreck

    How about not repeatedly testing the connection between your Elixir code and your database, by coding your models and your tests in a way that avoids hitting up the database at all, in a test context? (Or perhaps just once… just to verify it’s there.) Fast unit test vs. slow integration test… and a test suite should be 95% unit tests, if not more.

    Having worked in large codebases with large test suites, that’s one of the changes I would make right off the bat to keep things from getting slow down the road. And trust me, you WANT fast tests… Always. Quicker code validation turnaround = more productive and happier coders = happier managers = better product.

    What you DON’T want is what I’ve had to work with… A 45 minute long test suite after 15 or 20 man-years of development effort creating thousands of lines of code.

    And a valid unit test that doesn’t mutate the DB has other benefits… such as easy parallelization. You can’t have 2 test suites running at once which are both banging mutable CRUD on the same tables of the DB… Recipe for nondeterministic disaster! Back to that speed thing, again!

    We already went down this path with Rails, I’d prefer if Elixir (Phoenix, etc.) was a little smarter about it right off the bat and started a convention that didn’t assume that every test of business logic had to actually hit the DB.

    I know people are still mainly just using Elixir for small pet projects where the importance of this isn’t really apparent (yet). But trust me, it will be…

  • Bruce Tate


    Part of what I can do with Blacksmith is to create data that’s saved or unsaved, and any format that you want. In fact, that’s one of the reasons we built Blacksmith.

    Forge.user_list 5, roles: [“admin”]

    will give me a list of five users, unsaved, with the roles set to [“admin”]. One trivial change and the user is database backed. This interface is ideal for testing with and without database backing, and flipping back and forth between the two. Our tests right now run sub second, and we aim to keep them that way.

    But for the tests that have to be database backed, it’s important to be able to do that with concurrency. Blacksmith has the tools that let me do that too so that you don’t have the latency that’s usually associated with database backed tests. I can include keys that are guaranteed to be absolutely unique, so that they don’t collide with other database tests.

    Thanks for your comment.

  • Paul Smith

    Bruce, just wanted to say thank you for writing this library. We’re using it on a project now and so far it’s been great. Keep up the good work.

Leave a Comment

Contact Us

We're not around right now. But you can send us an email and we'll get back to you as soon as possible!

WordPress Lightbox Plugin