Better Phoenix APIs - featuring Ecto, Absinthe & GraphQL - Part 1

2018/10/12

At Abletech, I’ve been using Elixir full time for almost 9 months. During that time I have authored or been involved in more than 10 separate codebases, mostly using Phoenix to serve JSON APIs. I have come away from those codebases with a few opinions around how best to manage data. Full disclosure; I’m not an expert, and much of what I’ve done has been driven through conversations with talented developers in the Elixir community.

Part 1 doesn’t get into the code specifics of using Absinthe or GraphQL. Stay tuned for that

Associations, scoping, consistency and boilerplate

One of the best of ways of getting an application going quickly with Phoenix and Ecto is to use the built in phx generators. These are useful for basic CRUD, and will usually work well while you’ve got only a few schemas involved in your applications.

However, complexity increases very quickly when you add authentication, multiple assocations and scoping. In particular, when different scenarios require different associations to be preloaded and scoped.

My first low-tech solution for this is based off of the recommended usage for dataloader, a package I’ll get into more later.

Let’s take a look at some code and how it might progress naturally in a simple application. Consider listing users and their posts for a simple blog.

  1. The generated boilerplate 
@spec list_users() :: list(User)
def list_users do
  Repo.all(User)
end

This works perfectly for a simple concept.

  1. The associated schema

If we add in posts, we can simply load them for each user by preloading like so.

@spec list_users() :: list(User)
def list_users do
  User
  |> Repo.all()
  |> Repo.preload([:posts])
end

However, chances are we don’t always want to preload the posts each time we load users, so we add a second function or clause;

@spec list_users() :: list(User)
def list_users do
  Repo.all(User)
end

@spec list_users_with_posts() :: list(User)
def list_users_with_posts do
  User
  |> Repo.all()
  |> Repo.preload([:posts])
end

This works, but is not sustainable if we start adding other associations

  1. The preload list

That brings us to adding a list with the associations we want to preload.

def list_users(preloads \\ []) when is_list(preloads) do
  User
  |> Repo.all()
  |> Repo.preload(preloads)
end

Which works well until you need to do a search for users’ names. Perhaps in this scenario, you don’t want to preload associations - or only preload a subset of them.

  1. Keyword opts

First a totally naive implementation

def list_users(opts \\ []) do
  users = User
  users = if Keyword.has_key?(opts, :name) do
    name = opts[:name]

    if is_nil(name) do
      from u in users, where: is_nil(u.name)
    else
      from u in users, where: u.name == ^name
    end
  else
    users
  end
  
  users = if Keyword.has_key?(opts, :preloads) do
    preloads = opts[:preloads]
    from u in users, preload: ^preloads
  else
    users
  end
  
  Repo.all(users)
end

This is obviously not a good functional approach to the problem. Since we’re always building on the queryable depending on the next opt, we can simplify this using Enum.reduce:

def list_users(opts) do
  users = Enum.reduce(opts, User, fn
    {:name, nil}, users ->
      from u in users, where: is_nil(u.name)
    {:name, name}, users when is_binary(name) ->
      from u in users, where: u.name == name
    {:preloads, preloads}, users ->
      from u in users, preload: ^preloads
  end)
  
  Repo.all(users)
end

Bam! This is much more elegant and takes a functional approach. This isn’t something that came to me intuitively - this is a tip I received from @dpehrson on the Elixir Slack. This tip has helped pave the way to how I currently handle ecto queries. More on that later.

At some point, it’s likely we want to apply these filters and options to a query where we want only one result. Something like a get_user function. Let’s extract out all that option handling and let list_users and get_user handle their real responsibilities.

  1. query/2

Introducing the query/2 function. Totally derived from how dataloader is used - it did not make sense to me at first. However, its value is clear after going through this process once or twice.

@spec query(Ecto.Queryable.t, keyword) :: Ecto.Queryable.t
def query(queryable, opts \\ [])
def query(User, opts) do
  Enum.reduce(opts, User, fn
    {:name, nil}, users ->
      from u in users, where: is_nil(u.name)
    {:name, name}, users when is_binary(name) ->
      from u in users, where: u.name == name
    {:preloads, preloads}, users ->
      from u in users, preload: ^preloads
  end)
end

@spec list_users(keyword) :: {:ok, list(User)}
def list_users(opts \\ []) do
  users =
    User
    |> query(opts)
    |> Repo.all()
    
  {:ok, users}
end

@spec get_user(keyword) :: {:ok, User} | {:error, {:not_found, {:user, keyword}}}
def get_user(opts \\ []) do
  user =
    User
    |> query(opts)
    |> Repo.one()
    
  case user do
    %User{} = user ->
      {:ok, user}
    nil ->
      {:error, {:not_found, {:user, opts}}}
  end
end

Now we’ve factored the option handling out so that we can easily apply arbitrary filtering, pagination, scoping or otherwise in a single location. I have found this to be considerably more flexible and easy to handle, and is worth the small extra work to me as soon as one association is introduced. Not only that, but it works just fine with multiple schemas handled in a same context by pattern matching on the queryable argument in query/2

In part 2, we’ll introduce authorisation to the equation, and start discussing how Absinthe can help us solve some issues.

- Andrew