hendra.dev

A work in progress

Filtering Ecto Queries

Written on

Say you have blogging app with a Post schema, and you want to be able to search for posts that are written by certain author, posts that where published after a certain date, or posts that contains a certain string.

That is pretty simple to do with Ecto. You can start with something like this:

defmodule Blog do
  def get_posts(filters) do
    Post
    |> where(author_id: filters[:author_id])
    |> where([p], p > filter[:publication_date])
    |> where([p], like(p.title, "%#{^filters[:title]}%"))
  end
end

One obvious issue with this is that if you don’t provide a value for a particular key, Ecto will pass a nil into the query, which is not what you want. You can easily work around this by pattern-matching for the value passed into the filter, and returning the query as is if it matches nil. The schema module seems like a pretty good place for this, which also gives the added benefit of abstracting away the column implementation detail for each of the filtering mechanism.

defmodule Blog.Schema.Post do
  schema "posts" do
    #.....
  end

  def by_author(query, nil), do: query
  def by_author(query, author_id) do
    query
    |> where(author_id: author_id)
  end

  def after_publication_date(query, nil), do: query
  def after_publication_date(query, date) do
    query
    |> where([p], p.publication_date > ^date)
  end

  def title_contains(query, nil), do: query
  def title_contains(query, title) do
    query
    |> where([p], like(p.title, "%#{^title}%"))
  end
end

These functions can be consumed as such:

defmodule Blog do
  def get_posts(filters) do
    Post
    |> Post.by_author(filter[:author_id])
    |> Post.after_publication_date(filter[:publication_date])
    |> Post.title_contains(filter[:title])
  end
end

So far so good, but all the match-clauses for the nil value is getting pretty repetitive. Plus, if you were to add a new filtering option, you’d have to make the change on multiple places; once to add the filter function on the schema module, and also another one on the service module/ context module to call the new filter function.

There is another way to do this. Since in Elixir a map can be easily converted into a keyword list, and since keyword lists consists of 2-elements tuples, it is a good candidate for pattern matching.

defmodule Blog.Schema.Post do
  schema "posts" do
    #.....
  end

  def filter(query, filters) when is_map(filters), do: filter(query, Map.to_list(filters))
  def filter(query, []), do: query
  def filter(query, [{:author_id, author_id} | filters]) do
    where(query, author_id: author_id)
    |> filter(filters)
  end
  def filter(query, [{:publication_date, date} | filters]) do
    where(query, [p], p.publication_date > ^date)
    |> filter(filters)
  end
  def filter(query, [{:title, title} | filters]) do
    where(query, [p], like(p.title, "%#{^title}%"))
    |> filter(filters)
  end
  def filter(query, [_ | filters]), do: filter(query, filters)
end

It can be used with simply:

defmodule Blog do
  def get_posts(filters) do
    Post
    |> Post.filter(filters)
  end
end

It accepts both map and keyword list as argument, as the function will pattern match for a map and convert it into a keyword list. That part can be omitted if you prefer to restrict the filter to a keyword list for some reason.

The filter function will go through all the keys within the filters that is passed as argument. It uses Elixir’s pattern matching feature to extract the head and the tails of the filters list. If the head matches a key that it knows how to filter, it will apply the filter to the query and call the same function recursively with the remaining items in the tail. If it run into an unknown key, it will ignore the head and call the function again with the remaining tail and the unchanged query. And finally, when it has gone through all the items in the list, it will return the query.

As you can see, it is a pretty simple technique, there is no fancy math concept or complicated algorithms involved. It is implemented with the basic features of Elixir’s pattern matching, function guard, recursive function call.

From what I’ve seen, one of the more common road-bumps in learning Elixir coming from a OOP background is figuring out where some of these concepts come to use when they can use conditionals and loops that they are already familiar with to achieve the same thing. I’m not going to attempt to explain when/how to use these concepts in this post, but I’ve heard from some of my junior team members that some of these concepts finally “clicked” in their mind when they see it being used in the example above. Hopefully it also does something for you to get a better sense of how to better use Elixir’s language features to write a more succinct, more declarative idiomatic code.