Deep value access in Elixir

As a Ruby developer that dealt a lot with parsed json, xml, and elasticsearch documents, I grew to love the functions Hash#fetch and Hash#dig for accessing deeply nested values safely. If a key is missing or one of the nested data structures doesn’t exist, you can provide a sane default with a block or at least know a runtime error won’t be thrown attempting to access a value from a null pointer.

    map = {
      a: { aa: 1 },
      b: { bb: [ 2, 3 ] },
      c: nil
    }
    
    map.fetch(:d} { puts "The key does not exist so this block is called" }
    #=> The key does not exist so this block is called
    
    map.dig(:d)
    #=> nil
    
    map.dig(:b, :bb, 1) # dig also accepts array index values
    #=> 3
    
    map.dig(:c, :cc, :ccc)
    #=> nil

This is not so simple in Elixir though it gets close with Kernel#get_in/2 and Access key + at

    map = %{
      a: %{ aa: 1 },
      b: %{ bb: [ 2, 3 ] },
      c: nil
    }
    
    # atom keys work and `nil` is returned for missing keys
    Kernel.get_in(map, [:d])
    #=> nil
    
    # indexes must be done with Access.at/1
    Kernel.get_in(map, [:b, Access.at(1)])
    #=> 3
    
    # this fn lazily calls the dot access form %{}.c
    Kernel.get_in(map, [Access.key(:c)])
    #=> nil
    
    Kernel.get_in(map, [:c, :cc, :ccc])
    #=> ** (BadMapError) expected a map, got: nil
            (elixir 1.11.2) lib/map.ex:469: Map.get(nil, :cc, nil)
            (elixir 1.11.2) lib/access.ex:459: anonymous fn/5 in Access.key/2

The failure happens because it recursively attempts to look up by the key with a guard against missing keys to return nil early. If a key’s value is nil however, it attempts to dig further into that value and blows up.

I run into that failure mode a lot. The typical scenario I find myself in is retrieving a record’s associated rows from an Ecto and safely navigating that struct to perform additional work. Under the hood of an Elixir struct is really a map with a key __struct__ set to the module name for a value like %User{} == %{__struct__: User} (and under the hood of a Map isn’t a hash-table data structure but really a hash-trie so it can be persisted but that’s a different topic).

So say we have a Orders context function and a method to pull all addresses a the purchaser of an order has. We can preload the associated records and then attempt to dig into the values.

    defmodule Orders do
      def retrieve_user_addresses(order_id) do
        MyRepo.get(Order, order_id)
        |> MyRepo.preload([user: [:addresses]])
        |> Kernel.get_in(
             [
               Access.key(:user),
               Access.key(:addresses),
               Access.all()
             ]
           )
      end
    end

This is certainly a bit contrived because it could be done with a better SQL query. But often I have already loaded this association tree from a controller into memory and I do not need to make another DB query because I have the data that I’m just processing further - you can see where this accessing attempt fails if there isn’t a user associated to the order we’ll get the BadMapError from the lookup we saw earlier as this value will be nil.

To arrive at a better solution, let’s use what I’ve termed as “thinking on the left side of the equation”. Joe Armstrong (Erlang creator) equated the equals sign in Erlang to algebra.

When you write the equation x = a + 1, you are not assigning the value of a + 1 to x. Instead you’re simply asserting that the expressions x and a +1 have the same value. If you know the value of x, you can solve for a and vice versa.

- Dave Thomas in Programming Elixir1

Typically when I want to Hash#dig in Ruby I think “ok, I want x and how do I traverse y to get there”. In Elixir there is a lot of destructuring of results and arguments. We can say “function result f(y) will look like 2x + a, now here is the handling case for 4 = f(y) and here is nil + a = f(y)” etc.

This turns out to be very good at expressing data shapes.

So here is that same example with a pattern matched case statement instead:

    defmodule Orders do
      def retrieve_user_addresses(order_id) do
        MyRepo.get(Order, order_id)
        |> MyRepo.preload([user: [:addresses]])
        |> case do
          %Order{user: nil} -> {:ok, []} # or alternatively we can return `{:error, _}`
          %Order{user: %User{addresses: addrs}} -> {:ok, addrs}
          nil -> {:error, "No order exists for id: #{order_id}"}
        end
      end
    end

You can see it is a bit clearer what field we’re extracting and what the shape of the data we’re extracting from. Additionally we’ve been pushed to confront the failure modes too - either by handling with an error or providing a sane default - and arguably it is a bit more concise than a list of keys to delve into. If the case statement gets too verbose, it’s always possibly to pull that into a separate function and pipe into it.

I’ve been experimenting with this a lot whenever it crops up in my day job, so far this is the best I’ve come up with. Hope this was helpful!