Drain Heroku access logs to Logz.io

20 May 2019 in Infrastructure

Recently, I've been looking for a way to compare the visitor numbers to an application I run to the numbers reported in Google Analytics. Our target audience are technical people who are likely blocking trackers, and I wanted to see what kind of impact that was having on our statistics compared to the numbers shown by raw access logs.

Previously I've set up an ELK stack to ingest access logs and filter/aggregate them, but for a short term comparison it wasn't worth the investment. Thankfully, people like logz.io provide hosted ELK stacks for use. I wanted to use logz.io thanks to their excellent DevRel team - go team!

Searching around I found that there is a logz.io Heroku addon, but that starts at $150/month which is more that I'd like for some ad-hoc analysis. Looking at their pricing page I saw that there is a community plan for up to 3GB of logs per day (we're nowhere near that).

It turns out that once you find the correct docs, shipping Heroku logs to a free logz.io instance is easy!

The TL;DR is this:

  1. Get your account token from your logz.io dashboard
  2. Find the listener address for your region (it's probably listener.logz.io)
  3. Add a heroku drain that points to the listener. Make sure to use https and point to port 8081.
heroku drains:add "https://listener.logz.io:8081?token=<TOKEN>" -a "<HEROKU_APP_NAME>"