Hey everyone! We’re Christian and Eric from Frigade. Trench is open source analytics infrastructure for tracking events, identifying users, and querying data in real time.
It’s built on top of ClickHouse and Kafka for speed and scale, and it can be deployed with a single production-ready Docker image. Trench is a major part of Frigade’s infrastructure and powers our own in-product analytics and user targeting for millions of end users.
The code is MIT-licensed at https://github.com/frigadehq/trench. We just launched, and Trench gained over 1,000 GitHub stars ⭐ in less than a week.
Check out our open source demo to see how to build your own version of a Google Analytics dashboard powered by Trench in less than 15 minutes.
As we’ve scaled Frigade to millions of end users, our Postgres table we used for event tracking was quickly ballooning in cost and becoming a performance bottleneck. Many companies run into the same problem as us (e.g. Stripe, Heroku).
Engineers start by adding a basic events table to their relational database, which works at first, but breaks down as the user base scales. It's usually the biggest table in the database, the slowest one to query, and the longest one to back up. Postgres (or MySQL for that matter) simply isn't a good solution for tracking and querying events in real time at 1M+ end users scale.
We knew we wanted to move to technologies like Kafka and ClickHouse that are purpose-built for ingesting and querying thousands of events per second. When we looked for existing solutions, all the existing OSS projects we found were either bloated with unnecessary features, UIs and spaghetti code, or simply antiquated. So we built Trench.
With Trench, we've put together a single Docker image that provides a production-ready tracking event table that scales. When we migrated our tracking table from Postgres to Trench, we saw a 42% reduction in cost to serve on our primary Postgres cluster and eliminated all lag spikes from autoscaling under high traffic. We're happy with how Trench has solved event tracking for us, and now we’re excited to share that with other teams.
What can you build with Trench? There are many use cases:
We hope you’ll find Trench useful if you’re facing or have ever experienced a similar problem with analytics at scale. Here are some links to get started: