Take Back Control of Your LLM ✨
Daniel Lenton, Founder
TL;DR simple, lightweight, and fully hackable observability + evals. Cut out the noise, and focus on the data + metrics that matter to you 🎯
⬇️ Clone our Open Source Framework
🫠 Problem
Current LLM infra has too many layers of abstraction, with rigid schemas and several concepts to learn. This obfuscates way too much from users, making things seem much more complex than they really are, and paradoxically it also makes these tools rigid and inflexible.
LLMs are also only a part of a broader product, and so your observability and eval infra shouldn’t be exclusively obsessed with LLM abstractions. The product and your users are what really matter, and the needs can vary wildly from product-to-product. Your observability and eval tooling should reflect that!
🧘 Not Anymore
The core building block is simple, just “unify.log”. This lets you store any kind of data to your console for easy visualization, grouping, sorting, and plotting.
You can then hack together your own custom interface, for whatever you want using three basic tile types: Tables 🔢, Views 🔍 and Plots 📊
Use these three primitives to:
➕ create + visualize your datasets in a new tab
➕ monitor and probe production traffic in a new tab, with or without LLMs
➕ start an evaluation flywheel in a new tab, with or without LLMs
📉 optimize your product for your users, with or without LLMs
🧠 whatever else you can think of!
Check out our minimal demo, explaining how to use these building blocks to ship with speed and clarity ⚡
https://youtu.be/fl9SzsoCegw?si=GKDi4TLEUXNgfWy2
Getting Started 🛠️
Sign up, check out our repo, and you're ready to go!
from unify import Unify
unify = Unify("gpt-4o@openai")
msg = "hello"
response = unify.generate(msg)
unify.log(message=msg, response=response)
# visualize at https://console.unify.ai
🙏 Our Ask
Give Unify a try, and let us know what you think! Check out our walkthrough, and if you like it, then give our repo a star 😊 If you don’t think it’s useful, please tell us. We’d love to know!