Zenbase: Continuous prompt optimization from DSPy core contributors

TL;DR: Zenbase helps developers focus on programming by automating prompt engineering and model selection. We’re building developer tools and cloud infrastructure for teams to save time, never get stuck in prompt hell, and create AI apps that get smarter over time.

Hey there! We're Cyrus & Amir. In the past, we've both been lead engineers and founding CTOs. We became contributors to DSPy and discovered the future of programming with language models.

This is the story of how we came to this insight, our glimpse into the future, and 2 case studies on how Zenbase has helped companies escape prompt hell and scale prompt engineering.

Problem

Prompt “engineering” is the most time-consuming, stressful, and uncertain part of programming with LLMs. With DSPy, we had found something profound. It promised to save us from the all-too familiar user journey we — like so many others — had experienced.

DSPy kept growing. It became Stanford NLP's #1 GitHub repo with 16K stars. We started hearing of folks in Microsoft, Amazon, Google, and 40+ other companies using DSPy to prototype apps with it.

We began hearing the same things all over again. Although many found DSPy elegant and intuitive, countless folks found it impossible to grok. Those who managed to build something with it had headaches productionizing it; finding it difficult to scale, make reliable, and make performant.

So, we set out to create the productionized DSPy.

Solution

Zenbase lets you optimize your prompts and models. We offer:

1. zenbase/core is an open-source Python library that you can use to optimize your existing LLM pipelines using DSPy’s optimizers (versus having to rewrite your pipelines in DSPy)

2. A hosted API for creating AI functions that get smarter with time. We ingest user feedback to continuously optimize the prompt and model.

   We use the latest tricks from DSPy, our own custom optimizers, and fine-tuning as appropriate to execute your intents in a way that's good, fast, and at a reasonable price.

3. An on-prem API for businesses with data privacy requirements.

Use Cases

How Zenbase saved Vera from Prompt Hell

Zenbase came into the trenches with us to improve our evals from 10% to 80%. It really felt like they were a part of our team.

— Taeib, Cofounder @ Vera-Health.ai (YC S24)

They were staying up until 3am on multiple nights trying to prompt engineer their RAG query generator to retrieve the correct documents. Their progress was uncertain. It was stressful. We call this prompt hell.

Prompt engineering is the most uncertain, risky, and stressful part of programming with LLMs. There didn’t seem to be a way out, but with Zenbase, they saw the light at the end of the tunnel.

Zenbase makes prompting systematic and peaceful. We helped Vera go from demo to production, by optimizing the prompt of their query generator. With a product that could handle doctors’ stress tests, they could focus on selling, and go to bed at a good time.

How Zenbase helped Superfilter create an AI that’s personalized to its users

I’ve seen a lot of AI Devtools and Zenbase is solving a problem that everyone building with AI will have when going to production. The best part is their product is so easy to use that it’s a no brainer.

— Scott, CEO @ Superfilter.ai (YC S24)

It was all going great. Superfilter had just tested their AI email copilot with their beta users of investors and startup founders, and their users were excited. They onboarded a new cohort, and their prompts broke down. It worked well for the investors and startup founders, but not everyone.

Scott and his cofounder Travis realized that prompt engineering wasn’t going to scale to accurately categorize user emails into important, action required, or ignore.

Superfilter used our hosted API to create email categorizers that learned from users’ existing behaviour. With automatic prompt engineering, they were able to scale personalized experiences for every user.

Zenbase makes personalized AI apps easier to build and scale with automated prompt engineering.

Asks

1. Are you in prompt hell, and do you want to feel the Zen? Are you trying to scale the personalization of prompts for every account?
   
   Learn more on our website and schedule a demo with us so we can understand where you are and where you want to be. Let us be your LLM doctors and wizards as we guide you to where you want to go.
2. Use our MIT-licensed Python SDK to optimize your existing prompts with DSPy’s optimizers.
3. Kindly share this post with anyone you know who could benefit 🙏