TLDR:

We are excited to launch Haven’s fine-tuning platform for open source LLMs!

• Haven lets you fine-tune open source LLMs such as Llama or Mistral without writing code or setting up infrastructure. We charge $0.004/1k training tokens, and you get $5 in free credits after signing up
• You can host models for testing with <1s cold start times. We achieve this by running hundreds of model adapters on a single GPU and hot-swapping them based on user requests (will write more about this in a blog post)

You can get started here - alternatively, reach out to hello@haven.run or watch our demo if you want to learn more!

The Problem

Over the last months, we’ve identified two big pain points that make it hard to work with open source models:

1. Open source models work best when they are trained for specific use cases, but the fine-tuning process with existing tools is super annoying. We have found that most of our time is spent setting up infrastructure to go from finishing a training run to actually testing our models, rather than actually writing code and improving our models
2. Hosting custom models is expensive. Running a single Llama-7B model in float16 requires at minimum an A10 GPU, which costs $700+ per month. To run ten or a hundred specialized models for common tasks, this would mean that we have a monthly AWS bill of $7,000 or $70,000, respectively.

The Solution

Haven’s platform offers a super simple way to fine-tune models without managing infrastructure or writing code, and to test and run them with low costs and without any additional work.

We are able to provide a super short feedback loop of going from training to running a fine-tuned model by hosting multiple lora adapters in parallel. This makes it possible for us to host hundreds of fine-tuned models on-demand on a single GPU. For our users, this reduced model cold start times to <1s and internally, we are able to host a single fine-tuned model for a couple of dollars per month. We also enable our users to export their model weights to Huggingface, so that they can run models entirely on their terms.

Our Ask

Feel free to check out our platform and give us feedback! After signing up, you’ll receive $5 in credits to train a couple of models :) We are also happy to answer questions at hello@haven.run