Home›Companies›Ragas

Ragas

Y Combinator LogoW24
Active
artificial-intelligence
developer-tools
open-source
San Francisco
 
https://www.ragas.io

Building the open source standard for evaluating LLM Applications

The fragmented and proprietary evaluation tools today are leading to significant inefficiencies and confusion among developers. The world needs a standard everyone can rely on and that is why we are building Ragas as the open-source standard. We have 4k stars on GitHub, 1.3k members in our discord community, and over 80+ external contributors. We also have partnerships with key AI companies like Langchain, Llamaindex, Arize, Weaviate and more to help create a standard. We already process 5 million evaluations monthly for engineers from companies like AWS, Microsoft, Databricks, and Moody’s and it is growing at 70% month over month. We are building LLM application testing and evaluation infrastructure for Enterprises.
Ragas
Founded:2023
Team Size:2
Location:San Francisco
Group Partner:Nicolas Dessaigne
   

Active Founders

Jithin James, Founder

Jithin James (jjmachan), believes that life's purpose is found in the toil in mastering a craft and the fulfilment one gets from using it for the greater good. For him, this means excelling in software development, particularly in open-source and AI tooling, to empower fellow developers and help materialize their ambitious visions into reality. His journey is a testament to skill, dedication, and a commitment to community enrichment.
Jithin James
Jithin James
Ragas
  

Shahul ES, Founder

Shahul (aka ikka) discovered his fascination for AI in his sophomore year while pursuing his computer science degree. This early interest laid the foundation for his journey to becoming a Kaggle GrandMaster and led to his significant contributions to open-source AI, particularly in initiatives like OpenAssistant AI.
Shahul ES
Shahul ES
Ragas
  

Company Launches

Ragas: Open-source evaluation and testing Infrastructure for LLM applications

TL;DR

We are building Ragas — an open-source evaluation and testing infrastructure for LLM application developers to deploy their applications in production with confidence.

About us

We’re Jithin and Shahul! Having met in college, we’ve collaborated on various projects for almost a decade now.

Jithin takes care of building the software and infrastructure. He was an early employee at Bento ML, where he built and maintained tools like Bentoctl, Bentoml, and Yatai. Shahul is responsible for AI research and engineering. He is a Kaggle Grandmaster and a lead contributor to different open-source AI projects, including Open-Assistant AI.

Problem

Before 2023 software used to be written in code but with the emergence of foundational models software and applications are going to be compound systems containing code, prompts, and other components. This introduces several new problems

  • How do you select the best model or component suitable for your application from the abundance of available resources?
  • How do you test these systems and ensure continuous quality?
  • How do you derive insights from production to measure and improve your system?

As early adopters of this technology to build applications, we faced this problem while we were building RAG systems early last year.

Solution

We at Ragas make use of model-graded evaluations and testing techniques to ensure quality. This includes automated synthesis of test data points, explainable metrics, and adversarial testing.

We started by building this for RAGs, which is the most popular application of LLM as of today. Ragas is now the default open-source standard for evaluating RAG applications, processing over 4.7 million responses last month and used by engineers from enterprises like AWS, Microsoft, Databricks, Moody’s, UHG, and Tencent.

Our Ask

  • Checkout ragas on GitHub
  • If you’re building RAG applications, consider applying for the Ragas office hours program

YC Sign Photo

YC Sign Photo