Don’t ship on “vibes” alone. Measure your LLM application's
performance by testing across its development lifecycle.
LangSmith helps test application code
pre-release and while it runs in production.
A strong testing framework starts with building a reference dataset, often a tedious task. LangSmith streamlines this by letting you save debugging and production traces to datasets.
Datasets are collections of exemplary or problematic inputs and outputs that should be replicated or corrected, respectively.
.png)
When there are so many moving parts to an LLM-app, it can be hard to attribute regressions to a specific model, prompt, or other system change. LangSmith lets you track how different versions of your app stack up based on the evaluation criteria that you’ve defined.
While LangSmith has many options for automatic evaluation, sometimes you need a human touch. LangSmith speeds up the human labeler workflow significantly by supporting a feedback config and queue of traces that users can easily work through by annotating application responses with scores.
Testing needs to happen continuously for any live application. LangSmith helps you monitor not only latency, errors, and cost, but also qualitative measures to make sure your application responds effectively and meets company expectations.
Evaluation gives developers a framework to make trade-off decisions between cost, latency, and quality.
Get started with LangChain, LangSmith, and LangGraph to enhance your LLM app development, from prototype to production.
