Your proactive agent engineer

LangSmith Engine analyzes production traces, groups related failures, and recommends fixes so your team can improve agent quality faster.

Get started
Request a demo

Helping top teams ship great agents

Use cases in production
Detect
Fix
Prevent

Automatically detect issues in production traces

Engine analyzes LangSmith traces to look for patterns in unmet user expectations or agent failures. Based on your preferences, Engine will triage high severity issues, so you can understand where there’s opportunity for improvement or poor performance that needs to be addressed.

Engine can:

  • Run automatically in the background
  • Cluster related traces into issues
  • Prioritize issues based on severity and your preferences

Engine writes the fix

For each issue, Engine summarizes the failure mode, identifies what needs to change, and writes the prompt or code fix. If you connect your codebase, Engine can open a GitHub PR with the proposed change, ready for your team to review and merge.

Engine can:

  • Write prompt and code changes based on production failures
  • Show diffs with explanations
  • Open GitHub PRs for review

Prevent issues from coming back

Like a good engineer, Engine writes tests so mistakes don’t reappear. It suggests online evaluators to track whether the issue stays resolved, and recommends examples to add to your offline eval datasets so you can catch regressions as you evolve your agent.

Engine can:

  • Suggest online evals for production monitoring
  • Recommend examples for offline experiments
  • Build a feedback loop from traces to fixes to evals

Resources for LangSmith Engine

Get started with LangSmith Engine
Introducing LangSmith Engine
How We Built LangSmith Engine | Interrupt 26

FAQs for LangSmith Engine

What model do you use for LangSmith Engine?

LangSmith Engine uses LLMs provided through LangChain. We do not support bringing your own model for Engine so that we can control the full Engine experience and deliver the best results for customers. Please read more about our subprocessors here and our notice to customers here.

What can I share with our security team on using Engine?

All model providers operate under zero data retention and are contractually prohibited from training or fine-tuning on your data.

How is Engine priced?

LangSmith Engine is a standalone agent that consumes LangChain Compute Units while it works. LCUs are a normalized unit of work that account for compute, storage, memory, and LLM usage.

Usage depends on the number of traces Engine analyzes, the depth of analysis required, and the amount of work it performs. To learn more about pricing, visit our pricing page.

Ready to add a proactive agent engineer to your team?

Start building
Get a demo