Your agent already has a training set.
Turn real conversation logs into graded trajectories, curated datasets, better policies, and reproducible evidence—without reducing a multi-turn agent to isolated prompt-response pairs.
Improvement loop
pip install stateset-agents
# Bring the conversations your agent already creates
stateset-agents ingest \
--format openai \
--input agent-logs.jsonl \
--output transcripts/
# Grade, curate, and generate the next training step
stateset-agents improve run \
--transcripts transcripts/ \
--reward customer_support \
--output improved/Grade
Every turn
Curate
JSONL
Next
Train
Learn from real conversations
Turn OpenAI-format logs and LangChain traces into first-class multi-turn trajectories instead of rebuilding your agent around a training framework.
Bring your own logs • multi-turn episodes • async-first APIs
Reward what actually matters
Combine deterministic checks, domain rules, neural scorers, multi-objective rewards, and LLM judges around the outcome you want.
Heuristic • domain • neural • LLM judge • composite
Close the learning loop
Move from curation to group-based policy optimization, reproducible benchmarks, and OpenAI-compatible serving in one framework.
Train • benchmark • publish • serve • repeat
From production conversations to a better policy
Every iteration follows the same inspectable path: ingest behavior, grade outcomes, curate signal, train the policy, and prove the result before serving it.
Start with the agent you already have.
1. Ingest
Import conversation history from common agent stacks and normalize it into trajectories with turns, metadata, rewards, and tool calls.
- Accept OpenAI chat format and LangChain conversation traces.
- Preserve full dialogue rather than flattening isolated answers.
- Use a stub backend to exercise the loop without a GPU or API key.
Learning flow
Choose the update rule without rebuilding the environment
Agent, environment, trajectory, and reward abstractions remain stable while you select the optimization method that fits the task and data.
Group Sequence Policy Optimization
GSPO
Uses sequence-level importance ratios to stabilize updates when token-level ratios become brittle over long trajectories.
Best fit
Multi-turn dialogue, long outputs, and MoE models
Framework note
The repository default for scaffolded multi-turn projects.
The hard parts connect by design
Build environments and rewards once, then reuse them across data curation, online or offline optimization, evaluation, and deployment.
Multi-turn agents
Conversation and task environments model the full episode, including context, user responses, and tool results.
Composable rewards
Mix verifiers, business rules, safety signals, neural scorers, and judge models without coupling them to one trainer.
Group-based trainers
Swap among GRPO, GSPO, GEPO, DAPO, and VAPO behind shared agent, environment, and trajectory abstractions.
Sim-to-real transfer
Use domain randomization, system identification, and progressive transfer to bridge simulated episodes to live traffic.
Reproducible evaluation
Benchmark matrices, fixed seeds, provenance manifests, plots, and publish gates turn a training run into evidence.
Train-to-serve stack
Move a checkpoint into an OpenAI-compatible FastAPI service with vLLM, metrics, Helm, and GPU deployment profiles.
The trajectory is the unit of work
A strong final answer can depend on decisions made several turns earlier. StateSet Agents preserves that causal path through training and evaluation.
Stop treating every conversation like the first one.
Use the interactions your agent already has to define rewards, find strong behavior, train deliberately, and verify that the next policy is actually better.
Multi-turn by design
Full episodes, environments, tools, and conversational credit assignment
Open framework
Python package, CLI, MCP server, trainers, benchmarks, and examples
Evidence over anecdotes
Fixed seeds, baseline comparisons, variance, provenance, and publish gates