LLM / Agent Post-Training
We study post-training paradigms that turn general foundation models into robust reasoning and acting systems. The central challenge is not simply improving benchmark accuracy, but learning behaviors that remain stable and useful across long trajectories, heterogeneous tasks, tools, and environments.
Questions we care about
- How should rewards, verifiers, and process supervision be designed for long-horizon agent behavior?
- How can credit assignment remain informative when success is sparse, delayed, or only externally observable?
- What training mechanisms transfer reasoning and acting skills across environments rather than overfitting to one benchmark?
- How should post-training couple model policy, memory, tools, and harnesses?
RL Systems / Infrastructure
Agentic reinforcement learning creates systems problems that standard short-horizon RL stacks were not built for. We design infrastructure for asynchronous training, long rollouts, multi-environment execution, trajectory management, distributed inference and learning, and policy lag control.
System agenda
- Fully asynchronous agent RL where environment interaction is expensive and irregular.
- Stable training for 100+ step trajectories and heterogeneous environment runtimes.
- Unified rollout, storage, replay, evaluation, and policy update interfaces.
- Infrastructure that makes new training algorithms easy to express without rebuilding the full stack.
Recursive Self-Improvement
We study self-improvement as a system-level process rather than a one-shot parameter update. The object that evolves may include the policy, experience memory, skills, data distribution, reward model, environment curriculum, verifier, or harness itself.
Current themes
- Environment–policy co-evolution and active exploration.
- Skill and memory evolution from success and failure trajectories.
- Data–Environment–Verifier–Policy–Harness co-evolution.
- Self-optimizing discovery harnesses that revise how problems are decomposed, tested, and validated.
AI for Discovery
Most AI systems operate after a person has already specified the problem, representation, objective, tools, and evaluator. Discovery begins earlier. A discovery-oriented system must decide which unknowns matter, formulate researchable questions, construct or revise representations, form hypotheses, design interventions, and update its investigation when external evidence disagrees.
Discovery-centric model capabilities
- Problem discovery and problem formulation.
- Representation construction and revision.
- Hypothesis formation beyond a fixed search space.
- Intervention and experimentation.
- Evidence-grounded revision and continual discovery improvement.
Our long-term goal is to make discovery a first-class post-training target rather than treating it as an application wrapper around a static model.
Research principles
Mechanisms over demos. We value work that explains why a system improves, not only that it produces an impressive example.
Algorithms and systems together. Long-horizon learning often fails because training objectives and infrastructure are designed independently. We study them as one research object.
External grounding. For discovery, internal consistency is not enough. Progress must ultimately be constrained by evidence, experiments, or the world outside the model.
Open artifacts. Whenever possible, research ideas should become reusable systems, environments, models, datasets, or evaluation tools.