My long-term goal is to develop systems capable of open-ended scientific discovery. My current interests revolve around enabling agents to learn from their own experience in open-ended and complex settings. This involves two things: environments & signals.

• Scaling Data — Empowering agents to learn from experience in vast state & action spaces under long-horizon, complex tasks.
  • Can we build environments for models to gather real-world live data during training?
  • Can we intrinsically incentivize agents to perform efficient goal-directed exploration? How do we make agents learn what data to learn from"?
  • Which sets of tools maximize agent empowerment for exploration?
• Scaling Supervision — Studying and designing scalable feedback methods that provide training & sampling signals.
  • How do we construct signals for non-verifiable tasks?
  • How do we construct signals from natural language feedback? Can models learn from their own feedback?
  • How do we attribute credit to atomic actions (e.g., tokens)?
  • How do we extract informative dense signals from the strong priors LLMs/VLMs have?
  • How should we balance next-token/next-observation prediction with human-alignment objectives (e.g., task rewards or behavioral constraints)?
• Scaling Exploration & Introspection — Enabling agents to design & evolve their own data pipelines.
  • Should we focus signal design on downstream performance hoping for emergent capabilities, or on capabilities hoping for compositional skills?
  • How do we design & train models to identify their own weak/missing capabilities?
  • How do we train models to design and evolve environments & signals to fill their capability gaps?