Configuration¶

This document covers configuration options for llenvs.

OpenApps runtime options¶

OpenAppsAdapter.get_environment accepts browsergym_call_timeout (default 60 seconds), browser_scale_factor (default None, native scaling), and reference_time (default None, live app time), and reward_scope (default "native", with explicit "task_local" as an alternative), and recover_observation (default False, opt-in bounded re-reading of incomplete accessibility data without repeating actions). These are adapter controls and are not forwarded as BrowserGym constructor arguments. See OpenApps for replay scope and timeout recovery rules.

CLI Configuration¶

# config.yaml
environments:
  - name: leg_counting
    adapter: reasoning_gym
    size: 100
    seed: 42
    answer_extractors:
      - type: tag_based
        config: {tag_name: answer}
      - type: pattern_answer
      - type: numeric

model:
  backend: openai
  model: gpt-4o

inference:
  temperature: 0.0
  max_tokens: 2048

system_prompt: math_reasoning          # Pre-built prompt by name
model_profile: auto                    # Detect from model name
prompt_template: math                  # Global default template

output_dir: ./results

Run with:

llenvs run config.yaml
llenvs run config.yaml --limit 10
llenvs run config.yaml --environment leg_counting

Sampling Parameters¶

from llenvs.inference import SamplingParams

params = SamplingParams(
    max_tokens=2048,        # Maximum tokens to generate
    temperature=0.0,        # Sampling temperature (0 = greedy)
    top_p=1.0,              # Nucleus sampling parameter
    top_k=0,                # Top-k sampling (0 = disabled)
    stop_sequences=(),      # Stop generation at these strings
    presence_penalty=0.0,   # Penalize token presence
    frequency_penalty=0.0,  # Penalize token frequency
    n=1,                    # Number of completions
    logprobs=False,         # Return token logprobs
    num_logprobs=5,         # How many logprobs per token
)

Backend Configuration¶

OpenAI¶

from llenvs.inference.backends import OpenAIBackend

backend = OpenAIBackend(
    model="gpt-4o",
    api_key="sk-...",         # Optional, uses OPENAI_API_KEY env var
    organization="org-...",   # Optional
    base_url=None,            # Custom endpoint
    max_concurrency=64,       # Max concurrent batch requests
)

Anthropic¶

from llenvs.inference.backends import AnthropicBackend

backend = AnthropicBackend(
    model="claude-sonnet-4-20250514",
    api_key="...",            # Optional, uses ANTHROPIC_API_KEY env var
    max_concurrency=64,       # Max concurrent batch requests
)

vLLM¶

from llenvs.inference.backends import VLLMBackend

backend = VLLMBackend(
    model_path="meta-llama/Llama-3.1-8B-Instruct",
    tensor_parallel_size=2,   # Number of GPUs
    dtype="bfloat16",         # Model dtype
    max_model_len=4096,       # Context length
    gpu_memory_utilization=0.9,
)

OpenRouter¶

from llenvs.inference.backends import OpenRouterBackend

backend = OpenRouterBackend(
    model="anthropic/claude-sonnet-4-20250514",
    api_key="...",            # Optional, uses OPENROUTER_API_KEY env var
    site_url="https://mysite.com",
    app_name="MyApp",
    max_concurrency=64,       # Max concurrent batch requests
)

LiteLLM¶

from llenvs.inference.backends import LiteLLMBackend

backend = LiteLLMBackend(
    model="gemini/gemini-2.5-flash",  # litellm provider/model format
    api_key=None,             # Optional, defaults to the provider's native env var
    api_base=None,            # Optional, e.g. a LiteLLM proxy/gateway URL
    max_concurrency=64,       # Max concurrent batch requests
    rate_limit_wait=0.0,      # Seconds to wait before rate-limit retries (0 = off)
    rate_limit_max_retries=2,
    timeout=None,             # Per-request timeout in seconds
    num_retries=None,         # litellm in-SDK transient retries (default off)
    drop_params=True,         # Drop provider-unsupported params instead of erroring
)

YAML:

model:
  backend: litellm
  model: litellm_proxy/Qwen/Qwen3.6-35B-A3B
  max_concurrency: 32
  params:
    api_base: https://your-gateway.example.com

Codex CLI¶

from llenvs.inference.backends import CodexCLIBackend

backend = CodexCLIBackend(
    model="codex-mini-latest",
    max_concurrency=4,        # Recommended: each request spawns a local CLI process
    timeout=600.0,
    profile="default",        # Optional Codex profile
    # config_overrides={...}, # Optional `codex exec -c key=value` passthrough
)

YAML:

model:
  backend: codex
  model: codex-mini-latest
  max_concurrency: 4
  params:
    timeout: 600.0
    profile: default

Notes:

Each request runs in a fresh temporary directory with codex exec --sandbox read-only --ephemeral --skip-git-repo-check.
SamplingParams.max_tokens is accepted but ignored because the Codex CLI does not expose a direct equivalent.
Use model.params.config_overrides to pass Codex-specific -c key=value overrides when your local Codex CLI/config supports them.

Backend Capabilities¶

Backend	Logprobs	Prefix Continuation	Batching	Tool Calling
vLLM	✅	✅	✅ (GPU)	✅
HuggingFace	✅	✅	✅ (GPU)	❌
OpenAI	✅	❌	✅ (concurrent)	✅
Anthropic	❌	✅ (prefill)	✅ (concurrent)	✅
OpenRouter	varies	❌	✅ (concurrent)	varies
LiteLLM	varies	❌	✅ (concurrent)	varies
Codex CLI	❌	❌	✅ (subprocess)	❌

Check programmatically:

caps = backend.capabilities
print(f"Logprobs: {caps.supports_logprobs}")
print(f"Batching: {caps.supports_batching}")
print(f"Tools: {caps.supports_function_calling}")

Environment Configuration¶

Reasoning-Gym¶

from llenvs.core.registry import environment_registry

env = environment_registry.get(
    name="leg_counting",
    adapter="reasoning_gym",
    size=100,                      # Number of samples
    seed=42,                       # Random seed
    answer_extractor=None,         # Use default TagBasedExtractor
    # Additional dataset-specific kwargs passed through
)

# Add optional extra rewards (e.g., format compliance)
from llenvs.core.reward import FormatReward
env_with_format = environment_registry.get(
    name="leg_counting",
    adapter="reasoning_gym",
    size=100,
    extra_rewards=(FormatReward(env._answer_extractor),),
)

HuggingFace¶

from llenvs.core.registry import environment_registry

env = environment_registry.get(
    name="gsm8k",
    adapter="huggingface",
    subset="main",                 # Dataset subset/config
    split="test",                  # train, test, validation
    question_column="question",    # Column with questions
    answer_column="answer",        # Column with answers
    ground_truth_extractor="numeric",  # boxed, numeric, last_line, direct
    scoring="numeric",             # exact, numeric, numeric_tolerance
    size=100,                      # Limit to N examples
    seed=42,                       # Shuffle seed
)

GEM¶

from llenvs.core.registry import environment_registry

# Basic environment
env = environment_registry.get(
    name="game:Wordle-v0",
    adapter="gem",
    max_steps=6,
)

# Tool-enabled environment
env = environment_registry.get_environment(
    name="math:GSM8K",
    adapter="gem",
    tool_types=("python",),        # Tools to enable
    max_steps=10,
    # For search tool:
    search_url="http://localhost:8000/retrieve",
    search_topk=3,
)

WebShop¶

from llenvs.core.registry import environment_registry

env = environment_registry.get(
    name="webshop:text_rich",
    adapter="webshop",
    max_steps=15,
    num_products=1000,             # None for full dataset
    human_goals=True,              # Use human-written goals
)

Prompt Configuration¶

Configure system prompts, question templates, and model profiles. See the Prompts guide for full details.

system_prompt¶

A string or list of strings. Each string is resolved by looking up in the system prompt registry, then the fragment registry, then treating as literal text.

# Pre-built prompt by name
system_prompt: math_reasoning

# Composed from fragments
system_prompt:
  - math_expert
  - think_step_by_step
  - xml_answer

# Literal string
system_prompt: "You are a helpful assistant. Think step by step."

prompt_template¶

A string referencing a registered template name (plain, math, coding, reasoning) or a literal template with a {question} placeholder. Applied to the last user message at runtime.

prompt_template: math

model_profile¶

A string referencing a registered profile name (deepseek_r1, o1, llama3_instruct, qwen_chat) or "auto" to detect from the model name.

model_profile: deepseek_r1
model_profile: auto          # Detect from model name

Per-Environment Overrides¶

system_prompt and prompt_template can be set per-environment to override the eval-level defaults:

system_prompt: general_reasoning
prompt_template: reasoning

environments:
  - name: simple_arithmetic
    # Uses eval-level defaults

  - name: polynomial_equations
    system_prompt: math_reasoning    # Override for this env
    prompt_template: math            # Override for this env

prompts¶

A dict of named prompt component overrides for multi-step environments. Keys and their meaning are environment-specific. Single-turn environments ignore this field.

environments:
  - name: webshop
    adapter: webshop
    prompts:
      instruction_prefix: "Your goal: {instruction}"
      action_hint: "Navigate using search[keywords] or click[element]."

WebShop prompt components:

Key	Default	Description
`instruction_prefix`	`Instruction: {instruction}`	Template prepended to each observation
`action_hint`	`Actions: search[keywords] or click[element]`	Available actions hint

Extraction Configuration¶

Extractor Chains¶

Configure an ordered list of extractors to try. The first extractor that succeeds is used:

environments:
  - name: polynomial_equations
    adapter: reasoning_gym
    answer_extractors:
      - type: tag_based
        config: {tag_name: answer}
      - type: boxed
      - type: pattern_answer
      - type: numeric

Each entry has a type (registry name) and optional config (kwargs passed to the extractor constructor). Available types: tag_based, regex, gsm8k, multiple_choice, boxed, numeric, last_line, code_block, pattern_answer, raw, native.

The native type uses the adapter's built-in extraction (only supported by reasoning_gym).

As a shorthand, a single extractor can be specified:

environments:
  - name: test
    answer_extractor: tag_based
    answer_extractor_config: {tag_name: answer}

Cleaning Layer¶

Pre-cleaners run on the raw response before extraction. Post-cleaners run on the extracted answer after extraction. EnvironmentFactory applies cleaning automatically.

environments:
  - name: math_task
    answer_extractors:
      - type: boxed
      - type: numeric
    # Defaults: strip_special_tokens pre-cleaner, strip_trailing_punctuation post-cleaner

  - name: code_generation
    answer_extractors:
      - type: code_block
        config: {language: python}
    pre_cleaners: [strip_special_tokens]
    post_cleaners: []  # Disable post-cleaning for code

Semantics: - Not specified (None) — use defaults (strip_special_tokens pre, strip_trailing_punctuation post) - Empty list ([]) — disable cleaning entirely - Explicit list — use exactly those cleaners

Available pre-cleaners: strip_special_tokens, strip_thinking_tokens

Available post-cleaners: strip_trailing_punctuation, strip_surrounding_quotes, strip_latex_dollars

Parameterized Cleaners¶

Cleaners that take arguments are specified as dicts with type and optional config. They can be used in both pre_cleaners and post_cleaners:

environments:
  - name: math_task
    post_cleaners:
      - strip_trailing_punctuation
      - type: truncate_tail
        config:
          max_chars: 512

Available parameterized cleaners:

Name	Config	Default	Description
`truncate_tail`	`max_chars`	256	Keep only the last N characters (strips whitespace first)

Parameterized cleaners can be mixed freely with simple string cleaner names.

Prompt Pipeline Configuration¶

from llenvs.inference.prompting import build_standard_pipeline

pipeline = build_standard_pipeline(
    system_prompt="You are a helpful assistant.",
    examples=[("Q1", "A1"), ("Q2", "A2")],  # Few-shot examples
    use_cot=True,                           # Add chain-of-thought
    answer_format="xml_answer",             # xml_answer, json, boxed, gsm8k
    tag_name="answer",                      # For xml_answer format
)

Or compose manually:

from llenvs.inference.prompting import (
    SystemPromptInjector,
    FewShotInjector,
    ChainOfThoughtWrapper,
    AnswerFormatInjector,
)

pipeline = (
    SystemPromptInjector("You are an expert.")
    >> FewShotInjector([("Q", "A")])
    >> ChainOfThoughtWrapper("think_step_by_step")
    >> AnswerFormatInjector("xml_answer", tag_name="answer")
)

Log Configuration¶

from llenvs.evaluation import LogConfig

LogConfig(
    targets=("console",),        # Log targets: "console", "file", "wandb"
    log_dir=".logs",             # Directory for JSONL file logs
    strip_images=False,          # Strip images from logged events
    wandb_run=None,              # Existing wandb.Run (skips init)
    wandb_project=None,          # W&B project name (auto-creates run)
    wandb_name=None,             # W&B run name (auto-generated if None)
    wandb_config=None,           # Extra config dict for W&B
)

Pass log=LogConfig(...) to TrajectoryRunner, SegmentedTrajectoryRunner, run_evaluation(), or run_segmented_evaluation(). See the Evaluation guide for usage examples.

CLI usage:

llenvs run config.yaml --log console,file
llenvs run config.yaml --log wandb

Evaluation Runner Configuration¶

from llenvs.evaluation import TrajectoryRunner
from llenvs.inference import TEMPLATE_REGISTRY, PROFILE_REGISTRY

# TrajectoryRunner handles both text-only and tool environments.
# It auto-detects available tools and uses generate_with_tools when tools are present.
runner = TrajectoryRunner(
    environment=env,
    backend=backend,
    sampling_params=params,
    system_prompt="...",             # System prompt string
    prompt_template=TEMPLATE_REGISTRY["math"],  # Optional template
    model_profile=PROFILE_REGISTRY["deepseek_r1"],  # Optional profile
    prompt_pipeline=pipeline,        # Optional low-level pipeline
    max_image_history=None,          # Limit images in message history (None = keep all)
)

# Run evaluation
result = runner.run_trajectory(task_index=0)
batch = runner.run_batch(
    task_indices=list(range(100)),
    progress_callback=lambda c, t: print(f"{c}/{t}"),
)

Container Configuration¶

Run any environment inside a container or subprocess by adding a container field:

environments:
  - name: sudoku
    adapter: reasoning_gym
    size: 100
    container:
      runtime: docker           # or "process"
      image: llenvs-rg:latest   # required for docker runtime
      timeout: 60
      env_vars:
        CACHE_DIR: /data/cache
      volumes:
        /host/data: /data

from llenvs.container.config import ContainerConfig

ContainerConfig(
    runtime="docker",           # "docker" or "process"
    image="llenvs-rg:latest",   # Docker image (required for docker)
    port=None,                  # Host port (None = auto-select)
    timeout=60.0,               # Startup timeout in seconds
    env_vars={},                # Environment variables
    volumes={},                 # Volume mounts (host -> container)
    docker_command="docker",    # Path to docker CLI
)

When container is set, EnvironmentFactory.create() starts the runtime and returns a ContainerEnvironment proxy. See the Containers Guide for details.

Judge Configuration¶

Add LLM-as-a-judge scoring at the eval level or per-environment. See the Judge guide for full details.

judge:
  model:
    backend: openai
    model: gpt-4o-mini
  template: correctness             # correctness, helpfulness, safety, or literal
  system_prompt: null               # Override template default
  score_range: [1, 10]              # For normalization to [0,1]
  name: judge                       # Signal name
  reward_type: outcome              # outcome, step, format, process
  weight: 1.0                       # Signal weight
  normalize: true                   # Normalize to [0,1]
  inference:                        # Judge sampling params
    temperature: 0.0
    max_tokens: 512

from llenvs.core.config import JudgeConfig, ModelConfig

JudgeConfig(
    model=ModelConfig(backend="openai", model="gpt-4o-mini"),
    template="correctness",
    system_prompt=None,
    score_range=(1.0, 10.0),
    name="judge",
    reward_type="outcome",
    weight=1.0,
    normalize=True,
    inference=None,
)

Per-env overrides eval-level. Set judge on EnvironmentConfig or EvalConfig. Supports a single JudgeConfig or a list for multiple judges.

Environment LLM Configuration¶

Configure an environment-internal LLM for environments that use an LLM in their transition function (e.g., DialogueEnvironment). See the Dialogue guide for full details.

environments:
  - name: twenty_questions
    adapter: dialogue
    params:
      words: [cat, dog, elephant]
    env_llm:
      model:
        backend: openai
        model: gpt-4o-mini
      system_prompt: ""               # Base system prompt (preset provides default)
      inference:                      # Env LLM sampling params
        temperature: 0.0
        max_tokens: 256

from llenvs.core.config import EnvironmentLLMConfig, ModelConfig, InferenceConfig

EnvironmentLLMConfig(
    model=ModelConfig(backend="openai", model="gpt-4o-mini"),
    system_prompt="",                     # Base system prompt
    inference=InferenceConfig(            # Defaults: temp=0, max_tokens=512
        temperature=0.0,
        max_tokens=256,
    ),
)

When env_llm is set, EnvironmentFactory.create() creates a ModelBackend and passes it as env_llm along with sampling_params and system_prompt to the adapter.

Branching Configuration¶

Set a branching strategy preference per environment:

environments:
  - name: webshop
    adapter: webshop
    branching_strategy: process_fork   # direct, action_replay, process_fork, or null

from llenvs.core.config import EnvironmentConfig

config = EnvironmentConfig(
    name="webshop",
    adapter="webshop",
    branching_strategy="process_fork",
)

When None (default), BranchManager.create() auto-resolves the best strategy based on environment capabilities. See the Branching Guide for details.

Iterative Configuration¶

Wrap any single-turn environment in an iterative refinement loop. See the Iterative Refinement Guide for full details.

environments:
  - name: openai/openai_humaneval
    adapter: huggingface
    iterative:
      max_turns: 5                           # Maximum refinement turns
      include_history: true                  # Include previous attempts in feedback
      summarize_history: false               # Use env LLM to summarize history
      submit_keyword: SUBMIT                 # Keyword for early termination (null to disable)
      submission_extractor: code_block       # Extractor name for parsing submissions
      submission_extractor_config:
        language: python
      solved_threshold: 1.0                  # OUTCOME score threshold for solved
      code_execution:                        # Optional code execution
        timeout: 30.0                        # Execution timeout in seconds

from llenvs.core.config import EnvironmentConfig, IterativeConfig, CodeExecutionConfig

config = EnvironmentConfig(
    name="openai/openai_humaneval",
    adapter="huggingface",
    iterative=IterativeConfig(
        max_turns=5,
        submission_extractor="code_block",
        submission_extractor_config={"language": "python"},
        code_execution=CodeExecutionConfig(timeout=30.0),
    ),
)

When iterative is set, EnvironmentFactory.create() wraps the base environment in an IterativeEnvironment. The submission_extractor is resolved from the answer extractor registry, and code_execution creates a CodeExecutionReward with a SubprocessCodeExecutor.

MCP Server Configuration¶

from llenvs.core import MCPServerConfig, MCPToolExecutor

config = MCPServerConfig(
    command="npx",                      # Command to start server
    args=("-y", "@mcp/server", "/tmp"), # Command arguments
    env={"VAR": "value"},               # Environment variables
    timeout=30.0,                       # Request timeout
)

executor = MCPToolExecutor(config)