Request demo

Initialize a project

xreduce init scaffolds a new model folder with config templates. Run it once per model you want to evaluate.

Basic usage

xreduce init my-org/my-model

This creates two files:

my-org/my-model/
├── config.yaml                    # model identity + run settings
└── benchmarks/
    └── config.yaml                # field mapping + task-default system prompt

Both files start with sensible defaults that you'll refine in the next steps: Register a model (populates config.yaml) and adding a benchmark JSONL to benchmarks/.

Auto-inferred field mapping

If you've already placed a benchmark.jsonl or any .jsonl file inside benchmarks/ before running init, XReduce inspects the first line and writes the correct field_map automatically. It recognizes common field-name pairs:

Input field	Output field	Behavior
`input`	`expected_output`	SDK default - no `benchmarks/config.yaml` written
`question`	`answer`	field_map written
`prompt`	`completion`	field_map written
`prompt`	`sql`	field_map written
`input`	`sql`	field_map written
`input`	`output`	field_map written
`text`	`label`	field_map written
`query`	`target`	field_map written

If your JSONL has structured input (a JSON object or array per record instead of a string), input_serializer: json_to_string is also written automatically.

Auto-generated system prompt by task type

When the model's config.yaml has a task_type set, init looks it up in the task-defaults registry and writes a sensible system_prompt into benchmarks/config.yaml. The prompt is task-appropriate and can be edited freely.

Built-in task types and their default prompts:

`task_type`	Default `system_prompt`
`text-to-sql`	`You are a SQL generation engine. Return only valid SQL. Do not explain. Do not use markdown. Do not wrap SQL in code fences.`
`classification`	`You are a classification engine. Return only the predicted label. Do not explain.`
`function-calling`	`You are a function-calling engine. Return only valid JSON matching the required function schema. Do not explain.`
`text-generation`	`You are a text generation engine. Produce clear, concise output.`
`summarization`	`You are a summarization engine. Return a concise summary of the input.`
`qa`	`You are a question answering system. Answer the question directly and concisely.`

After init runs, benchmarks/config.yaml looks something like this for a text-to-sql model:

field_map:
  input: prompt
  expected_output: sql

system_prompt: |
  You are a SQL generation engine.
  Return only valid SQL.
  Do not explain.
  Do not use markdown.
  Do not wrap SQL in code fences.

Edit system_prompt freely. It's a normal YAML string - any prompt that works for your benchmark is fair game. If you change task_type later, run xreduce remap to regenerate the default system prompt for the new task type (your other edits in benchmarks/config.yaml are preserved).

Flags

Flag	Description
`path` (positional)	Target folder path, e.g. `model-examples/distil-labs/text2sql-qwen3-4b`.
`--force` / `-f`	Overwrite existing `config.yaml` and `benchmarks/config.yaml`.
`--with-system-prompt`	Override the task-default and write `system_prompt: "<FILL IN>"` as a placeholder for you to edit. Mutually exclusive with `--system-prompt`.
`--system-prompt "TEXT"`	Override the task-default and write `system_prompt: "TEXT"` directly. Mutually exclusive with `--with-system-prompt`.

Next steps

Populate config.yaml by running xreduce model create (see Register a model), or by editing the placeholder values manually.
Add a benchmark JSONL to benchmarks/ (see Benchmark data).
Run xreduce remap if you added the JSONL after init, to regenerate the field mapping and pick up the task-default system prompt.