Every model folder has a config.yaml at its root. It tells XReduce what to load, how to load it, and how to score the outputs.
You don't write this file by hand. xreduce model create generates it for you with the right identity, auth, and task settings. This page documents what each field means so you can read it, edit it, and understand what XReduce is going to do at run time.
Two config files, two different schemas, both auto-generated. The model folder has a config.yaml at its root (covered on this page, generated by xreduce model create) and a separate benchmarks/config.yaml inside the benchmarks/ subfolder (generated by xreduce init / xreduce remap). They look similar but contain different things: config.yaml - model identity, auth, run settings, task type. Covered here. benchmarks/config.yaml - benchmark field mapping, input serializer, system prompt. Covered in Benchmark data and Field mapping. Both are auto-generated and both are safe to edit afterward - XReduce preserves your edits on subsequent regenerations.
Folder layout
my-org/my-model/
├── config.yaml ← model-root config (this page)
└── benchmarks/
├── config.yaml ← benchmark config (Benchmark data / Field mapping)
└── *.jsonl ← your test data
A complete example
model_name: "Qwen/Qwen3-0.6B"
model_id: "c05702f8-ed73-4440-b158-8c109a8df903"
account_id: "e604abb4-294d-4d75-b6c2-38d2a3a7fac1"
api_key: "ck_8d28b95a44ae08cf16a0fcc6fa88d4025c021e98f44d13d2b558697b812c96b7"
model_class: "AutoModelForCausalLM"
task_type: "completion"
num_inferences: 50
objective: "completion"
quality_mode: "balanced"
interaction_pattern: "single_shot"
structure_requirement: "unstructured"
judge_model: "openai/gpt-oss-120b"
profiling_prompt: "Once upon a time, in a land far, far away,"
Identity and auth
Field
Required
Description
model_name
Yes
The HuggingFace identifier for the model. Used to pull weights via transformers.from_pretrained().
model_id
Yes
UUID identifying this specific model in your XReduce account. Different per model.
account_id
Yes
UUID identifying your XReduce account. The same across every model in your account.
api_key
Yes
Authorizes telemetry. Shared across every model in your account. See API Keys.
Model loading
Field
Required
Description
model_class
Yes
The transformers class to instantiate. Common values: AutoModelForCausalLM, AutoModelForSequenceClassification, AutoModelForSeq2SeqLM. Use "Custom" with a loader script for models that don't load via transformers.
Task definition
Field
Required
Description
task_type
Yes
What the model does. Controls scoring logic. Supported: classify, tool_calling, qa, completion, summarization, translation. See Benchmark data for the scoring rules each one uses.
num_inferences
Yes
How many benchmark samples to process in an evaluation run. XReduce stops after this many samples even if the JSONL has more.
Evaluation profile
These four fields describe how the model is expected to behave and drive rubric selection during scoring. Together they let XReduce choose the right evaluation logic for your model and task.
Field
Required
Description
objective
Yes
What the model is optimized to produce. Example values: completion, structured_generation, classification.
quality_mode
Yes
How strict the evaluation should be. Example values: high_precision, balanced, lenient.
interaction_pattern
Yes
Whether the model is evaluated per single prompt or across multi-turn conversations. Example values: single_shot, multi_turn.
structure_requirement
Yes
How strict the expected output structure is. Example values: strict_schema, semi_structured, unstructured.
Scoring and profile
Field
Required
Description
judge_model
No
The model used for LLM-as-judge scoring on tasks that require semantic evaluation. See Understanding results for how LLM-as-judge works today and where it's going.
profiling_prompt
No
Default prompt used by xreduce profile. Can be overridden with --prompt at runtime.
What's next
Mapping non-standard JSONL fields. See Field mapping.
