Request demo

Profile

xreduce profile loads your model, runs one prompt, and emits telemetry. Use it as a sanity check before committing to a full evaluation run.

When to use it

Verifying setup. If your credentials or model config are wrong, profile tells you in seconds. Evaluate would take minutes to hours.
First look at a new model. Before running a full benchmark, confirm the model loads, runs, and emits telemetry.
Quick sanity after config changes. Changed the model_class or switched quality_mode? Profile confirms nothing broke.

Basic usage

xreduce profile --config my-org/my-model/config.yaml

Example output:

Profiling Qwen/Qwen3-0.6B (AutoModelForCausalLM)...
   Config: my-org/my-model/config.yaml
   Prompt: Once upon a time, in a land far, far away,

Profile complete
Telemetry captured
   Run ID:        8f3a1e22-4d57-4c1a-a8e3-6f02b7e9c0a1
   Inferences:    100
   Avg latency:   142 ms
   GPU power:     78 W
   Energy/sample: 3.1 mWh

Flags

Flag	Description
`--config` / `-c` (required)	Path to `config.yaml`.
`--prompt` / `-p`	Override the profiling prompt. Priority: flag value > `profiling_prompt` in config > generic default.
`--verbose` / `-v`	Show full logging output, including the raw telemetry payload.

Using your own prompt

The default prompt is a generic completion starter. For a more representative test, override it at runtime:

xreduce profile --config my-org/my-model/config.yaml \
  --prompt "Translate this to French: The meeting starts at 3pm."

Or set a default in config.yaml so you don't have to pass it every time:

profiling_prompt: "Translate this to French: The meeting starts at 3pm."

Reading the output

Profile emits a one-page summary. The fields most worth your attention:

Field	What it tells you
Run ID	UUID to look up this run in the dashboard or pass to `compare`.
Inferences	How many forward passes were run (from `num_inferences`).
Avg latency	Mean wall-clock time per inference, in milliseconds.
GPU power	Average GPU power draw during the run, in watts.
Energy/sample	Energy used per inference, derived from power × duration. Reported in µWh / mWh / Wh depending on scale.

Run xreduce profile --verbose to also see load time, CPU power, RAM usage, activation memory, and the raw telemetry payload.

Gated models on HuggingFace

If your model weights are gated on HuggingFace (e.g. Llama, some Mistral variants), profile will fail with a 401 unless you've authenticated with HuggingFace first. Run:

hf auth login

and paste your HF token at the prompt. This is interactive and the token never enters any other shell or transcript.

Using the Claude Skill? The Skill detects gated- model 401 errors and walks you through the HF login handoff before retrying.

What's next

Run a full evaluation. Once profile works, Evaluate takes you the rest of the way with benchmark data.
Compare multiple runs. Use the run IDs in Compare.