bitpro가 원문에서 글자만 모은 내용이에요. 표·그림·수식은 빠지거나 깨질 수 있어요. 전체 16쪽 중 2쪽
a GPU is available. |
| `--torch-dtype` | `bfloat16` on Apple silicon | Set the torch dtype explicitly. In `v0.8.0`, Apple silicon defaults to `bfloat16` instead of `float16`, which keeps nearly every operation on the GPU instead of falling back to the CPU. |
| `--n-partitions` | `1` | <p>Split the model across <em>N</em> pipeline partitions to run models that don't fit on a single GPU.<br><br>Requires N ≥ 2 GPUs: otherwise, setting this on a single-GPU host will <em>fail</em>. See <a href="/tech/ree/advanced-usage#pipeline-parallelism">Pipeline Parallelism</a>.<br><br>This argument can still be passed through Extra Args when driving REE from the CLI, but in the TUI you should use the dedicated <code>Partitions</code> field.</p> |
| `--temperature` | `1.0` | Sampling temperature. Higher values produce more random output. |
| `--top-k` | `50` | Top-k sampling cutoff. |
| `--top-p` | `1.0` | Nucleus sampling threshold. |
| `--min-p` | Disabled | Min-p sampling threshold. |
| `--do-sample` / `--no-do-sample` | Enabled | Enable or disable stochastic sampling. |
| `--repetition-penalty` | `1.0` | Repetition penalty multiplier. |
| `--force-model-export` | `false` | Re-export the ONNX model even if a cached version exists. |
| `--disable-kv-cache` | `false` | Disable KV cache during generation. |
| `--short-circuit-length` | — | Generation step at which to inject the short-circuit token. |
| `--short-circuit-token` | — | Token ID to inject when short-circuiting. |
**Example:** `--operation-set reproducible --temperature 0.7 --top-p 0.9`
#### Operation Set
This flag is called `--operation-set` because it controls which set of operations the PyTorch runtime uses during inference: either standard PyTorch kernels, PyTorch's deterministic kernels, or Gensyn's RepOp kernels. It is the *most important flag* you'll set in the **Extra Args** field.
It determines what kind of reproducibility guarantees your run has, which directly affects whether your receipt can be verified by others.
There are three options:
1. **`default`**: Uses standard PyTorch kernels with no reproducibility guarantees. The same run on the same machine might produce different outputs each time. Use this when you're experimenting or testing and don't need a verifiable receipt.
2. **`deterministic`**: Uses PyTorch's built-in deterministic algorithms. Your results will be consistent across multiple runs on the same machine, but will differ if someone tries to verify your receipt on different hardware. Use this when you need repeatable results for your own work but aren't sharing receipts for third-party verification.
3. **`reproducible`:** Uses Gensyn's [RepOp](/tech/ree/advanced-usage/internals) kernels, which guarantee bitwise-identical results across any supported hardware. This is the mode you want when generating receipts that others will verify. It's slightly slower than the other modes, but it's the only one that makes your receipt truly portable.
{% hint style="success" %}
**As a rule of thumb:** if someone else will ever run `verify` on your receipt, use `reproducible`. If it's just for you, `deterministic` is fine. If you don't care about reproducibility at all, `default` is the fastest.
{% endhint %}
#### Tool Calls
The TUI does not expose tool-call configuration. For tool-call workflows, you need to use the Python SDK and `InferenceSession.complete(messages=..., tools=...)`.
{% hint style="success" %}
Use the TUI for interactive runs, validation, and verification. Use the SDK for reusable sessions and tool-augmented workflows.
{% endhint %}
#### Thinking Mode
The TUI does not expose `enable_thinking` (v0.4.0). For direct thinking on/off control, use the SDK. For CLI/TUI runs on reasoning models, [see this section.](/tech/ree/examples#short-circuiting-reasoning-models)
### Running a Generation
Once your fields are configured, press **`r`** to start the run.
The TUI switches to a progress view showing each stage of the pipeline as it completes. You'll see status updates as REE pulls the container image, prepares the model, runs inference, and assembles the receipt.
<figure><img src="https://content.gitbook.com/content/jHECdpSAZDuPfU2oZmM2/blobs/mzNaYAsdoiIxg2k9PcJf/ree_running_completing.gif" alt=""><figcaption></figcaption></figure>
If something goes wrong, the progress view will show which stage failed and the **Logs** section will contain the full output for debugging.
### Reading the Output
After a successful run, the TUI displays several populated fields:
* **REE Output:** The path to the receipt file and the model's generated text.
* **Logs:** Full pipeline output, useful for debugging or inspecting what happened during each stage.
* **Receipt path:** The location of your receipt file (e.g., `metadata/receipt_20260311_155048.json`).
#### Demonstrations
You can find a list of common workflow [examples and demonstrations here](/tech/ree/examples) along with the additional arguments and parameters you'll need to set.
### Controls Reference
| Key | Action |
| -------------------------------------------- | -------------------------------------------------------- |
| `Enter/Return` | Edit the selected field |
| `up`, `down`, `left`, `right` *(arrow keys)* | Move between fields |
| `r` | Run the generation (or re-run from results view) |
| `q` | Quit the TUI |
| `c` | Cancel a running generation |
| `e` | Reset the run state and return to the configuration form |
| `l` | View the REE license |
# Receipts
Understand what receipts prove, what they contain, and how to validate them.
### Receipts
A **receipt** is a cryptographic thumbprint of a completed REE execution. It chains together hashes of the inputs (model, prompt, configuration) and outputs (generated tokens) so that anyone can verify whether a claimed result is consistent with the claimed inputs without needing to re-execute the computation.
After every successful run, the TUI displays the path to the receipt file. Receipts are designed to be shared with others, who can then use them to verify that they can reproduce the same output using REE.
{% hint style="info" %}
We recommend sharing receipts alongside any claims about model behavior, so that others can verify those claims for themselves.
{% endhint %}
#### Receipt Compatibility
{% hint style="warning" %}
**`v0.8.0` compatibility:** Receipts from earlier REE versions may not re-verify with `v0.8.0`.
{% endhint %}
You can use `validate` to check a receipt's stored hashes. A legacy receipt can pass `validate` but fail `verify` with `v0.8.0`. `verify` re-runs inference and compares its output, but be aware that changes between releases can alter that output.
As a rule of thumb, you should verify legacy receipts with the REE version that generated them and only generate new receipts with `v0.8.0`.
#### What You’re Actually Verifying
REE guarantees reproducibility of *computation*, not the correctness of *inputs* or the correctness of the *outputs.*
A receipt proves that a given model produced a given output from a given prompt. It does not verify that the information in the prompt is accurate or that any action was taken based on the output.
#### Receipts and Tool Calls
Receipts include tool definitions when (and if) they are provided through the SDK. These are stored under `input.tools` and committed through `input.tools_hash`, which is included in the final `receipt_hash`.
Receipts do not execute tools and do not prove that an external tool returned accurate, complete, or unchanged data. If your application executes tools, you should separately record the tool call arguments, tool outputs, timestamps, provider metadata, and any external source data passed back to the model.
{% hint style="warning" %}
Tool definitions are hashed separately as `tools_hash` and are not part of `config_hash`.
{% endhint %}
#### What a Receipt Contains
| Field | Type | Description |
| ----------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `model_name` | string | Hugging Face model ID used for inference. |
| `commit_hash` | string | Specific Hugging Face model revision (git commit hash). |
| `config_hash` | string | Hash of the full task configuration (`config.json`). |
| `prompt` | string | The raw prompt text that was provided. |
| `prompt_hash` | string | Hash of the prompt. |
| `parameters` | object | Sampling and generation parameters used (temperature, top\_k, top\_p, etc.). |
| `parameters_hash` | string | Hash of the parameters object. |
| `tokens_hash` | string | Hash of the generated output tokens. |
| `token_count` | int | Number of tokens generated. |
| `finish_reason` | string | Why generation stopped: `"eos_token"` (model produced end-of-sequence naturally) or `"max_length"` (hit the max token limit). |
| `text_output` | string | The decoded text output of the generation. |
| `device_type` | string | Device used for inference (e.g., `cpu`, `cuda`). |
| `device_name` | string | Specific device identifier. |
| `receipt_hash` | string | Hash over `commit_hash`, `config_hash`, `prompt_hash`, `tools_hash`, `parameters_hash`, and `tokens_hash`. |
| `version` | string | The receipt schema version. |
| `ree_version` | string | The version of the REE SDK that generated the receipt. |
| `tools` | array/null | Tool definitions provided to the model for SDK-based tool-call workflows. Null when no tools were provided. |
| `tools_hash` | string | Hash of the canonicalized tool definitions. |
#### What a Receipt Does NOT Contain
A receipt does not include the model weights themselves, intermediate compilation artifacts, or timing/performance data.
It is a proof of *what* was computed and *what* was produced, not how fast or on which exact machine, though the receipt does include both `device_type` and `device_name` (for example, "NVIDIA A100") as informational metadata. Neither of these are part of the `receipt_hash`.
### Validating a Receipt
Validation recomputes hashes from the data stored within the receipt itself and compares them against the hash values in the receipt. This checks internal consistency, that the receipt hasn't been tampered with or corrupted, *not* that it matches anything on disk.
A valid receipt must satisfy all of the following:
* `prompt_hash` matches a recomputed hash of the prompt
* `parameters_hash` matches a recomputed hash of the parameters
* `receipt_hash` matches a recomputed hash over all receipt fields
* `tools_hash` matches a recomputed hash of the stored tool definitions (or null)
To validate a receipt. i.e., to check that all its hashes are internally consistent, switch the **Subcommand** field to `validate` and provide the path to a receipt JSON file saved locally on your device.
This would be either:
* A receipt you generated yourself
* or, a JSON file you create with the contents of someone else's receipt (e.g., by copying the JSON from someone else's receipt and saving it as a local file)
### Verifying a Receipt
Validation checks that a receipt is well-formed, but it *doesn't* prove the result is reproducible. To do that, use `verify`.
Switch the **Subcommand** to `verify` and paste the receipt path into the **Receipt Path** field, then press `r` to run.
REE will re-execute the full inference pipeline described in the receipt using the same *model, prompt* and *parameters* and compare the output against what the receipt claims. If the outputs match, the receipt is verified.
{% hint style="success" %}
This is the check you'd use when someone sends you a receipt and you want to independently confirm the result on your own hardware.
{% endhint %}
#### Receipts and Thinking Mode
When `enable_thinking` is explicitly set on `InferenceSession.complete()`, it is recorded in `input.parameters.enable_thinking` and included in `parameters_hash`.
Omitted (`None`) means the parameter is not stored.
# Supported Models
A list of models verified to work with REE.
## Verified-Compatible Models
The following models have been verified to work with REE. Any Hugging Face model compatible with the system *may* work, but this list represents models that have been explicitly tested.
If you have a specific model in mind that doesn't work with REE, you can reach out to the Gensyn team by creating an issue in the [GitHub repository](https://github.com/gensyn-ai/ree/issues) and we'll do our best to support it.
{% hint style="info" %}
Models above \~32B typically require pipeline parallelism to run.
Use the `--n-partitions` flag to split the model across multiple GPUs. See [Pipeline Parallelism](https://docs.gensyn.ai/tech/ree/advanced-usage#pipeline-parallelism) for details.
{% endhint %}
### Tool-Calling Models
REE can run many Hugging Face models, but tool-call behavior depends on the selected model.
Tool definitions are passed through the tokenizer's chat template when using `InferenceSession.complete(messages=..., tools=...)`. Models with chat templates and tool/function-calling instruction tuning are more likely to emit useful structured tool-call output.
REE SDK support *does not make every supported model equally capable of using tools.* Test your selected model against your tool schemas before relying on it in an application workflow.
#### Thinking-Capable Models
`enable_thinking` (`v0.4.0`, SDK only) applies only when the model's tokenizer chat template accepts an `enable_thinking` parameter (for example, Qwen3). Other models ignore or do not support this toggle.
### Qwen
| Model | Parameters |
| ---------------------------------- | ---------- |
| `Qwen/Qwen2.5-72B-Instruct` | 72B |
| `Qwen/Qwen3-32B` | 32B |
| `Qwen/Qwen3-8B` | 8B |
| `Qwen/Qwen3-4B` | 4B |
| `Qwen/Qwen3-1.7B` | 1.7B |
| `Qwen/Qwen3-0.6B` | 0.6B |
| `Qwen/Qwen2.5-32B-Instruct` | 32B |
| `Qwen/Qwen2.5-14B-Instruct` | 14B |
| `Qwen/Qwen2.5-7B-Instruct` | 7B |
| `Qwen/Qwen2.5-7B` | 7B |
| `Qwen/Qwen2.5-3B-Instruct` | 3B |
| `Qwen/Qwen2.5-0.5B-Instruct` | 0.5B |
| `Qwen/Qwen2.5-0.5B` | 0.5B |
| `Qwen/Qwen2.5-Coder-7B-Instruct` | 7B |
| `Qwen/Qwen2.5-Coder-0.5B-Instruct` | 0.5B |
| `Qwen/Qwen2-1.5B-Instruct` | 1.5B |
### Meta Llama
| Model | Parameters |
| ------------------------------------- | ---------- |
| `meta-llama/Llama-3.1-8B-Instruct` | 8B |
| `meta-llama/Llama-3.1-8B` | 8B |
| `meta-llama/Meta-Llama-3-8B` | 8B |
| `meta-llama/Meta-Llama-3-8B-Instruct` | 8B |
| `meta-llama/Llama-3.2-3B-Instruct` | 3B |
| `meta-llama/Llama-3.2-1B-Instruct` | 1B |
| `meta-llama/Llama-3.2-1B` | 1B |
| `meta-llama/Llama-3.1-70B-Instruct` | 70B |
| `meta-llama/Llama-3.3-70B-Instruct` | 70B |
### DeepSeek
| Model | Parameters |
| ------------------------------------------ | ---------- |
| `deepseek-ai/DeepSeek-R1-Distill-Qwen-32B` | 32B |
### Mistral
| Model | Parameters |
| ------------------------------------ | ---------- |
| `mistralai/Mistral-7B-Instruct-v0.2` | 7B |
### Code Models
| Model | Parameters |
| ---------------------------------- | ---------- |
| `codellama/CodeLlama-7b-hf` | 7B |
| `bigcode/starcoder2-3b` | 3B |
| `Qwen/Qwen2.5-Coder-7B-Instruct` | 7B |
| `Qwen/Qwen2.5-Coder-0.5B-Instruct` | 0.5B |
### Other Models
{% hint style="warning" %}
Starting in `v0.7.0`, Transformers 5.x tokenizes prompts differently for some models, including `allenai/OLMo-1B-hf` and `01-ai/Yi-1.5-6B-Chat`. Receipts for these models generated with `v0.6.0` or earlier may not re-verify with `v0.7.0` or later.
{% endhint %}
| Model | Provider | Parameters |
| ------------------------------------ | ------------ | ---------- |
| `01-ai/Yi-1.5-6B-Chat` | 01.AI | 6B |
| `llm-jp/llm-jp-3-3.7b-instruct` | LLM-JP | 3.7B |
| `TinyLlama/TinyLlama-1.1B-Chat-v1.0` | TinyLlama | 1.1B |
| `HuggingFaceTB/SmolLM-1.7B-Instruct` | Hugging Face | 1.7B |
| `allenai/OLMo-1B-hf` | Allen AI | 1B |
| `facebook/opt-125m` | Meta | 125M |
| `stabilityai/stablelm-2-1_6b` | Stability AI | 1.6B |
### Using an Unlisted Model
REE is not limited to the models above.
 `v0.6.0` improved ONNX export compatibility through PyTorch's dynamo path and supports ONNX LayerNormalization nodes without a bias input and `v0.7.0` further expanded compatibility, including xLSTM, DeepSeek-V2 MoE blocks, and nested multimodal configs. 
{% hint style="info" %}
An unlisted model may work when it exports and executes successfully.
{% endhint %}
To try an unlisted model, enter its Hugging Face model ID in the [**Model Name** field in the TUI](/tech/ree/using-the-tui) (e.g., `organization/model-name`) and run it.
# Examples
Short, practical recipes for the most common REE workflows.
## Common Workflow Examples
Try some of these ready-to-use TUI configurations for common workflows like test runs, production inference, prompt files, and more.
### Minimal: Test Model
Use a small test model to verify REE is working. 
{% hint style="info" %}
Note that test models have random weights and will produce nonsensical output. This is expected.
{% endhint %}
In the TUI, fill in the following parameters:
* **Model Name:** `hf-internal-testing/tiny-random-LlamaForCausalLM`
* **Prompt Text:** `Hello world`
* **Max New Tokens:** `24`
* Press `r` to run.
<figure><img src="https://content.gitbook.com/content/jHECdpSAZDuPfU2oZmM2/blobs/w6HvHXK2i83qu4ipUAYU/run_example.png" alt=""><figcaption></figcaption></figure>
### Production: Reproducible Inference with a Real Model
* **Model Name:** `Qwen/Qwen3-0.6B`
* **Prompt Text:** `Explain quantum entanglement in simple terms.`
* **Max New Tokens:** `256`
* **Extra Args:** `--operation-set reproducible --temperature 0.7 --top-p 0.9`
* Press `r` to run.
<figure><img src="https://content.gitbook.com/content/jHECdpSAZDuPfU2oZmM2/blobs/pBjce2jdh0zupJGDvxeo/production_inference_example.png" alt=""><figcaption></figcaption></figure>
### Using a Prompt File
Save your prompt to a .JSONL file:
```json
{"prompt": "What is 2 + 2? Show your reasoning step by step."}
```
In the TUI:
* **Model Name:** `Qwen/Qwen3-0.6B`
* **Prompt Text:** *(leave blank)*
* **Prompt File:** `/path/to/your/prompt.JSONL`
* **Max New Tokens:** `128`
* **Extra Args:** 
```bash
--operation-set reproducible
```
* Press `r` to run.
<figure><img src="https://content.gitbook.com/content/jHECdpSAZDuPfU2oZmM2/blobs/mmQFU2wZwidZrt5DeQYm/reproducible_with_prompt_file_example.png" alt=""><figcaption></figcaption></figure>
### Short-Circuiting (Reasoning Models)
Short-circuiting forces the model to exit a generation phase early by injecting a specific token at a given step. This is useful for reasoning models (e.g., Qwen3) that have thinking/end-thinking phases, where you want to limit the token budget spent on "thinking."
Both `--short-circuit-length` and `--short-circuit-token` must be provided together in **Extra Args**.
* **Model Name:** `Qwen/Qwen3-14B`
* **Prompt Text:** `Solve this math problem.`
* **Max New Tokens:** `300`
* **Extra Args:**
```bash
--operation-set reproducible --short-circuit-length 100 --short-circuit-token 151668
```
* Press `r` to run.
<figure><img src="https://content.gitbook.com/content/jHECdpSAZDuPfU2oZmM2/blobs/fXKLwUikBHXPDm8U6Mww/short_circuiting_example.png" alt=""><figcaption></figcaption></figure>
{% hint style="info" %}
**`v0.4.0`:** To turn thinking off entirely (rather than truncating it), use `enable_thinking=False` on `InferenceSession.complete(messages=...)` in the SDK. 
Short-circuiting remains useful when you want a bounded thinking budget on CLI/TUI runs.
{% endhint %}
### Large Models with Pipeline Parallelism
Split a large model across multiple GPUs using pipeline parallelism. This is the mode to use for models above \~32B parameters.
* **Model Name:** `Qwen/Qwen2.5-72B-Instruct`
* **Prompt Text:** `Summarize the key ideas behind pipeline parallelism.`
* **Max New Tokens:** `256`
* **Partitions:** `4`
* **Extra Args:** `--operation-set reproducible`
* Press `r` to run.
{% hint 프로젝트로 돌아가기
