Parts of the evaluation harness execute model-generated Python programs. Generated code is untrusted input and may read or modify files, consume resources, spawn processes, or attempt network access.
Do not run code-generation evaluation directly on a workstation, shared server, or machine containing credentials or valuable data. Use a disposable, least-privilege sandbox or container with:
- no secrets, SSH agents, cloud credentials, or mounted personal directories;
- network access disabled unless strictly required;
- read-only inputs and a disposable writable directory;
- CPU, memory, process-count, file-size, and wall-clock limits;
- an unprivileged user and no access to the container host socket.
The evaluator's subprocesses and timeouts are reliability measures; they are
not a security boundary. Inspect commands and paths before running any
fine-tuning or evaluation job. Never place an API key in source files,
config.yaml.example, logs, commits, issues, or release archives.