Skip to content

update new ARCHITECTURE docs #1655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
xwings merged 1 commit into from
Sep 1, 2026
Merged

update new ARCHITECTURE docs #1655

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
317 changes: 249 additions & 68 deletions ARCHITECTURE.md
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions ARCHITECTURE/arch.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ covered by `tests/test_cpu_models.py`.
| `qiling/arch/models.py` | CPU model enums (`X86_CPU_MODEL` … `RISCV64_CPU_MODEL`) |
| `qiling/arch/msr.py`, `cpr.py`, `cpr64.py` | x86 MSRs, ARM/ARM64 coprocessor registers |
| `qiling/arch/utils.py` | `QlArchUtils`: disassembly output for verbose/trace modes |
| `qiling/cc/__init__.py` + `intel.py`, `arm.py`, `mips.py`, `ppc.py`, `riscv.py` | Calling conventions (arg/retval marshalling) consumed by `os/fcall.py` |
| `qiling/cc/__init__.py` + `intel.py`, `arm.py`, `mips.py`, `ppc.py`, `riscv.py` | Calling conventions (arg/retval marshalling) consumed by `qiling/os/fcall.py` |

## Key Types and Entry Points

Expand All @@ -35,7 +35,7 @@ covered by `tests/test_cpu_models.py`.
- `qiling/arch/x86.py:22,53,79,111` - `QlArchIntel` / `QlArchA8086` / `QlArchX86` / `QlArchX8664`.
- `qiling/arch/cortex_m.py:67` - `QlArchCORTEX_M(QlArchARM)` - plus `QlInterruptContext` (`:25`) for exception entry/exit in MCU mode.
- `qiling/arch/models.py` - CPU model enums selected via the `cputype` kwarg (resolved in `select_arch`, `qiling/utils.py:376`).
- `qiling/cc/__init__.py:9` - `QlCC` - abstract calling convention; `QlCommonBaseCC` (`:104`); e.g. `cc/intel.py` defines `cdecl`/`stdcall`/`ms64`/`macosx64`.
- `qiling/cc/__init__.py:9` - `QlCC` - abstract calling convention; `QlCommonBaseCC` (`:110`); e.g. `qiling/cc/intel.py` defines `cdecl`/`stdcall`/`ms64`/`macosx64`.

## Interactions

Expand Down
8 changes: 6 additions & 2 deletions ARCHITECTURE/debugger.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ reverse debugging. Mature released infrastructure; maturity-based status.
| File | Role |
| ---- | ---- |
| `qiling/debugger/debugger.py` | Base `QlDebugger` |
| `qiling/debugger/gdb/gdb.py` | `QlGdb`: GDB remote-serial-protocol server |
| `qiling/debugger/gdb/gdb.py` | `QlGdb`: GDB remote-serial-protocol server, plus the `GdbSerialConn` transport |
| `qiling/debugger/gdb/utils.py` | `QlGdbUtils`: breakpoint table and the per-instruction `dbg_hook` that services breakpoints, stepping, and async interrupts |
| `qiling/debugger/gdb/xmlregs.py`, `gdb/xml/` | Target-description XML per arch for modern GDB clients |
| `qiling/debugger/qdb/qdb.py` | `QlQdb`: interactive Cmd-based debugger |
| `qiling/debugger/qdb/arch/` | Per-arch Qdb support (arm, intel, mips) |
Expand All @@ -27,7 +28,10 @@ reverse debugging. Mature released infrastructure; maturity-based status.
## Key Types and Entry Points

- `qiling/debugger/debugger.py:13` - `QlDebugger` - base; `run()` starts the session.
- `qiling/debugger/gdb/gdb.py:68` - `QlGdb(QlDebugger)` - listens on ip:port, translates RSP packets to Qiling hook/mem/reg operations.
- `qiling/debugger/gdb/gdb.py:84` - `QlGdb(QlDebugger)` - listens on ip:port, translates RSP packets to Qiling hook/mem/reg operations; `run()` (`:139`) serves the session.
- `qiling/debugger/gdb/gdb.py:817` - `GdbSerialConn` - the socket transport; `poll_interrupt()` (`:856`) is a non-blocking check for a client `\x03`, wired into `QlGdbUtils.check_interrupt` (`qiling/debugger/gdb/gdb.py:145`) so a running guest can be broken into asynchronously.
- `qiling/debugger/gdb/utils.py:16` - `QlGdbUtils` - `dbg_hook` (`:48`) runs per instruction to service breakpoints, single-step, and interrupts; `bp_insert`/`bp_remove` (`:81`/`:94`); `resume_emu` (`:107`).
- Stop replies report `SIGTRAP` (`qiling/debugger/gdb/gdb.py:49`) for both single-step (`:243`) and async-interrupt stops (`:262`).
- `qiling/debugger/qdb/qdb.py:59` - `QlQdb(Cmd, QlDebugger)` - CLI loop; `rr` mode enables record/replay reverse debugging.
- Activation: set `ql.debugger = True | "gdb" | "gdb:0.0.0.0:9999" | "qdb" | "qdb:rr"` (`qiling/core.py:437`); instantiated lazily in `Qiling.run` via `select_debugger` (`qiling/utils.py:332`).
- `qltool` flags: `--gdb` and `--qdb` (see [cli.md](cli.md)).
Expand Down
8 changes: 4 additions & 4 deletions ARCHITECTURE/hw.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ firmware images (UART echo, freertos, blink, crc, dma_clock, i2c/spi/lcd).

## Key Types and Entry Points

- `qiling/hw/hw.py:14` - `QlHwManager` - available as `ql.hw`; created by core only when `ql.baremetal` (`qiling/core.py:191`).
- `qiling/hw/hw.py:23` - `QlHwManager.create(label, struct, base)` - instantiates a peripheral from the profile entry and maps its MMIO region.
- `qiling/hw/hw.py:82` - `QlHwManager.step()` - advances every peripheral one tick; called from the MCU run loop.
- `qiling/hw/hw.py:60` - `QlHwManager` - available as `ql.hw`; created by core only when `ql.baremetal` (`qiling/core.py:191`).
- `qiling/hw/hw.py:67` - `QlHwManager.create(label, struct, base)` - instantiates a peripheral from the profile entry and maps its MMIO region.
- `qiling/hw/hw.py:135` - `QlHwManager.step()` - advances every peripheral one tick; called from the MCU run loop.
- `qiling/hw/peripheral.py:132` - `QlPeripheral(QlPeripheralUtils)` - base class: a ctypes register struct + read/write handlers on the MMIO region.
- `qiling/extensions/mcu/stm32f4xx/stm32f407.py` (and siblings) - chip `env` dicts consumed at `Qiling(..., env=...)` construction.
- `qiling/extensions/mcu/stm32f4/stm32f407.py` (and siblings) - chip `env` dicts consumed at `Qiling(..., env=...)` construction.

## Interactions

Expand Down
2 changes: 1 addition & 1 deletion ARCHITECTURE/loader.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Test command proves the ELF path end-to-end.

- `qiling/loader/loader.py:21` - `QlLoader` - base; `Image` NamedTuple (`:15`), abstract `run()` (`:62`), `skip_exit_check` (`:27`).
- `qiling/loader/elf.py:68` - `QlLoaderELF` - loads binary + interpreter, builds stack/auxv (`AUXV` enum `:32`), sets `entry_point`/`elf_entry`.
- `qiling/loader/pe.py:666` - `QlLoaderPE(QlLoader, Process)` - `Process` (`:69`) builds PEB/TEB/LDR; `QlPeCache` (`:41`) caches parsed DLLs behind the `libcache` kwarg (wired in `select_loader`, `qiling/utils.py:300`).
- `qiling/loader/pe.py:817` - `QlLoaderPE(QlLoader, Process)` - `Process` (`:77`) builds PEB/TEB/LDR; `QlPeCache` (`:49`) caches parsed DLLs behind the `libcache` kwarg (wired in `select_loader`, `qiling/utils.py:300`).
- `qiling/loader/pe_uefi.py:26` - `QlLoaderPE_UEFI` - loads DXE/SMM modules and installs protocols into the UEFI context.
- `qiling/loader/macho.py:70` - `QlLoaderMACHO`.
- `qiling/loader/mcu.py:57` - `QlLoaderMCU` - with `IhexParser` (`:15`).
Expand Down
21 changes: 15 additions & 6 deletions ARCHITECTURE/os-baremetal.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ released infrastructure; maturity-based status.
## Status

`done` — MCU covered by `tests/test_mcu.py` (STM32F1/F4, GD32VF1 firmware),
BLOB by `tests/test_blob.py` (u-boot) and `tests/test_edl.py`.
BLOB by `test_blob.BlobTest.test_uboot_arm` and `tests/test_edl.py`. The other
case in `tests/test_blob.py` is blocked on a missing fixture (see Open Gaps).

## Code Structure

Expand All @@ -23,10 +24,10 @@ BLOB by `tests/test_blob.py` (u-boot) and `tests/test_edl.py`.

## Key Types and Entry Points

- `qiling/os/mcu/mcu.py:37` - `QlOsMcu(QlOs)` - run loop; steps hardware between execution chunks and delivers interrupts.
- `qiling/os/mcu/mcu.py:41` - `QlOsMcu(QlOs)` - run loop; steps hardware between execution chunks and delivers interrupts.
- `qiling/os/mcu/mcu.py:17` - `MCUTask(UnicornTask)` - the firmware execution task.
- `qiling/extensions/multitask.py:26` / `:152` - `UnicornTask` / `MultiTaskUnicorn(Uc)` - task-switching Unicorn subclass MCU mode runs on.
- `qiling/os/blob/blob.py:12` - `QlOsBlob(QlOs)` - runs `entry_point` → `exit_point` with no OS services.
- `qiling/os/blob/blob.py:14` - `QlOsBlob(QlOs)` - runs `entry_point` → `exit_point` with no OS services.
- MCU selection: `QL_OS.MCU` is the only member of `QL_OS_BAREMETAL` (`qiling/const.py:74`); `ql.baremetal` (`qiling/core.py:357`) gates hardware-manager creation.

## Interactions
Expand All @@ -40,13 +41,21 @@ BLOB by `tests/test_blob.py` (u-boot) and `tests/test_edl.py`.
## How to Test

```sh
cd tests && python3 test_blob.py # pass = unittest "OK", exit 0
cd tests && python3 -m unittest test_blob.BlobTest.test_uboot_arm # pass = "OK", exit 0
```

- MCU (also proves [hw.md](hw.md)): `cd tests && python3 test_mcu.py`.
- Qualcomm EDL loader: `cd tests && python3 test_edl.py`.
- MCU (also proves [hw.md](hw.md)): `cd tests && python3 test_mcu.py` — pass = `Ran 18 tests … OK`.
- Qualcomm EDL loader: `cd tests && python3 test_edl.py` — pass = `Ran 1 test … OK`.
- The whole `test_blob.py` file does **not** pass from a clean checkout; see
Open Gaps.

## Open Gaps / Roadmap

- Supported chip families are those with board definitions in `qiling/extensions/mcu/` (STM32F1/F4, GD32VF1, NXP, Atmel, BES); new chips need new peripheral maps.
- BLOB mode provides no services by design — targets needing hardware must use MCU mode instead.
- `test_blob.BlobTest.test_blob_raw` (`tests/test_blob.py:85`) errors with
`FileNotFoundError` on a clean checkout: it reads
`examples/rootfs/blob/example_raw.bin` (`tests/test_blob.py:96`), but the
pinned `examples/rootfs` submodule ships only `u-boot.bin.img`. Either the
fixture must be added upstream and the submodule bumped, or the test skipped
when the fixture is absent.
10 changes: 5 additions & 5 deletions ARCHITECTURE/os-base.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,13 @@ everything else is exercised by every OS suite.

## Key Types and Entry Points

- `qiling/os/os.py:24` - `QlOs` - composes utils/stats/path/fs-mapper; `user_defined_api` keyed by `QL_INTERCEPT`; abstract `run()` (`:239`).
- `qiling/os/os.py:224` - `QlOs.set_api(target, handler, intercept)` - user override of an emulated API (address- or name-based).
- `qiling/os/memory.py:23` - `QlMemoryManager` - `ql.mem`; instantiated by core *before* the OS (`qiling/core.py:188`).
- `qiling/os/memory.py:658` - `QlMemoryHeap` - heap used by Windows/UEFI APIs and the sanitizers extension.
- `qiling/os/os.py:24` - `QlOs` - composes utils/stats/path/fs-mapper; `user_defined_api` keyed by `QL_INTERCEPT`; abstract `run()` (`:240`).
- `qiling/os/os.py:225` - `QlOs.set_api(target, handler, intercept)` - user override of an emulated API (address- or name-based).
- `qiling/os/memory.py:40` - `QlMemoryManager` - `ql.mem`; instantiated by core *before* the OS (`qiling/core.py:188`).
- `qiling/os/memory.py:678` - `QlMemoryHeap` - heap used by Windows/UEFI APIs and the sanitizers extension.
- `qiling/os/fcall.py:21` - `QlFunctionCall` - argument/return marshalling on top of `qiling/cc/`.
- `qiling/os/mapper.py:64` - `QlFsMapper` - behind `ql.add_fs_mapper` (`qiling/core.py:701`).
- `qiling/os/path.py:14` - `QlOsPath` - rootfs-confined path resolution.
- `qiling/os/path.py:16` - `QlOsPath` - rootfs-confined path resolution.
- `qiling/os/thread.py:11` - `QlThread(Greenlet)` - gevent-based thread base.

## Interactions
Expand Down
4 changes: 2 additions & 2 deletions ARCHITECTURE/os-windows.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,14 @@ run on Linux. Windows tests are platform-gated locally (need collected DLLs).

- `qiling/os/windows/windows.py:33` - `QlOsWindows(QlOs)` - `__setup_components` (`:156`) builds handle manager/registry/clipboard/fiber; `run()` (`:201`).
- `qiling/os/windows/fncc.py` - `@winsdkapi` decorator - declares an API's calling convention and typed params; implementations live in `dlls/`.
- API dispatch is address-based: IAT addresses recorded by the PE loader are hooked and marshalled through `QlFunctionCall` — user overrides via `QlOs.set_api` (`qiling/os/os.py:224`).
- API dispatch is address-based: IAT addresses recorded by the PE loader are hooked and marshalled through `QlFunctionCall` — user overrides via `QlOs.set_api` (`qiling/os/os.py:225`).
- `qiling/os/uefi/uefi.py:22` - `QlOsUefi(QlOs)` - executes DXE/SMM modules; services in `bs.py`/`rt.py`/`smm.py` are installed as callable tables.
- `qiling/os/dos/dos.py:33` - `QlOsDos(QlOs)` - dispatches BIOS/DOS interrupts from `interrupts/`.

## Interactions

- All three subclass [os-base.md](os-base.md) `QlOs` and allocate from `QlMemoryHeap`.
- [loader.md](loader.md): `QlLoaderPE` builds PEB/TEB and records IAT hook addresses; `QlLoaderPE_UEFI` installs protocols into `os/uefi/context.py`; `QlLoaderDOS` sets real-mode state.
- [loader.md](loader.md): `QlLoaderPE` builds PEB/TEB and records IAT hook addresses; `QlLoaderPE_UEFI` installs protocols into `qiling/os/uefi/context.py`; `QlLoaderDOS` sets real-mode state.
- Argument marshalling uses `QlFunctionCall` + `qiling/cc/intel.py` conventions ([arch.md](arch.md)).
- The registry emulation reads hive files from the rootfs via [os-base.md](os-base.md) path services.
- `examples/uefi_sanitized_heap.py` pairs UEFI with the heap sanitizer from [extensions.md](extensions.md).
Expand Down
Loading