Compiler Architecture

This document describes the intended architecture of the Silk compiler implemented in Zig.

Current hard implementation limits (file size caps, current maxima, etc.) are documented in limits.

High-Level Structure#

The compiler is implemented in Zig and organized into three major layers:

• Front-end:
• Lexer: implements the token set and literals from operators and the literal docs.
• Parser: implements the grammar from grammar.
• Type checker: enforces the type rules from types and related concept docs.
• Verifier: handles Formal Silk constructs from formal verification.
• Middle-end:
• IR representation for Silk programs, including regions, buffers, concurrency, and FFI constructs (see ir overview for the current IR design and roadmap).
• Optimizations that respect the language’s safety guarantees.
• Back-end:
• Code generation for executables, static libraries, and shared libraries using a Silk-owned backend (IR + codegen), not by “transpiling to C”.
• Emission of object files and archives that can be linked into executables and libraries.
• C99 ABI mappings for interop with libsilk.a.

The driver gives each type-check invocation an isolated scratch lifetime. A recycling allocator services the checker's frequent short-lived allocations and is backed by a per-invocation arena; after checker diagnostics have released their owned state, the arena reclaims any scratch intentionally retained until the phase boundary. Repeated in-process checks therefore do not accumulate checker memory or report phase-owned scratch as application leaks.

In terms of concrete targets and file formats, the back-end MUST eventually support:

• ELF for Unix-like systems:
• linux/x86_64 is the initial full IR-backed target (executables, objects, static libraries, and shared libraries).
• linux/aarch64 (ARM64) is a required future IR-backed target (beyond const-only executables).
• position-independent code and shared objects (.so) for dynamic libraries.
• Mach-O for macOS:
• both Intel (x86_64) and Apple Silicon (arm64) MUST be supported,
• dynamic libraries (.dylib) for loading Silk packages at runtime.
• PE/COFF for Windows:
• initially x86_64, with other architectures considered later as needed,
• DLLs for dynamic loading.

Current snapshot (Silk (ABI) 0.1.1):

• src/backend_const.zig provides a target-aware const-main stub backend that emits minimal executables for a fully-constant executable entrypoint (main reduces to a constant integer or a void main falls through; supports fn main () -> int, fn main () -> void, and the standard fn main(argc: int, argv: u64) -> int / fn main(argc: int, argv: u64) -> void forms when arguments are unused):
• ELF64: linux-x86_64, linux-x86_64-musl, linux-aarch64, linux-aarch64-musl, android-aarch64
• Mach-O 64-bit: macos-x86_64, macos-aarch64, ios-aarch64, ios-simulator-aarch64, ios-simulator-x86_64
• PE32+: windows-x86_64, windows-aarch64 This backend does not link the full runtime/stdlib; it only encodes “exit with this integer”. macOS host note:
• macos-aarch64 const-main executables are now emitted directly by the Silk-owned Mach-O byte backend,
• when the host is macOS and the output target is macos-x86_64 or macos-aarch64, the driver still performs an ad hoc host codesign -s - pass so the generated executable is runnable immediately on macOS hosts.
• src/backend_macho_aarch64_host.zig provides a temporary Apple Silicon host-backed Mach-O backend:
• it currently covers the validated scalar IR executable subset on macos/aarch64 hosts for:
• macos-aarch64,
• ios-aarch64,
• ios-simulator-aarch64,
• and ios-simulator-x86_64,
• emits target-specific arm64 or x86_64 assembly, assembles it with host clang -c, links executables and dylibs with host ld, and builds static archives with Apple libtool -static,
• macos-aarch64 also expands bundled libsilk_rt*.a archives into temporary object members before host linking so runtime-backed scalar executables build on Apple Silicon,
• macos-aarch64, ios-aarch64, ios-simulator-aarch64, and ios-simulator-x86_64 also reuse the same host-backed object emitter to produce Mach-O relocatable objects, static library archives via Apple libtool -static, and Mach-O dylibs via the Apple linker for the current supported library IR subset,
• ios-aarch64, ios-simulator-aarch64, and ios-simulator-x86_64 are intentionally narrower than macos-aarch64, but now include:
• pure-Silk scalar executables,
• reachable float-to-int lowering via target-correct helper objects compiled from src/silk_rt_f128.c,
• and portable bundled runtime helpers compiled on demand for the requested iOS SDK target (number / regex / unicode / filesystem / dns / bounded socket networking / process / signal / term / pty / readline / task-pool / async),
• mixed .slk + native .c / .h / .m / .o / .a executable, static-library, and shared-library inputs, with .m compiled as Objective-C and linked against libobjc for executable/shared outputs,
• native-input-only executables whose main comes from linked objects or archives,
• and object/static/shared library outputs for the supported library IR subset,
• this host-backed subset is now wired through the silk CLI / driver path (with the same host codesign -s - post-pass on macOS targets that need it),
• and is an explicit bring-up step until the non-const Mach-O executable path is emitted fully by Silk-owned codegen.
• src/backend_ir_elf.zig provides an IR→ELF backend for linux-x86_64 and linux-x86_64-musl outputs (a growing subset of the language, including multi-function programs, rodata, and link-input builds). This backend is host-agnostic for x86_64 Linux ELF outputs: it can emit Linux ELF artifacts even when the compiler itself is running on a non-linux/x86_64 host. The driver selects glibc or musl loader/libc defaults from the explicit target, --elf-interp, manifest elf_interp, or SILK_ELF_INTERP. Shared IR type resolution preserves the language's transparent-alias rule: a named import of an exported type alias resolves through that alias to its concrete struct or enum carrier, including a monomorphized generic carrier, in both expression- and statement-position lowering.
• src/backend_wasm_ir.zig provides the IR-backed backend for wasm32-unknown-unknown and wasm32-wasi outputs.
• src/lower_gpu_ir.zig builds target-neutral device ir.Program graphs from launchable GPU entries and their eligible direct helper call graphs. Portable device primitives are represented as compiler-owned semantic externs in IR; they never become link imports. backend gpu defines the shared contract consumed by AMDGPU and the GPU-v1 NVIDIA PTX selector.
• src/backend_amdgpu.zig provides the first standalone AMDGPU backend encoder. It recognizes AMDHSA code-object targets for gfx942, gfx1100, and gfx1151, emits dependency-light ELF64 AMDGPU code-object bytes plus MessagePack metadata, and serializes 64-byte HSA AQL kernel-dispatch packets. silk build --kind object --target amdgcn-* can emit an AMDHSA .hsaco for exactly one exported root-package void source kernel with up to 32 immutable u64 parameters whose body is empty or contains only supported compiler-backed GPU call statements. This is not yet a general Silk IR-to-GPU lowering path. Linux x86_64 executable builds may also use --gpu-target amdgcn-*: attr(device=gpu) functions are compiled into individual code objects, removed from host lowering, and appended in a provider-tagged versioned bundle consumed by std::gpu through the bundled GPU runtime. The same path accepts NVIDIA sm80 and embeds Silk-emitted PTX for the CUDA Driver API. The checked gpu (grid=..., workspace=...) { kernel(args...); } form resolves the entry name at compile time and dispatches through std::gpu::launch_and_synchronize. It returns both phase statuses as std::gpu::DispatchResult in value position and discards them in statement position; separate manual functions remain available for overlapping execution. Launch blocks are accepted in ordinary, async, and task host functions, with synchronization acting as a blocking host call. std::gpu::launch automatically packs the explicit u64 arguments, and std::gpu::device provides target-neutral global-index and packed-u32 load/store operations. The earlier fill and threshold-classification operations remain compatibility helpers. See backend gpu, backend amdgpu, and backend nvidia for the portable and provider contracts.
• src/backend_nvidia.zig is the GPU-v1 NVIDIA selector. It emits null-terminated PTX directly from the same target-neutral ir.Program, and the provider-neutral bundle records CUDA plus its PTX target tag. The hosted runtime loads that text through the CUDA Driver API without a link-time CUDA dependency or an external compilation command. See backend nvidia.

Portable Silk-owned Mach-O and PE/COFF IR-backed object/static/shared library emission, and additional IR-backed architectures (notably AArch64) are explicit future requirements and MUST be planned and implemented as the back-end matures. The current Apple target object/static/shared support is Apple Silicon host-backed bring-up, not the final portable Mach-O backend.

An initial IR-driven, native backend is being prototyped alongside the existing constant-expression emitter:

• the front-end (parser + checker) produces ast.Module values,
• a lowering pass in src/lower_ir.zig translates a constrained subset of fn main() -> int and fn main() -> void programs into ir.Function graphs, using integer arithmetic, comparisons, and simple control flow (Br / BrCond),
• a target-independent IR interpreter in src/ir_eval.zig provides reference semantics for these IR functions,
• the constant-expression backend emits a minimal target-specific executable stub (ELF64/Mach-O/PE32+) whose entrypoint terminates the process with the evaluated main return value,
• a dedicated IR→ELF backend module (src/backend_ir_elf.zig) will gradually assume responsibility for emitting native code directly from ir.Function graphs, starting with a single-function, integer-returning subset and expanding as more language features are lowered to IR.

Packages, Modules, Imports, and Exports#

Silk programs are organized into packages and modules:

• A module is a single source file and the natural unit of parsing and type checking.
• A package is a collection of modules that share a namespace and build configuration (e.g. the main package, std::, and third-party packages).
• Packages may:
• export symbols (types, functions, constants) that are visible to importers,
• import symbols from other packages via explicit imports.

The compiler MUST:

• represent packages and their dependency graph explicitly in the middle-end,
• implement an import resolver that:
• maps import paths to source modules/packages,
• enforces acyclic and well-formed package graphs,
• implement symbol visibility rules:
• distinguish exported vs internal symbols within a package,
• ensure only exported symbols are visible across package boundaries.

Front-end work (parser, checker, resolver) and back-end work (linkage, symbol emission) MUST be designed so that:

• importing and exporting package symbols is a first-class, well-specified feature,
• building advanced programs that span multiple modules and packages (including std:: and user packages) is supported by both the CLI (silk) and the C ABI (libsilk.a).

The concrete surface syntax for packages, imports, and exports is specified in packages imports exports and is being implemented incrementally in the front-end. Resolver and back-end integration will follow that spec.

The implementation must remain spec-driven: any architectural decision should be traceable back to a document in docs/.

Executable Entrypoint#

For executable builds driven via the C ABI (SILK_OUTPUT_EXECUTABLE) and, eventually, the silk CLI, the compiler enforces a simple, explicit entrypoint:

• there MUST be exactly one top-level function with one of these signatures:

  fn main() -> int { ... }
  fn main() -> void { ... }
  

• this function:

• takes no parameters,

• returns int or void,

• serves as the process entrypoint when an executable is produced.

In the initial bring-up, this requirement is enforced by the front-end (via silk_compiler_build) and a minimal back-end that supports only constant integer main functions. This is a temporary measure; the long-term back-end is a true Silk code generator, not a C transpiler.

Module Layout (Draft)#

This is a draft module layout for the Zig implementation. Exact file names may change, but the layering should be preserved.

• src/ (compiler implementation):
• src/driver.zig — CLI entry points and high-level orchestration.
• src/lexer.zig — tokenization and trivia handling.
• src/parser.zig — AST construction.
• src/ast.zig — AST node definitions.
• src/types.zig — type system representation and operations.
• src/checker.zig — type checking and semantic analysis.
• src/formal_silk.zig — Formal Silk VC generation and verification (Z3-backed).
• src/z3_api.zig — Z3 C API shim (static-by-default, optional dynamic override).
• src/ir.zig — core intermediate representation.
• src/codegen.zig — target-independent code generation logic.
• src/backend_amdgpu.zig — standalone AMDHSA ELF code-object and AQL packet encoder for the initial AMD GPU backend surface.
• src/abi.zig — C99 ABI and FFI glue for libsilk.a.
• src/std_integration.zig — integration with the std:: package and stdlib selection.
• src/cli/ (optional breakdown):
• src/cli/options.zig — option parsing.
• src/cli/commands.zig — build, check, abi subcommands.

In addition to the core compiler, a separate language server executable (silk-lsp) is provided for editor and IDE integrations. It is implemented in Zig, reuses the front-end modules above (lexer, parser, type checker), and speaks the Language Server Protocol as specified in lsp silk. The language server does not introduce new language features; it is a tooling layer over the existing compiler.

Test code is expected to live alongside these modules (via Zig test blocks) and/or under dedicated test drivers.

Test Layout (Draft)#

Testing is incremental and must be developed alongside the implementation:

• Zig unit tests:
• Each core module (lexer.zig, parser.zig, checker.zig, etc.) contains Zig test blocks that exercise its behavior.
• Additional integration tests may live in dedicated files (e.g. src/tests_frontend.zig) that compile sample Silk programs drawn from docs/language/.
• C99 tests:
• A separate directory (e.g. c-tests/) will contain C test programs and harnesses that:
• link against libsilk.a,
• use the C ABI (silk/silk.h) to drive compilation/execution,
• validate FFI and ABI behavior.

The build system (Zig build file and any supporting scripts) must be wired so that:

• running the Zig test suite exercises all relevant test blocks,
• running the C test suite builds and runs the C harnesses,
• both suites can be invoked easily during development and CI.