Oro Computer logooro
Inside
Silk
Standard library / std::runtime

std::runtime

std::runtime defines a runtime interface layer that sits underneath the rest of the standard library.

The goal is to make OS- and environment-specific primitives pluggable while keeping the public std::... APIs stable. The default std shipped with the compiler targets a hosted POSIX baseline, but other environments (Windows, non-POSIX, embedded, sandboxed runtimes) should be able to provide their own runtime implementation by supplying an alternate stdlib root with compatible std::runtime::... modules.

Motivation#

  • std::fs, std::task, std::sync, and other OS-facing std modules need low-level primitives (files, clocks, threads, syscalls).
  • Those primitives differ significantly across platforms.
  • Keeping these differences confined to std::runtime::... avoids scattering ext and platform #if style logic across the entire stdlib.

Structure#

The std runtime is organized as:

Example:

In the shipped stdlib today:

The long-term shape is still that std::runtime::<area> remains the stable interface point, while platform backends (such as std::runtime::posix::<area> and std::runtime::windows::<area>) can exist as separate modules in an alternate stdlib root without changing higher-level std::... modules.

Interface Design Rules#

  • The std::runtime::... surface is allowed to be low-level and unsafe: raw pointers, integer error codes, and OS-specific constants are acceptable.
  • When an operation can fail, prefer returning the error code as a value (via std::result::Result(T, int) or an optional error int?) so callers do not need to pair a sentinel return with a separate errno() query.
  • Higher-level, ergonomic, and allocation-aware APIs belong in std::... modules (for example std::fs::File.read_to_end).
  • std::runtime::... modules should avoid exposing platform-specific struct layouts directly to Silk code when possible; prefer integer-like handles and pointer-plus-size patterns.
  • The stable contract is the Silk-level signature in std::runtime::..., not the specific ext spellings used by the POSIX backend.
  • Low-level primitives should be localized:
  • libc allocator ext bindings (malloc/free/realloc) and compiler-backed raw-memory/string intrinsics (__silk_*) live in std::runtime::posix::mem (and analogous backend mem modules),
  • other runtime backend modules should call those exported wrappers instead of declaring duplicate allocator/intrinsic ext sites.
  • for example, the shipped WASI backends (std::runtime::wasi::io and std::runtime::wasi::time) use std::runtime::wasi::mem for allocation and __silk_* intrinsics.

Considerations#

Runtime areas in the shipped stdlib:

  • std::runtime::mem — low-level allocation and compiler-backed intrinsics used by higher-level std modules (alloc/realloc/free, raw load/store, and string view helpers), plus basic environment queries used by higher-level wrappers (for example page_size() for mmap alignment).
  • when an active region context is established with with (regions), allocations are routed to that region for the dynamic extent of the with block (including calls into stdlib code):
  • std::runtime::mem::alloc allocates from the active region instead of the heap,
  • std::runtime::mem::realloc reallocates region pointers by allocating a new region block and copying bytes (it never calls libc realloc on a region-backed pointer),
  • std::runtime::mem::free is a no-op for region-backed pointers.
  • pointers returned by std::runtime::mem::alloc are owned by Silk; they must be released with std::runtime::mem::free and are not valid to pass to libc free() directly.
  • std::runtime::build — build metadata provided by the compiler:
  • is_debug() -> bool returns true when the current artifact was compiled with silk ... --debug (or -g).
  • kind() -> string returns the current build kind ("executable", "object", "static", or "shared").
  • mode() -> string returns the current build mode ("debug", "release", or "test").
  • version() -> string returns the current package version when building a package, otherwise "0.0.0".
  • the module also exports Formal Silk build-gating theories:
  • build_kind_is(...)
  • build_mode_is(...)
  • requires_debug_mode()
  • requires_release_mode()
  • requires_executable_kind()
  • requires_object_kind()
  • requires_static_kind()
  • requires_shared_kind()
  • build_version_at_least(...)
  • std::runtime::gpu — Linux host-side GPU discovery, tracked device memory, bounded copies, packed explicit-kernarg kernel launch, synchronization, and last-error access used by std::gpu; see runtime gpu.
  • std::runtime::fs — filesystem primitives used by std::fs (hosted baseline; on wasm32-wasi the shipped backend supports a small subset using the first preopened directory as a sandbox root, and resolves relative paths against a virtual cwd (std::runtime::wasi::cwd)).
  • includes read-only mapping helpers (mmap_readonly / munmap); on wasm32-wasi mapping is currently unsupported and reports InvalidInput.
  • includes mkstemp(template_ptr) for creating unique temporary files from a writable NUL-terminated template ending in XXXXXX (hosted POSIX baseline). On wasm32-wasi this operation is currently unsupported and reports InvalidInput.
  • includes raw stat metadata queries (stat, lstat, fstat) plus path classification (path_kind) and owned canonical path resolution (realpath) for higher-level wrappers such as std::fs::{stat,lstat,fstat,path_kind,is_regular_file,realpath}.
  • on the hosted POSIX baseline, stat / path_kind follow symlinks, lstat reports the link itself, and realpath resolves symlinks via the underlying OS,
  • on wasm32-wasi, stat / lstat / fstat are implemented from WASI preview1 filestat syscalls with a reduced metadata set, path_kind is supported, and realpath is currently unsupported and reports InvalidInput.
  • std::runtime::io — low-level stdio primitives used by std::io (on wasm32-wasi, rewritten to std::runtime::wasi::io, which maintains a POSIX-shaped errno cell for wrappers that still query errno()). This surface also includes async fn wrappers (read_async / write_async) backed by the hosted async runtime on linux/*; on other targets they complete immediately by issuing the blocking read/write operation. It also includes PTY helpers used by std::process::child::Command.spawn_pty on the hosted POSIX baseline (pty_open, pty_prepare_child).
  • std::runtime::task — hosted task/runtime primitives used by std::task (sleep/yield_now/available parallelism; currently blocking OS-thread operations; delegates to std::runtime::posix::task in the shipped stdlib).
  • std::runtime::sync — hosted synchronization primitives used by std::sync (mutexes/condvars and allocation helpers; delegates to std::runtime::posix::sync in the shipped stdlib. On wasm32-wasi the compiler rewrites this to std::runtime::wasi::sync, which is a single-thread stub backend.)
  • std::runtime::time — hosted time primitives used by std::temporal and other std modules:
  • monotonic clock reads (monotonic_now_ns),
  • Unix wall-clock timestamp reads (unix_now_ns / unix_now_ms),
  • delegates to std::runtime::posix::time in the shipped stdlib,
  • the POSIX implementation reads into a calling-thread stack timespec through the bundled runtime, so clock reads are allocation-free, reentrant across task workers, and supported by --noheap builds.
  • std::runtime::env — hosted environment primitives used by std::env (process environment variables; delegates to std::runtime::posix::env in the shipped stdlib on hosted targets. On wasm32-wasi the compiler rewrites the backend to std::runtime::wasi::env, which implements getenv via environ_sizes_get / environ_get (caching the environment snapshot for the process lifetime) and leaves setenv unsupported).
  • std::runtime::process — hosted process primitives used by std::process (current working directory plus child-process primitives for std::process::child; delegates to std::runtime::posix::process in the shipped stdlib on hosted targets. On wasm32-wasi, _exit is implemented via proc_exit, while chdir/getcwd are implemented via a virtual cwd layer (std::runtime::wasi::cwd); hosted child-process operations remain unsupported).
  • std::runtime::net — hosted networking primitives used by std::net (IPv4/IPv6 TCP + UDP sockets plus hostname resolution used by std::net::resolve_host; delegates to std::runtime::posix::net in the shipped stdlib).
  • Apple Security runtime helpers — bundled silk_rt_apple_crypto_* symbols used by the Apple platform security provider for std::crypto core/random operations. These helpers are statically linked when referenced and require Security.framework on Apple targets.
  • std::runtime::z3 — low-level ext bindings for the Z3 C API (built-in on the glibc hosted layout; musl targets require an explicit downstream Z3 library).
  • std::runtime::regex / std::runtime::unicode / std::runtime::number / std::runtime::readline — non-OS-specific runtime helpers used by std::{regex,unicode,number,readline}. These are implemented via ext bindings to a small bundled runtime support library (libsilk_rt) that ships alongside the compiler.
  • std::runtime::readline now also includes process-global completion-list management used by the public std::readline::{clear_completions,add_completion} surface.
  • the compiler statically links this bundled runtime support into executable and shared-library outputs (no runtime DT_NEEDED dependency on libsilk_rt*).
  • embedders can override internal allocation used by libsilk_rt (for example regex runtime compilation) by calling silk_rt_set_allocator (see include/silk/rt.h) before invoking any silk_rt_* entrypoints. This hook affects allocations routed through silk_rt_malloc_bytes / silk_rt_realloc_bytes / silk_rt_free_bytes; it does not change the allocator used by std::runtime::mem for heap-backed pointers.
  • allocator changes affect future bundled-runtime allocations. Any pointer returned by silk_rt_malloc_bytes(...) remembers the realloc/free hooks that created it, so later silk_rt_realloc_bytes(...) / silk_rt_free_bytes(...) calls keep allocator identity stable across later allocator changes. Runtime-owned regex bytecode uses that same lifetime rule.
  • the allocator override is process-global, but bundled-runtime helper calls and silk_rt_set_allocator are internally synchronized while reading or updating the current hook set. Concurrent allocator changes can still affect which hook future bundled-runtime allocations use.
  • silk_rt_realloc_bytes(...) and silk_rt_free_bytes(...) only accept pointers that correspond to a currently live allocation previously returned by silk_rt_malloc_bytes(...) or silk_rt_realloc_bytes(...). Foreign pointers, forged helper headers, stale pre-realloc pointers, and already-freed helper pointers are all treated as non-live inputs and ignored safely (realloc returns NULL, free is a no-op).
  • when building with --noheap, the compiler links libsilk_rt_noheap.a instead of libsilk_rt.a. In that configuration, libsilk_rt performs no default heap allocation unless an embedder installs an allocator via silk_rt_set_allocator.
  • std::runtime::window — low-level target/provider detection, opaque provider handles, nonblocking provider event polling, high-level provider run(...) / run_ex(...) boundaries, and native window-control hooks used by std::window. The shipped runtime currently opens AppKit windows with stdlib creation options, pumps one AppKit event at a time, exposes AppKit title/visibility/focus/size/position/minimize/maximize/always-on-top/ background controls, enters UIApplicationMain and creates a visible UIWindow on iOS when launched from the generated app bundle, and rejects GTK/unsupported targets through local stubs that do not declare window provider externs.
  • std::runtime::graphics::metal — low-level macOS Metal runtime boundary used by std::graphics::metal and std::graphics::window. It declares the silk_rt_metal_* device, queue, layer, drawable, render-pass, encoder, buffer, library, pipeline, draw, and compatibility clear-window ABI only for macOS. Unsupported-target behavior stays in the public graphics facades so non-Metal targets do not lower Metal provider externs.

Follow-ups are expected to introduce additional runtime areas:

  • Async event loop / executor integration (std::runtime::event_loop) for hosted async/await:
  • the compiler already ships a bundled bring-up executor in libsilk_rt (src/silk_rt_async.c) and lowers async/await to it on the hosted linux/x86_64 target,
  • the std::runtime::event_loop module now exposes low-level awaitable building blocks (timers + fd readiness, including fd_wait_readable2 and fd_wait_readable_any) and an explicit Handle/poll surface for manually driving the hosted executor/event loop. Higher-level async adapters are still follow-up work (see async runtime).
  • the hosted executor is thread-affine: Handle.poll() / Handle.deinit() must be called from the same OS thread that created the handle; cross-thread wake is supported via Handle.wake().
  • abort-aware wrappers exist for cooperative cancellation (std::abort_controller):
  • sleep_ms_abortable(ms, sig) -> bool
  • fd_wait_readable_abortable(fd, sig) -> bool
  • fd_wait_writable_abortable(fd, sig) -> bool In the Supported forms, aborts are generally observed only before/after the awaited operation. For fd_wait_readable_abortable, when the runtime can provide a pollable abort fd (AbortSignalBorrow.wait_fd()), aborts can interrupt an in-flight wait by awaiting fd_wait_readable2(fd, abort_fd). When no pollable abort fd is available, it falls back to the before/after checks.
  • WASI networking (via WASI sockets or similar proposals) when supported by the toolchain targets.

Providing a Custom Runtime#

To provide your own runtime implementation underneath the standard library, ship an alternate stdlib root that includes compatible std::runtime::... modules.

For a CLI-focused walkthrough of selecting a std root and archive, see howto custom stdlib root.

At a minimum, your stdlib root should provide the runtime areas used by the higher-level std modules you want to reuse. For example, to reuse the shipped std::task and std::sync, provide:

and similarly for std::fs (std/runtime/fs.slk) if you reuse std::fs.

To reuse std::io, provide std/runtime/io.slk implementing the std::runtime::io interface (STDIN_FD, STDOUT_FD, STDERR_FD, read, write, dup, read_async, write_async, puts, and hosted fd helpers used by std::process::child such as dup2, pipe, poll, and set_cloexec).

Fallible operations should return errors directly:

  • value-returning operations use std::result::Result(T, int) where Err(int) is a stable, area-specific error code consumed by higher-level std::... wrappers (for example std::io::IOFailed.code),
  • status operations use optional errors (int?), returning None on success and Some(code) on failure.

On hosted POSIX, runtime wrappers typically map errno into these stable codes inside std::runtime::<area> so callers do not need to pair sentinel returns with a separate errno() query.

To reuse hosted time helpers in std::temporal, provide std/runtime/time.slk implementing the std::runtime::time interface (monotonic_now_ns, unix_now_ns, and unix_now_ms).

Selecting the Runtime (Toolchain)#

Because std::runtime is part of the stdlib source tree, selecting a custom runtime is done by selecting a custom stdlib root:

  • CLI: pass --std-root <path> (and optionally --std-lib <path> to provide a prebuilt std archive), or set SILK_STD_ROOT / SILK_STD_LIB.
  • Embedding ABI: set silk_compiler_set_std_root (and optionally set SILK_STD_LIB to point at a prebuilt std archive).

When no suitable std archive is provided, the compiler can fall back to compiling the reachable std sources as part of the build on supported targets.

Building a Custom Std Archive#

For supported native hosted archive targets in the current toolchain (linux/x86_64 and macos/aarch64), a prebuilt stdlib archive (libsilk_std.a) contains one object per std module.

Archive member naming requirement (current scheme):

  • the archive member name is the module path relative to the std root with / replaced by _, and .slk replaced by .o,
  • for example: std/runtime/posix/task.slk → runtime_posix_task.o.

The in-repo make stdlib target produces archives with this naming scheme.

Source repository · Edit this page · View Markdown

Previousstd::ssh2
Nextstd::runtime::event_loop