External Declarations (ext)

Silk’s external declaration feature lets Silk code call foreign functions and access foreign variables.

• The core construct is the ext definition, which declares:
• external C functions and their Silk function types, or
• external C variables and their Silk types.
• The compiler and runtime perform marshalling between Silk’s internal representations and the C ABI, following a documented mapping.

Declaring an External Binding#

Example:

ext foo = fn (string) -> void;
ext bar = u32;

Here:

• foo is a C function named foo with the given Silk function type.
• bar is a C variable of type u32.

Binding a Different External Symbol Name#

Sometimes you want the Silk binding name to differ from the linked external symbol name (for example, when writing wrapper modules that want to expose stable public API names without colliding with imported libc names).

Syntax:

// The binding name is `c_malloc`, but the linked symbol is `malloc`.
ext c_malloc "malloc" = fn (i64) -> u64;
ext c_free "free" = fn (u64) -> void;

Rules:

• The identifier after ext is the Silk binding name (used for imports and calls from Silk code).
• The optional string literal is the external symbol name used for linking (native) or as the import name (wasm).
• If the string literal is omitted, the external symbol name is the same as the binding name.

Avoiding Shadowing (Global ::...)#

If an ext binding is declared in the global namespace (a module with no package ...; or header-form module ...; declaration) and a local declaration shadows it (for example, a wrapper function named malloc), use the global-name prefix to force lookup of the global binding:

return ::malloc(bytes);

The global-name prefix is not limited to ext: it also applies to type names and enum variant paths in expression and type positions (for example, ::Foo, ::Foo{...}, or ::E::Variant), always forcing resolution in the global (unnamed) package.

Verification and ext (Silk rule)#

External declarations have no body available to the verifier.

Therefore:

• It is a compile-time error to attempt to verify an ext declaration.
• It is a compile-time error for verified code (code whose compilation requires proofs) to call an ext function or read an ext variable.

This intentionally limits verification across the ext boundary.

Notes#

Silk currently implements this feature under the ext keyword. The docs treat ext as canonical.

Currently supported:

• parsing ext external declarations and representing them in the AST,
• optional external symbol aliases (ext local "extern" = ...;),
• ext functions with fixed parameter lists (ext name = fn (T0, T1) -> R;) as callable symbols in Silk (C variadic ... is not implemented yet),
• ext function parameters of function type (fn(...) -> R) as C-compatible function pointers:
• at the ABI level, these are passed as a single u64 code pointer (no closure environment),
• arguments must be either:
• a top-level function name, or
• a non-capturing fn (...) -> ... expression,
• capturing closures (and arbitrary function-typed locals) are rejected for ext function-pointer parameters in the Supported forms,
• c_fn (...) -> R types as explicit C callback pointers (Supported forms):
• c_fn is a code-pointer-only function pointer type intended for FFI,
• unlike fn (...) -> R function values, c_fn (...) -> R values do not carry a closure environment and are safe to store in locals/struct fields and pass through APIs,
• a c_fn value may be formed only from:
• a top-level function name, or
• a non-capturing fn (...) -> ... expression,
• capturing closures are rejected when a c_fn is required.
• ext variables of scalar type (ext name = T; where T is a supported scalar such as int, fixed-width ints, bool, char, or f32/f64) as readable values in Silk,
• string parameters in ext function calls are lowered as C-string pointers (const char *) in Silk; the compiler-emitted backing bytes include a trailing NUL terminator, while the Silk string length excludes it.
• borrowed-view types are restricted at the external boundary:
• opaque handle types declared via struct Name; may be used behind a reference (&Name) in ext function parameters and results,
• ordinary references (&T) and slices (T[]) are rejected at ext boundaries,
• the same ordinary-borrow restriction also applies to unnamed-package C-facing export fn signatures because they participate in the external ABI surface,
• named-package Silk object exports may use slice parameters inside the compiler-owned package ABI; those parameters lower as { ptr: u64, len: i64 } and are not part of the C ext/header surface.
• lowering calls to ext functions when building:
• silk build --kind object, and
• silk build --kind static,
• silk build --kind shared,
• silk build --kind executable, producing relocations against undefined external symbols in the generated .o / .a, dynamic imports in the generated .so, or dynamic imports in the generated dynamically-linked executable (linux/x86_64).
• for shared libraries and dynamically-linked executables, external calls are routed through a GOT slot that is filled by the dynamic loader.
• ext variable reads are supported for the same outputs, producing relocations against undefined external data symbols (.o / .a) or dynamic imports (.so / dynamically-linked executable) routed through the GOT.
• for wasm targets (wasm32-unknown-unknown, wasm32-wasi), ext declarations map to wasm imports:
• ext foo = fn (...) -> ...; becomes an imported wasm function env.foo,
• ext bar = T; becomes an imported wasm global env.bar (for scalar T),
• parameter/result types follow the compiler’s current scalar lowering (for example int → wasm i64).

Not implemented yet (documented design, future work):

• writing to ext variables (they are read-only in the Supported forms),
• ext variables of non-scalar types (strings, structs, optionals, arrays),
• richer string and aggregate marshalling (for example: returning string from ext calls as an owned Silk value, passing/returning user-defined structs by value beyond the current ABI-safe POD subset, and C-facing array/slice bridging).
• calling back into Silk from foreign code with capturing closures or richer closure environments (only plain non-capturing function pointers are supported as ext parameters in the Supported forms).

Passing Callbacks to C (c_fn)#

Use c_fn (...) -> R to model C callback pointers you want to store and pass to foreign code.

Example:

type I64BinOp = c_fn (i64, i64) -> i64;

// C provides: int64_t call_i64_binop(int64_t (*cb)(int64_t, int64_t), int64_t a, int64_t b);
ext call_i64_binop = fn (I64BinOp, i64, i64) -> i64;

fn add (a: i64, b: i64) -> i64 {
  return a + b;
}

fn main () -> int {
  let cb: I64BinOp = add;
  let out: i64 = call_i64_binop(cb, 40, 2);
  if out != 42 { return 1; }
  return 0;
}

Notes:

• c_fn values are code pointers only; they cannot capture local variables.
• A raw dynamic symbol address can be converted explicitly with addr as raw c_fn (...) -> R or an alias of that type. This is intended for loader APIs such as std::dylib; Silk does not validate the symbol signature at runtime, so the declared c_fn type must match the foreign ABI exactly.
• A c_fn value can be called directly from Silk using the same call syntax as ordinary function values. The current value is invoked as a plain C code pointer with no closure environment.
• If a C API needs context, pass an explicit context pointer (e.g. a u64 that is a void * in C) alongside the callback and include that context parameter in the callback signature.

Opaque Struct Handles#

Opaque structs are intended for representing foreign pointers/handles safely. They strengthen type safety at the language boundary by preventing accidental mixups between different handle types and by disallowing invalid operations in Silk.

Declare an opaque handle type with a fieldless struct declaration:

// runtime.slk
struct StringBuilder;

ext sb_new = fn () -> &StringBuilder;
ext sb_append = fn (&StringBuilder, string) -> void;
ext sb_destroy = fn (&StringBuilder) -> void;

Use the handle by importing the type name and the ext functions:

import { StringBuilder, sb_new, sb_append, sb_destroy } from "./runtime.slk";

fn main () -> int {
  let sb: &StringBuilder = sb_new();
  sb_append(sb, "hello");
  sb_destroy(sb);

  // Using `sb` after destroy is UNDEFINED BEHAVIOR (dangling foreign pointer).
  return 0;
}

Rules:

• The handle type must be used behind & (&StringBuilder), not by value.
• Opaque structs cannot be instantiated and do not support member access.

Safety:

• You are responsible for managing the lifetime of foreign handles. Most C APIs provide explicit create/destroy functions; always call the destruction function when you are done.
• Using a handle after destruction is undefined behavior; the compiler does not currently enforce this at compile time.

Notes on executable ext calls (current linux/x86_64 implementation):

• When an executable uses ext calls or ext variable reads, the compiler emits a dynamically-linked ELF64 executable (PIE-style ET_DYN with PT_INTERP, .dynamic, .rela.dyn, and a .got).
• External symbols are resolved by the platform dynamic loader. Dependencies can be declared via the CLI (silk build --needed <soname> ...) or via the C99 embedding API (silk_compiler_add_needed_library), and runtime search paths can be declared via --runpath / silk_compiler_add_runpath.

ABI Contract (Overview)#

The language defines two closely related views of the ABI:

• A “fat pointer” internal representation for string and regexp:

• conceptually: struct string { ptr: ptr, len: i64 } where ptr is a UTF‑8 pointer.

• conceptually: struct regexp { ptr: ptr, len: i64 } where ptr is an engine-owned bytecode pointer.

• A C ABI contract (e.g. via silk/silk.h) using an explicit struct:

  typedef struct {
      char   *ptr;
      int64_t len;
  } SilkString;
  

• A mapping to an LLVM type used internally by the compiler:

  %silk.string = type { i8*, i64 }
  

When calling conventional C APIs, the compiler may pass a const char * derived from this structure, with the guarantee that the underlying data is null‑terminated. This distinction is important:

• Internal/runtime ABI: operates on { ptr, len } structs (SilkString).
• Compatibility calls to typical C libraries: may expose const char * for parameters declared as string in Silk ext declarations, with the compiler extracting the ptr.

Our embedding ABI for libsilk.a will treat SilkString as the canonical C representation; details are further specified in abi libsilk.

Primitive Type Mapping#

The spec includes a table mapping Silk primitive types to C types, for example:

• i8, u8 → int8_t, uint8_t
• i16, u16 → int16_t, uint16_t
• i32, u32 → int32_t, uint32_t
• i64, u64 → int64_t, uint64_t
• i128 → SilkI128 (see abi libsilk; { lo, hi } lanes)
• u128 → SilkU128 (see abi libsilk; { lo, hi } lanes)
• int → int64_t (current linux/x86_64 baseline; do not assume C int)
• f32 → float
• f64 → double
• f128 → SilkF128 (see abi libsilk; IEEE binary128 bits in { lo, hi })
• bool → bool (or _Bool)
• char → uint32_t (UTF‑32)
• string → SilkString ({ char *ptr; int64_t len; })
• regexp → SilkString ({ char *ptr; int64_t len; }, opaque bytecode view)
• void → void

Notes:

• For FFI with APIs that use a C int (for example many POSIX syscalls), prefer i32/u32 in your ext declarations rather than int.
• The stable C99 ABI does not use compiler-specific __int128 or __float128 types for these primitives; it uses explicit { lo, hi } structs so the ABI is portable and can be expressed in strict C99.

These mappings must be reflected exactly in the C99 ABI.

Strings and Passing Convention#

For strings, the spec makes the following points:

• Silk’s string is represented internally as a { ptr, len } pair.
• For ext calls to typical C APIs:
• the compiler can extract ptr and pass it as a const char *,
• the data is guaranteed to be null‑terminated so standard C string functions are safe.

For regex bytecode values (regexp):

• Silk’s regexp is represented internally as a { ptr, len } pair with the same slot layout as string, but the bytes are not text and are not required to be null‑terminated.
• At ABI boundaries, regexp uses the same C shape as SilkString, but C code must treat it as an opaque (ptr, len) byte span (not a C string).
• Runtime regex helpers validate malformed or undersized foreign regexp payloads before execution and report them as invalid input, but C code must still not fabricate regex bytecode as if it were a stable public format.
• Runtime regex helpers also track which bytecode buffers they actually allocated: only std::regex::RegExp.compile(...) produces an owned regex allocation, while borrowed/literal/foreign regexp views are ignored by the regex free/drop path instead of being freed as if they were runtime-owned.

For the embedding ABI (libsilk.a):

• We treat SilkString ({ char *ptr; int64_t len; }) as the primary C representation of Silk string values.
• Functions exported by libsilk.a will use SilkString in their signatures wherever strings cross the boundary.

This layered design allows:

• idiomatic FFI to existing C libraries using const char *,
• a precise, length‑carrying ABI (SilkString) for embedding the compiler/runtime.

Safety & Ownership#

The external interface rules must ensure:

• No C code can violate Silk’s invariants about ownership and lifetimes.
• Any shared data representation (strings, structs, arrays, closures) is documented and stable.

Typed Errors and the ext boundary#

Typed errors (error, panic, and T | ErrorType...) must not cross the ext boundary.

Rules:

• ext function types must not use | in their return types.
• Silk-to-C ABI surfaces must not expose | in exported function signatures. Shims should convert typed errors into explicit error codes, optionals, or domain-specific error types, or terminate in a platform-appropriate way.

Implementation

• The current compiler rejects ext declarations that include |, and rejects exporting error-producing functions to C ABI outputs.

The spec also includes a “Structs, Arrays, and Closures (Complex Types)” subsection for FFI. As the implementation proceeds, this document must be extended to:

• describe how user‑defined structs map to C structs (respecting the layout rules in structs-impls-layout.md),
• define how arrays and slices are represented across the boundary,
• document any stable closure representation, if exposed in the C ABI.

Structs#

The full language design includes rich user-defined structs and nested aggregates. The current compiler implementation supports only a small subset of structs in code generation:

• structs with 0+ fields of supported value types (scalar primitives, string, nested structs, and supported optionals) in function bodies and internal helper calls,
• on linux/x86_64, passing and returning these structs by value at ABI boundaries using a scalar-slot lowering model:
• a struct value lowers to N scalar “eightbyte” slots in field order, and each slot is classified as INTEGER (integer-like scalars such as int, fixed-width integers, bool, char, Instant, Duration) or SSE (f32/f64),
• exported function parameters accept these slots as separate parameters; for 1–2 slot structs this is ABI-compatible with a by-value C struct parameter for the 8-byte-field subset, while for packed structs with smaller fields ABI compatibility with an equivalent C struct layout is not yet implemented/validated; for 3+ slot structs downstream C callers should declare separate parameters for the slots,
• exported function returns support 1+ slot structs; 1–2 slot results return in rax/rdx and/or xmm0/xmm1 accordingly, while 3+ slot results return indirectly via a hidden sret pointer.

This subset is intended as a stepping stone toward fully general struct layout and SysV ABI classification (including packed layout for smaller fields such as f32 and small integers, nested structs, and larger aggregates returned via hidden sret pointers).

Optionals#

The full language design includes rich optional patterns (?., match, nested optionals, etc.). The current compiler implementation supports only a limited optional subset in code generation:

• optionals whose payload type is a supported scalar, string, or a supported ABI-safe struct (i.e. after slot-flattening, all scalar slots are i64/u64/f64),
• construction via None and Some(value),
• unwrapping via ?? with short-circuit fallback evaluation,
• and nested optionals (T??) for the same supported payload subset, including unwrapping T?? to T? via ??.

At ABI boundaries in the current linux/x86_64 subset, optionals are lowered as a Bool tag followed by the payload scalar slots in order:

• (tag, payload) for scalar payload optionals,
• (tag, ptr, len) for string?,
• (tag, slot0, slot1, ...) for struct? where the payload lowers to N scalar slots.

For nested optionals (T??) in this subset, the payload slots are the full inner optional representation (for example int?? lowers as (tag0, tag1, i64 payload)).

For exported functions, these slots consume the normal scalar argument and result locations (registers then stack), and 3+ scalar results return via a hidden sret pointer.

Compiler requirements:

• Implement ext declarations as specified.
• Map Silk types to C types per the ABI document.
• Enforce the documented passing conventions and ownership rules for external-call strings and other bridged types.
• Keep this document and abi libsilk in sync with the actual codegen strategy.