`std::protobuf`

std::protobuf provides dependency-free Protocol Buffers binary wire helpers for Silk code and for modules generated by silk proto.

The module covers the proto3 wire-format primitives:

varint keys and integer payloads,
ZigZag sint32 / sint64,
little-endian fixed32/fixed64 payloads,
length-delimited strings, bytes, embedded messages, and packed repeated fields,
field skipping,
and preservation of unknown field bytes.

It intentionally does not shell out to protoc and does not depend on a C or third-party protobuf runtime.

Types#

WireType is the protobuf wire type tag enum. Use wire_type_code() and wire_type_from_code() when storing or validating numeric wire tags.
DecodeErrorKind is the stable decode error category enum.
DecodeError carries code, byte offset, field_number, and numeric wire_type context. kind() converts the stored code back to DecodeErrorKind; message() returns a short human-readable message.
Key is the decoded (field_number, wire_type) record key returned by Reader.read_key().
Reader is a borrowing binary reader over std::arrays::ByteSlice.
Decode*Result aliases are std::result::Result(..., DecodeError) shapes used by reader methods.

Strings and byte slices decoded by Reader borrow from the input byte slice. Callers must keep the original byte storage alive for as long as decoded strings or bytes are used.

Reading#

Create a reader with:

let mut reader = std::protobuf::Reader.from_bytes(bytes);

Then repeatedly call read_key() until reader.eof():

let key_r = reader.read_key();
if let Err(e) = key_r {
  return 1;
}

let key = match key_r {
  Ok(value) => value,
  Err(_) => std::protobuf::Key{ field_number: 0, wire_type: std::protobuf::WireType::Varint },
};

Payload helpers include:

read_varint()
read_length()
read_i32()
read_i64()
read_uint32()
read_sint32()
read_sint64()
read_fixed32_bits()
read_fixed64_bits()
read_float()
read_double()
read_bool()
read_bytes()
read_string()
skip_field(key)

Generated string decoders use read_bytes() and convert the byte slice to a borrowed string view, which keeps string payload handling explicit.

Writing#

Writers append to std::buffer::BufferU8 and return std::memory::OutOfMemory?:

let mut out = BufferU8.empty();
if std::protobuf::write_int64(mut out, 1, 42) != None {
  return 1;
}

Payload helpers include:

write_varint(out, value)
write_key(out, field_number, wire_type)
write_bool(out, field_number, value)
write_uint64(out, field_number, value)
write_int64(out, field_number, value)
write_sint32(out, field_number, value)
write_sint64(out, field_number, value)
write_fixed32_bits(out, field_number, value)
write_fixed32_payload(out, value)
write_fixed64_bits(out, field_number, value)
write_fixed64_payload(out, value)
write_float(out, field_number, value)
write_double(out, field_number, value)
write_bytes(out, field_number, bytes)
write_string(out, field_number, value)
append_unknown_field(out, source, start, end)

append_unknown_field() copies a raw field record from source[start..end) and is used by generated decoders to preserve forward-compatible data.

`silk proto`#

Use silk proto to compile .proto files to Silk modules:

silk proto -I schemas -o generated schemas/person.proto

The compiler accepts proto3 files with syntax = "proto3"; and covers:

packages and imports,
normal imports, import public re-exports, and missing import weak declarations when no referenced type depends on the weak file,
adjacent protobuf string literals in string-valued schema positions, including syntax, import paths, and reserved names,
protobuf integer literals in decimal, hexadecimal, octal, and signed forms where the schema grammar permits signed integers,
file/message/field/enum/service options,
messages and nested messages,
enums and aliases,
scalar, message, enum, optional, repeated, oneof, and map fields,
reserved field names and ranges,
services/rpcs as generated descriptor structs and lookup functions,
recursive import loading through -I / --proto-path,
default emission of imported schema dependencies needed by generated Silk imports,
complete --descriptor-out JSON summaries for files, imports, options, messages, fields, oneofs, nested types, enums, services, RPCs, reserved declarations, resolved type names, and generated module/type names,
field-number validation, reserved-range validation, duplicate detection, generated Silk name-collision validation, generated storage-field collision validation, map key validation, enum-zero validation, type-reference validation, cross-file import visibility validation, cyclic import detection, and targeted rejection of unsupported extend, extensions, and group declarations.

Generated modules include:

Silk structs for messages,
Silk enums for known constants, raw-preserving EnumValue wrappers for enum field storage, and numeric conversion helpers for protobuf enum values,
vector aliases for repeated/map fields,
T? storage for singular message fields and explicit optional fields, with None representing absent fields and Some(value) representing presence,
empty(), drop(), encode(), decode(), and merge_from() methods for messages,
service RPC descriptor structs plus <Service>_RPC_COUNT and <Service>_rpc(index) metadata helpers,
raw unknown-field storage slots plus unknown_fields() for preserved unknown field bytes.

Generated modules type-check, expose their public API surface, build as object code, and can be imported by Silk programs that construct messages, encode them, decode them, and inspect decoded fields. The regression suite covers both direct std::protobuf wire helpers and an end-to-end generated-message round-trip executable.

Generated cross-file references use named Silk imports. The generator keeps the original proto type name when it is unique in the generated module and uses a deterministic alias only when a name collision would otherwise occur.

Compatibility#

std::protobuf follows proto3 binary wire-format compatibility rules:

singular scalar fields default to the proto3 zero value,
singular message fields use presence, default to None, and encode only when set to Some(value); repeated wire occurrences merge into the existing message payload,
explicit optional fields use T? storage and follow None/Some(value) presence,
enum fields use generated raw-value wrappers, so numeric values unknown to the current schema are preserved and re-encoded instead of being coerced to a default or discarded,
repeated primitive and enum fields accept packed and unpacked encodings,
repeated message, string, and bytes fields accept length-delimited unpacked records,
generated decoders validate a known field's wire type before consuming its payload,
unknown fields are preserved verbatim when generated decoders skip them,
and unsupported or missing imports are reported by silk proto instead of being silently ignored.

Generated encoders emit proto3 packed encoding for repeated scalar and enum fields by default. A field option of [packed = false] forces unpacked encoding. Decoders accept both packed and unpacked encodings for packable fields so schema changes remain wire-compatible.

Descriptor JSON#

silk proto --descriptor-out <path> writes a deterministic JSON document:

top-level version is 1;
files[] records path, source_path, syntax, package, generated module, explicit_input, imports, options, messages, enums, and services;
message descriptors include generated Silk names, options, reserved names and ranges, fields, oneofs, nested messages, and nested enums;
field descriptors include protobuf name/number/label/type, semantic kind (scalar, message, enum, or map), resolved full type names when applicable, oneof membership, packed-encoding status, map key/value metadata, and raw field options;
enum descriptors include values, aliases as repeated numeric values, options, and reserved declarations;
service descriptors include RPC names, request/response type names, resolved full type names, streaming flags, and options.

std::protobuf