# [`std::protobuf`](/silk/docs/std/protobuf/)

[`std::protobuf`](/silk/docs/std/protobuf/) provides dependency-free Protocol Buffers binary wire helpers
for Silk code and for modules generated by `silk proto`.

The module covers the proto3 wire-format primitives:

- varint keys and integer payloads,
- ZigZag `sint32` / `sint64`,
- little-endian fixed32/fixed64 payloads,
- length-delimited strings, bytes, embedded messages, and packed repeated
 fields,
- field skipping,
- and preservation of unknown field bytes.

It intentionally does not shell out to `protoc` and does not depend on a C or
third-party protobuf runtime.

## Types

- `WireType` is the protobuf wire type tag enum. Use `wire_type_code()` and
 `wire_type_from_code()` when storing or validating numeric wire tags.
- `DecodeErrorKind` is the stable decode error category enum.
- `DecodeError` carries `code`, byte `offset`, `field_number`, and numeric
 `wire_type` context. `kind()` converts the stored code back to
 `DecodeErrorKind`; `message()` returns a short human-readable message.
- `Key` is the decoded `(field_number, wire_type)` record key returned by
 `Reader.read_key()`.
- `Reader` is a borrowing binary reader over [`std::arrays::ByteSlice`](/silk/docs/std/arrays/).
- `Decode*Result` aliases are [`std::result::Result(..., DecodeError)`](/silk/docs/std/result/) shapes
 used by reader methods.

Strings and byte slices decoded by `Reader` borrow from the input byte slice.
Callers must keep the original byte storage alive for as long as decoded
strings or bytes are used.

## Reading

Create a reader with:

```silk
let mut reader = std::protobuf::Reader.from_bytes(bytes);
```

Then repeatedly call `read_key()` until `reader.eof()`:

```silk
let key_r = reader.read_key();
if let Err(e) = key_r {
  return 1;
}

let key = match key_r {
  Ok(value) => value,
  Err(_) => std::protobuf::Key{ field_number: 0, wire_type: std::protobuf::WireType::Varint },
};
```

Payload helpers include:

- `read_varint()`
- `read_length()`
- `read_i32()`
- `read_i64()`
- `read_uint32()`
- `read_sint32()`
- `read_sint64()`
- `read_fixed32_bits()`
- `read_fixed64_bits()`
- `read_float()`
- `read_double()`
- `read_bool()`
- `read_bytes()`
- `read_string()`
- `skip_field(key)`

Generated string decoders use `read_bytes()` and convert the byte slice to a
borrowed string view, which keeps string payload handling explicit.

## Writing

Writers append to [`std::buffer::BufferU8`](/silk/docs/std/buffer/) and return
[`std::memory::OutOfMemory?`](/silk/docs/std/memory/):

```silk
let mut out = BufferU8.empty();
if std::protobuf::write_int64(mut out, 1, 42) != None {
  return 1;
}
```

Payload helpers include:

- `write_varint(out, value)`
- `write_key(out, field_number, wire_type)`
- `write_bool(out, field_number, value)`
- `write_uint64(out, field_number, value)`
- `write_int64(out, field_number, value)`
- `write_sint32(out, field_number, value)`
- `write_sint64(out, field_number, value)`
- `write_fixed32_bits(out, field_number, value)`
- `write_fixed32_payload(out, value)`
- `write_fixed64_bits(out, field_number, value)`
- `write_fixed64_payload(out, value)`
- `write_float(out, field_number, value)`
- `write_double(out, field_number, value)`
- `write_bytes(out, field_number, bytes)`
- `write_string(out, field_number, value)`
- `append_unknown_field(out, source, start, end)`

`append_unknown_field()` copies a raw field record from `source[start..end)` and
is used by generated decoders to preserve forward-compatible data.

## `silk proto`

Use `silk proto` to compile `.proto` files to Silk modules:

```sh
silk proto -I schemas -o generated schemas/person.proto
```

The compiler accepts proto3 files with `syntax = "proto3";` and covers:

- packages and imports,
- normal imports, `import public` re-exports, and missing `import weak`
 declarations when no referenced type depends on the weak file,
- adjacent protobuf string literals in string-valued schema positions, including
 syntax, import paths, and reserved names,
- protobuf integer literals in decimal, hexadecimal, octal, and signed forms
 where the schema grammar permits signed integers,
- file/message/field/enum/service options,
- messages and nested messages,
- enums and aliases,
- scalar, message, enum, optional, repeated, oneof, and map fields,
- reserved field names and ranges,
- services/rpcs as generated descriptor structs and lookup functions,
- recursive import loading through `-I` / `--proto-path`,
- default emission of imported schema dependencies needed by generated Silk
 imports,
- complete `--descriptor-out` JSON summaries for files, imports, options,
 messages, fields, oneofs, nested types, enums, services, RPCs, reserved
 declarations, resolved type names, and generated module/type names,
- field-number validation, reserved-range validation, duplicate detection,
 generated Silk name-collision validation, generated storage-field collision
 validation, map key validation, enum-zero validation, type-reference
 validation, cross-file import visibility validation, cyclic import detection,
 and targeted rejection of unsupported `extend`, `extensions`, and `group`
 declarations.

Generated modules include:

- Silk structs for messages,
- Silk enums for known constants, raw-preserving `EnumValue` wrappers for enum
 field storage, and numeric conversion helpers for protobuf enum values,
- vector aliases for repeated/map fields,
- `T?` storage for singular message fields and explicit `optional` fields, with
 `None` representing absent fields and `Some(value)` representing presence,
- `empty()`, `drop()`, `encode()`, `decode()`, and `merge_from()` methods for
 messages,
- service RPC descriptor structs plus `<Service>_RPC_COUNT` and
 `<Service>_rpc(index)` metadata helpers,
- raw unknown-field storage slots plus `unknown_fields()` for preserved unknown
 field bytes.

Generated modules type-check, expose their public API surface, build as object
code, and can be imported by Silk programs that construct messages, encode
them, decode them, and inspect decoded fields. The regression suite covers both
direct [`std::protobuf`](/silk/docs/std/protobuf/) wire helpers and an end-to-end generated-message
round-trip executable.

Generated cross-file references use named Silk imports. The generator keeps the
original proto type name when it is unique in the generated module and uses a
deterministic alias only when a name collision would otherwise occur.

## Compatibility

[`std::protobuf`](/silk/docs/std/protobuf/) follows proto3 binary wire-format compatibility rules:

- singular scalar fields default to the proto3 zero value,
- singular message fields use presence, default to `None`, and encode only when
 set to `Some(value)`; repeated wire occurrences merge into the existing
 message payload,
- explicit `optional` fields use `T?` storage and follow `None`/`Some(value)`
 presence,
- enum fields use generated raw-value wrappers, so numeric values unknown to the
 current schema are preserved and re-encoded instead of being coerced to a
 default or discarded,
- repeated primitive and enum fields accept packed and unpacked encodings,
- repeated message, string, and bytes fields accept length-delimited unpacked
 records,
- generated decoders validate a known field's wire type before consuming its
 payload,
- unknown fields are preserved verbatim when generated decoders skip them,
- and unsupported or missing imports are reported by `silk proto` instead of
 being silently ignored.

Generated encoders emit proto3 packed encoding for repeated scalar and enum
fields by default. A field option of `[packed = false]` forces unpacked
encoding. Decoders accept both packed and unpacked encodings for packable fields
so schema changes remain wire-compatible.

## Descriptor JSON

`silk proto --descriptor-out <path>` writes a deterministic JSON document:

- top-level `version` is `1`;
- `files[]` records `path`, `source_path`, `syntax`, `package`, generated
 `module`, `explicit_input`, `imports`, `options`, `messages`, `enums`, and
 `services`;
- message descriptors include generated Silk names, options, reserved names and
 ranges, fields, oneofs, nested messages, and nested enums;
- field descriptors include protobuf name/number/label/type, semantic kind
 (`scalar`, `message`, `enum`, or `map`), resolved full type names when
 applicable, oneof membership, packed-encoding status, map key/value metadata,
 and raw field options;
- enum descriptors include values, aliases as repeated numeric values, options,
 and reserved declarations;
- service descriptors include RPC names, request/response type names, resolved
 full type names, streaming flags, and options.