`bin`

Binary data - u32 lane arithmetic, big/little-endian integer encoding, row-based pack/unpack codec, memory views, and incremental stream reading.

Load with: use bin

What this module does

bin is the foundation for any code that works with raw binary data - network packets, file formats, hardware protocols, or byte-level codecs.

It operates in the u32 lane: integers are clamped to the range 0–4294967295 before bitwise operations, matching common C/hardware unsigned 32-bit semantics. On top of that it gives you: big-endian and little-endian encoding/decoding for u8, u16, u32, i16, and i32; a row-based pack/unpack codec driven by tagged field descriptors; O(1) substring windows via memory views; and an incremental stream reader for parsing from a stream without loading everything into memory.

Quick example

use bin

# Encode a struct as big-endian bytes
msg = binrp([
    {t: "u8",    v: 0xFF},
    {t: "u16be", v: 1200},
    {t: "u32be", v: 305419896},
    {t: "zs",    v: "hello"},
    {t: "pad",   n: 3}
])

# Decode it back with the same spec
vals = binru1(msg, [
    {t: "u8"},
    {t: "u16be"},
    {t: "u32be"},
    {t: "zs"},
    {t: "pad", n: 3}
])
prn(vals)  # [255, 1200, 305419896, "hello", 0]

# Memory view - O(1) substring window
v = binva(msg)
prn(binv8(v, 0))  # 255  (first byte)

Functions

u32 lane and bitwise

binu3(n)

Clamps any integer to the u32 lane (0–4294967295). Adds or subtracts 4294967296 until in range.

binan(a, b) / binxo(a, b) / binor(a, b) / binno(a)

AND, XOR, OR, bitwise NOT - all operating in the u32 lane.

binsl(a, b) / binsr(a, b)

Left shift and right shift in the u32 lane.

binbt(a, i)

Tests bit at position i. Returns 1 if set, 0 if clear.

binrl(a, k) / binrr(a, k)

Rotate left / rotate right by k bits in the u32 lane. Normalises k to 0–31.

binmk(w)

Creates a bitmask of w low bits: e.g. binmk(8) → 0xFF. Returns 0 for w ≤ 0 and 4294967295 for w ≥ 32.

binzx(v, w)

Zero-extends - masks off all bits above position w: v AND binmk(w).

Encoding integers to bytes

bin6b(n) / bin6l(n)

Encode unsigned 16-bit integer as 2 bytes, big-endian (bin6b) or little-endian (bin6l). Errors if n is out of 0–65535 range.

bin3b(u) / bin3l(u)

Encode unsigned 32-bit integer as 4 bytes, big-endian (bin3b) or little-endian (bin3l). Clamps to u32 lane first.

bini6(n) / bini61(n)

Encode signed 16-bit integer (i16) as 2 bytes, big-endian / little-endian. Two's complement. Errors if outside −32768–32767.

bini3(n) / bini31(n)

Encode signed 32-bit integer (i32) as 4 bytes, big-endian / little-endian. Converts negative values to unsigned via + 4294967296 then encodes.

bini6l(n)

Encode a 64-bit integer as 8 bytes little-endian. Splits into low and high u32 halves.

bini8n(n) / bin.i8new(n)

Encode a single unsigned byte (0–255) as a 1-character string. Errors if out of range.

Decoding bytes to integers

binde(s, p) / binde1(s, p)

Decode u16 big-endian / little-endian from string s at position p. Returns {v, pos} where pos is p + 2.

binde2(s, p) / binde3(s, p)

Decode u32 big-endian / little-endian from string s at position p. Returns {v, pos} where pos is p + 4.

bini3a(s, i) / bin.i32at(s, i)

Read a u32 big-endian from string s at byte offset i. Errors if offset + 4 > len(s).

bini6a(s, i) / bin.i64at(s, i)

Read a u64 little-endian (as two u32 halves) from string s at byte offset i.

bini8a(s, i) / bin.i8at(s, i)

Read a single byte at position i as an unsigned integer.

Row pack / unpack codec

Row encoding drives a list of {t, v, ...} field descriptors to pack structured binary data. Supported tags: "u8", "u16be", "u16le", "u32be", "u32le", "i16be", "i16le", "i32be", "i32le", "zs" (null-terminated string), "raw" (fixed-length bytes), "pad" (zero padding).

binrp(rows)

Pack a list of {t, v} (or {t, n} for pad) rows into a binary string. Each row type maps to its byte encoding.

binru(s, specs, pos)

Unpack a binary string s starting at pos using a list of type specs. Returns {vals, pos}. For "zs" reads until a null byte; for "raw" reads spec.ln bytes; for "pad" skips spec.n bytes.

binru1(s, specs)

Like binru starting at position 0 - errors if the string has trailing bytes after all specs are consumed. Returns just vals.

binpk(rows, o)

Pack an ilusm object o into bytes using an ordered list of {nm, t, ...} rows. Each row's nm field is used to look up the value from o.

binup(rows, s, pos)

Unpack bytes from s at pos into an object, keyed by the nm fields in rows. Returns {o, pos}.

binup1(rows, s)

Like binup at position 0 - errors if trailing bytes remain. Returns the object directly.

binro(rows, s, pos)

Incremental parse - reads exactly one field (the first row) from rows at position pos in s. Returns {nm, v, pos}.

Memory views

A memory view is an O(1) window into an immutable byte string - no copying, just offsets.

binvm(dat, off, ln)

Creates a view into string dat starting at byte offset off with length ln. Errors if the window is out of bounds.

binva(s) / bin.vall(s)

Creates a view covering the entire string s.

binv8(v, i)

Reads the unsigned byte at view-relative index i. Errors if out of view bounds.

binvs(v) / bin.tostr(v) / bin.binvall(v)

Materialises the view to a string substring.

binvd(v, n)

Returns a new view with the first n bytes dropped (advance).

binvt(v, n)

Splits a view at position n. Returns {hd, tl} - head view of n bytes and tail view of the remainder.

Stream reading

binst(r, n)

Reads exactly n bytes from stream reader r. Errors with "bin: stm rd" if EOF is reached before n bytes are available.

binsr1(r, sp)

Reads one field from a stream using a type spec {t, ...}. Supports all row tags - for "zs" reads byte by byte until a null. For "pad" reads and discards the bytes, returns 0.

binsu(r, rows)

Reads a full message from a stream using an ordered {nm, t, ...} row list. Returns the decoded object.

Schema integration

binsc(pl)

Converts a schema binary plan (from sch.schbinplan) into row specs. "num" hints become "u32be" rows; "str" hints become "zs" rows. Lists, unions, and schema references are rejected.

binvl(o, sch)

Validates every field of object o against a schema using sch.schvalidatefieldval. Returns o on success.

Notes

All bitwise operations work in the u32 lane (0–4294967295). This matches C uint32_t semantics.
The bin object at the end of the module exports: bin.i64le, bin.i32at, bin.i64at, bin.i16be, bin.tostr, bin.i32le, bin.i8new, bin.i8wro, bin.i8at, bin.f64at, bin.i32be, bin.vall, bin.binrd, bin.binvall.
bin.f64at always errors - it requires a host native __bin_f64_at that is not yet bound.
Requires trl, txt, enc, sch, and strm.