Binary Serialization Deep Dive

mion's binary serialization is a custom protocol designed specifically for TypeScript applications. Unlike generic binary formats, it leverages compile-time type information to generate highly optimized serializers that support the full range of TypeScript features.

Why Not Protocol Buffers?

Protocol Buffers, MessagePack, and similar formats are excellent general-purpose solutions, but they come with limitations for TypeScript applications:

Since mion controls both client and server serialization, we can optimize the protocol specifically for our use case without needing to maintain compatibility with external systems.

Architecture Overview

The binary serialization system consists of two main components:

1. Type Analysis - reflection system extracts type metadata at compile time
2. JIT Compilation - Serialization functions are generated at runtime based on the type structure

TypeScript Type → reflection → JIT Compiler → Optimized Serializer

Supported TypeScript Features

Primitive Types

All JavaScript primitives are supported with optimized encodings:

type Primitives = {
    str: string;        // Length-prefixed UTF-8
    num: number;        // 64-bit float
    bool: boolean;      // Single byte
    big: bigint;        // String-encoded for precision
    sym: symbol;        // Description as string
    nil: null;          // Single byte marker
    undef: undefined;   // Single byte marker
};

Union Types

Union types are encoded with a discriminator index followed by the value:

type Result = string | number | boolean;
// Binary: [union_index: uint8, value: ...]

type ApiResponse = 
    | { type: 'success'; data: User }
    | { type: 'error'; message: string };
// Discriminated unions use the discriminator property

The JIT compiler generates code that:

1. Determines which union member matches the value
2. Writes the member index (uint8 for ≤255 members, uint16 otherwise)
3. Serializes the value according to its type

Intersection Types

Intersection types are flattened and serialized as a single object:

type Named = { name: string };
type Aged = { age: number };
type Person = Named & Aged;
// Serialized as: { name: string, age: number }

Circular/Recursive Types

mion fully supports circular type definitions:

type TreeNode = {
    value: number;
    children: TreeNode[];
};

type LinkedList = {
    value: string;
    next?: LinkedList;
};

The JIT compiler handles circular references by generating recursive serialization code with proper termination conditions.

Tuples and Rest Parameters

Tuples are serialized as fixed-length arrays:

type Point3D = [number, number, number];
// Binary: [x: float64, y: float64, z: float64]

type LogEntry = [timestamp: Date, level: string, ...messages: string[]];
// Binary: [timestamp: float64, level: string, length: uint32, ...messages]

Rest parameters in function signatures are fully supported:

type Logger = (level: string, ...messages: string[]) => void;

Maps and Sets

Native JavaScript collections are serialized efficiently:

type UserCache = Map<string, User>;
// Binary: [size: uint32, ...entries: [key, value][]]

type Tags = Set<string>;
// Binary: [size: uint32, ...values: string[]]

Classes

Classes are serialized as plain objects, with support for custom serialization:

class User {
    constructor(
        public id: number,
        public name: string,
        public createdAt: Date
    ) {}
}
// Serialized as: { id: number, name: string, createdAt: Date }

Binary Encoding Format

Memory Alignment

All data is encoded into a Uint8Array using patterns optimized for CPU performance:

• 32-bit alignment for integers and floats where possible
• Direct typed array access for fast read/write operations
• Pre-allocated buffers with simple index tracking

Type Encodings

Object Encoding

Objects use a schema-aware encoding that eliminates property names:

Required Properties: [value1, value2, value3, ...]
Optional Properties: [bitmap, present_value1, present_value2, ...]

Required properties are serialized in declaration order without names, since both encoder and decoder know the schema.

Optional properties use a bitmap where each bit indicates whether the corresponding property is present:

type User = {
    id: number;           // Required - always present
    name: string;         // Required - always present
    email?: string;       // Optional - bit 0
    phone?: string;       // Optional - bit 1
    avatar?: string;      // Optional - bit 2
};
// Binary: [id, name, bitmap: uint8, ...present_optionals]

Array Encoding

Arrays are prefixed with their length:

[length: uint32, element1, element2, ...]

Union Encoding

Unions include a discriminator index:

[member_index: uint8|uint16, value: ...]

JIT Compilation

The JIT compiler generates specialized serialization functions for each type. This approach provides:

• Zero runtime type checking - Types are known at compile time
• Inlined operations - No function call overhead for primitives
• Optimized branching - Union type checks are ordered by likelihood

Example: Generated Serializer

For a simple type like:

type Point = { x: number; y: number };

The JIT compiler generates code equivalent to:

// Serialization (simplified)
function serialize(value: Point, buffer: SerContext): void {
    buffer.view.setFloat64(buffer.index, value.x, true);
    buffer.index += 8;
    buffer.view.setFloat64(buffer.index, value.y, true);
    buffer.index += 8;
}

// Deserialization (simplified)
function deserialize(buffer: DesContext): Point {
    const x = buffer.view.getFloat64(buffer.index, true);
    buffer.index += 8;
    const y = buffer.view.getFloat64(buffer.index, true);
    buffer.index += 8;
    return { x, y };
}

Complex Type Example

For a more complex type with optional properties:

type User = {
    id: number;
    name: string;
    email?: string;
    age?: number;
};

The generated serializer handles the optional property bitmap:

// Serialization (simplified)
function serialize(value: User, ser: SerContext): void {
    // Required properties first
    ser.view.setFloat64(ser.index, value.id, true);
    ser.index += 8;
    ser.serString(value.name);
    
    // Initialize bitmap for optional props
    const bitmapIndex = ser.index;
    ser.view.setUint8(ser.index++, 0);
    
    // Optional properties with bitmap
    if (value.email !== undefined) {
        ser.serString(value.email);
        ser.setBitMask(bitmapIndex, 0);
    }
    if (value.age !== undefined) {
        ser.view.setFloat64(ser.index, value.age, true);
        ser.index += 8;
        ser.setBitMask(bitmapIndex, 1);
    }
}

Union Type Example

For union types, the compiler generates type-checking code:

type Result = string | number | { error: string };

// Serialization (simplified)
function serialize(value: Result, ser: SerContext): void {
    if (typeof value === 'string') {
        ser.view.setUint8(ser.index++, 0); // Union index 0
        ser.serString(value);
    } else if (typeof value === 'number') {
        ser.view.setUint8(ser.index++, 1); // Union index 1
        ser.view.setFloat64(ser.index, value, true);
        ser.index += 8;
    } else if (typeof value === 'object' && value !== null) {
        ser.view.setUint8(ser.index++, 2); // Union index 2
        ser.serString(value.error);
    } else {
        throw new Error('Value does not belong to union');
    }
}

Performance Characteristics

Payload Size Comparison

Binary serialization typically produces smaller payloads than JSON:

When to Use Binary

Binary serialization is most effective for:

• Numeric-heavy data - Financial data, sensor readings, coordinates
• Large arrays - Bulk data transfers
• Bandwidth-constrained environments - Mobile apps, IoT devices
• Real-time applications - Gaming, live updates

JSON may be preferable for:

• Debugging - Human-readable format
• Interoperability - Third-party API integration
• Small payloads - Overhead may not justify complexity

Security Considerations

The binary serializer includes protections against common attacks:

• Prototype pollution protection - Property names are validated during deserialization
• Buffer bounds checking - Prevents buffer overflow attacks
• Type validation - Values are validated against expected types

Conclusion

mion's binary serialization provides a powerful, type-safe alternative to JSON for TypeScript applications. By leveraging compile-time type information and JIT compilation, it achieves excellent performance while supporting the full range of TypeScript features that other binary formats cannot handle.

The key advantages are:

1. No schema files - Your TypeScript types are the schema
2. Full TypeScript support - Unions, intersections, generics, and more
3. Optimized encoding - Type-aware serialization eliminates redundancy
4. JIT compilation - Runtime-generated serializers for maximum performance