Skip to main content

Serialization

syft-flwr uses multiple serialization mechanisms to transmit data securely across the federated learning network.

Serialization Layers​

syft-rpc Serialization​

Low-level object serialization protocol:

  • Converts Python objects to/from bytes
  • Supports complex data types
  • Type validation and security
  • UTF-8 encoding support

Supported Types:

  • Pydantic models
  • Dataclasses
  • Standard Python types (dict, list, str, int, float, etc.)
  • NumPy arrays
  • PyTorch tensors (via special handlers)

Flower Serialization​

Flower's native serialization for FL:

  • Model parameters (weights and biases)
  • Training metrics
  • Configuration parameters
  • Custom messages

File-Based Serialization​

For larger objects stored in SyftBox:

  • Model checkpoints
  • Training datasets (references, not data)
  • Results and logs
  • Configuration files

Serialization Process​

Model Updates​

┌─────────────┐        ┌──────────────┐        ┌─────────────┐
│   Model     │        │  Serialize   │        │ SyftBox     │
│  Parameters │───────▶│  (syft-rpc)  │───────▶│ File Sync   │
│  (Client)   │        │              │        │             │
└─────────────┘        └──────────────┘        └─────────────┘

Data Flow​

  1. Client → Server

    • Model parameters serialized using syft-rpc
    • Written to SyftBox datasite
    • Synced to server via SyftBox protocol

  2. Server → Client

    • Aggregated model serialized
    • Written to server's datasite
    • Synced to clients via SyftBox protocol

Security Considerations​

Data Privacy​

  • Raw training data is NEVER serialized or transmitted
  • Only model updates (gradients/weights) are shared
  • Differential privacy can be applied to updates

Encryption​

  • End-to-end encryption via syft-crypto
  • Encrypted serialization for sensitive metadata
  • Secure key exchange using X3DH protocol

Validation​

  • Type checking on deserialization
  • Schema validation for RPC messages
  • Integrity checks for model updates

Custom Serialization​

For custom data types:

# Example: Custom serializer registration
from syft_rpc import register_serializer

@register_serializer(CustomType)
def serialize_custom(obj: CustomType) -> bytes:
    # Custom serialization logic
    return obj.to_bytes()

@register_deserializer(CustomType)
def deserialize_custom(data: bytes) -> CustomType:
    # Custom deserialization logic
    return CustomType.from_bytes(data)
note

Complete serialization API documentation coming soon

See Also​