13 · Sharded distribution

Weft

Deterministic 4 MiB chunking, a signed binary Merkle manifest, and parallel fetch with per-chunk verification. Weft is what lets MARS-anchored artifacts and Sigil-committed datasets cross the P2P fabric without trusting any single peer. Records the DHT serves: /weft/v1/manifest, /weft/v1/chunk, /weft/v1/availability.

Concepts

What you’re working with.

  • 01Deterministic 4 MiB chunking — tunable from 1 byte to 64 MiB.
  • 02BLAKE3 chunk IDs + a deterministic binary Merkle tree.
  • 03Signed manifests on the weave.weft.manifest.v1 schema.
  • 04Parallel fetch with per-chunk verification — bad bytes never reach the app.
  • 05DHT records: /weft/v1/manifest/<root>, /weft/v1/chunk/<id>, /weft/v1/availability/<root>.
  • 06Pairs with Sigil + MARS — anchored artifacts ship over the P2P fabric.
Code

A handful of lines.

use weft::{Node, ManifestBuilder, FetchConfig};

let node = Node::open(".weave/weft").await?;

// Shred a multi-GB dataset, sign the manifest, publish to the DHT.
let manifest = ManifestBuilder::new()
    .chunk_size(4 * 1024 * 1024)
    .from_stream(tokio::fs::File::open(&path).await?)
    .await?
    .sign(&keypair);
let root = node.publish(manifest).await?;

// Anywhere on the network — parallel fetch, per-chunk verify.
let bytes = node.fetch(&root, FetchConfig::parallel(8)).await?;

Ready to build?

Start with the SDK or load the agent context. Every library is independently documented and versioned.