malt

module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 2, 2026 License: MIT

README

MALT

Go CI

MALT is an authenticated mutable structure layer over immutable content-addressed storage.

Immutable payload bytes still live in ordinary CAS blocks and keep ordinary CIDs. MALT adds a verifiable structure layer above those payload CIDs, so a client can ask for a path or range, receive result + ProofList, and verify that answer against a structure root without trusting the daemon, cache, or materialized index state.

Why This Exists

Traditional Merkle-DAG traversal authenticates structure by embedding child links in parent content. A local structural change can force rootward object rewrites because the relationship and the object identity are coupled.

MALT separates those concerns:

  • payload content remains immutable CAS data
  • mutable structure is authenticated by independent structure roots
  • list/map semantics define typed read and write operations
  • reads return verifier-facing ProofList evidence
  • local structure updates advance structure roots without rewriting unrelated payload blocks

The claim is not that updates are free. The claim is that MALT replaces implicit ancestor-rewrite cost with explicit, verifiable structure maintenance.

Current Status

MALT is a research prototype and Go implementation. It is suitable for experimentation, evaluator work, and design review. It is not yet a production storage service, and several schemas and policies are intentionally still open.

Current in-tree capabilities:

  • root-centric malt CLI for local daemon lifecycle, add, resolve, and verify
  • proof-bearing HTTP reads for file bytes, directory JSON, and byte ranges
  • pure MALT UnixFS-style layout built from map/list semantics and CAS blobs
  • stateless commitment backends for semantic proof primitives
  • ArcTable-backed structure materialization with overwrite and versioned modes
  • malt-eval workloads for read queries, write traces, CAS models, proof overhead, and storage overhead

Known non-goals for the current prototype:

  • no managed global head publication service
  • no multi-writer merge or freshness protocol
  • no tenant, quota, pinning, or garbage-collection policy
  • no stable public API compatibility guarantee yet
  • response-body binding for large-file byte ranges is still a ProofList-schema design item

Quick Start

Prerequisites:

  • Go 1.25.7 or newer
  • Git

Build the two local binaries:

mkdir -p bin
go build -buildvcs=false -o bin/malt ./cmd/malt
go build -buildvcs=false -o bin/malt-eval ./cmd/eval/malt-eval

Initialize the local runtime. The default configuration uses a local embedded mock CAS at 127.0.0.1:4318 and a daemon API at 127.0.0.1:4317.

bin/malt init --non-interactive
bin/malt start
bin/malt status

Add a file, resolve it, and verify the returned ProofList:

printf 'hello malt\n' >/tmp/malt-hello.txt
ROOT=$(bin/malt add /tmp/malt-hello.txt | awk '/Result root:/ {print $3}')
bin/malt resolve "$ROOT" malt-hello.txt >/tmp/malt-resolve.json
bin/malt verify --prooflist /tmp/malt-resolve.json

Stop the managed daemon when finished:

bin/malt stop

Developer Workflow

Run the full Go validation suite from the repository root:

go test ./...
go vet ./...

Inspect the command surfaces:

bin/malt --help
bin/malt-eval --help
bin/malt-eval schema

Run the local smoke evaluation plan:

bin/malt-eval run --plan examples/eval-smoke-plan.json --run-id smoke

The evaluator writes disposable workspace state under output/<run_id> and durable result artifacts under result/<run_id>. Those directories are ignored by git.

Core Model

MALT's current implementation is easiest to read through these layers:

Layer Role
Semantic layer Abstract list/map read and write semantics
ArcTable Namespace-scoped arcset persistence and materialization
Commitment backend Stateless proof primitives over semantic representations
Graph ports Resolver read path and writer mutation path
Server API Runtime surface for daemon HTTP routes
Application layout Product data model above list/map/CAS blobs

The verifier-facing shape is:

Read(root, query) -> result + ProofList
VerifyRead(root, query, result, ProofList) -> valid / invalid

ApplyMutation(baseRoot, semantic mutation) -> newRoot + writeReceipt

list describes stable-indexed child references. map describes authenticated key-to-target relations and reserves @payload as the terminal materialization binding for map semantic objects. Layouts translate source-domain data into semantic mutations; they do not define the core semantics.

For a deeper implementation walkthrough, see ARCHITECTURE.md.

Repository Layout

cmd/malt/                      primary runtime CLI
cmd/eval/                      malt-eval workloads, schemas, and helpers
api/http/                      daemon request/response DTOs
auth/                          arcset, commitment, proof, list/map interfaces
graph/                         resolver and writer graph ports
layout/unixfs/                 UnixFS-style layout over map/list/CAS blobs
runtime/                       node, graph, ArcTable, metrics, implementations
sdk/client/                    Go daemon client facade
server/                        daemon HTTP server
storage/                       CAS and KV storage libraries
wire/maltcid/                  MALT map/list root CID codecs
docs/                          public contributor and maintainer docs
examples/                      small runnable plans and examples

Evaluation

malt-eval supports both direct commands and a framework runner:

  • malt-eval read emits read-query JSONL records for MALT and IPLD UnixFS baselines
  • malt-eval write replays Git traces and emits write-amplification JSONL
  • malt-eval run executes JSON plans and writes manifest.json, raw envelopes, and summary CSVs under result/<run_id>
  • malt-eval schema lists or prints embedded JSON schemas
  • malt-eval summarize regenerates summary CSVs from a result directory
  • malt-eval metrics inspects daemon evaluation metrics

See docs/evaluation.md for commands, result layout, and schema notes.

Roadmap

The near-term roadmap is focused on stabilizing proof schemas, evaluator outputs, and the UnixFS-style layout boundary before claiming production readiness. See ROADMAP.md.

Contributing

Contributions are welcome once the repository is public. Start with CONTRIBUTING.md, keep changes small, and include focused tests for behavior changes.

Please report security issues through the private process in SECURITY.md, not through public issues.

License

MALT is released under the MIT License.

Directories

Path Synopsis
api
http
Locked path stat/content HTTP semantics.
 Locked path stat/content HTTP semantics.
auth
arcset
Package arcset defines interfaces for arc set views.
 Package arcset defines interfaces for arc set views.
commitment
Package commitment defines cryptographic commitment interfaces.
 Package commitment defines cryptographic commitment interfaces.
commitment/ipa
Package ipa provides an IPA (Inner Product Argument) commitment backend.
 Package ipa provides an IPA (Inner Product Argument) commitment backend.
commitment/kzg
Package kzg provides a KZG polynomial commitment backend.
 Package kzg provides a KZG polynomial commitment backend.
proof/evidence
Package evidence defines the Evidence interface and its implementations.
 Package evidence defines the Evidence interface and its implementations.
proof/prooflist
Package prooflist defines the verifier-facing read proof transcript shape.
 Package prooflist defines the verifier-facing read proof transcript shape.
semantic
Package structure defines the public structural semantics layer for MALT.
 Package structure defines the public structural semantics layer for MALT.
semantic/list
Package list defines the public stable-indexed list semantic for MALT.
 Package list defines the public stable-indexed list semantic for MALT.
semantic/mapping
Package mapping defines the public keyed-map semantic for MALT.
 Package mapping defines the public keyed-map semantic for MALT.
cmd
eval/command
Package command assembles the unified MALT evaluation CLI.
 Package command assembles the unified MALT evaluation CLI.
eval/helper/adapters/hamt
Package hamt configures the IPLD UnixFS HAMT baseline adapter.
 Package hamt configures the IPLD UnixFS HAMT baseline adapter.
eval/helper/adapters/maltflat
Package maltflat replays Git snapshots into the MALT-flat UnixFS layout.
 Package maltflat replays Git snapshots into the MALT-flat UnixFS layout.
eval/helper/adapters/merkledag
Package merkledag replays Git snapshots into IPLD UnixFS Merkle DAGs.
 Package merkledag replays Git snapshots into IPLD UnixFS Merkle DAGs.
eval/helper/evalmetrics
Package evalmetrics provides daemon evaluation metrics commands.
 Package evalmetrics provides daemon evaluation metrics commands.
eval/helper/evalread
Package evalread provides the read-latency evaluation command.
 Package evalread provides the read-latency evaluation command.
eval/helper/evalrun
Package evalrun provides the unified evaluation plan runner command.
 Package evalrun provides the unified evaluation plan runner command.
eval/helper/evalschema
Package evalschema exposes evaluator JSON schemas through the CLI.
 Package evalschema exposes evaluator JSON schemas through the CLI.
eval/helper/evalsuites
Package evalsuites assembles the suite registry used by malt-eval run.
 Package evalsuites assembles the suite registry used by malt-eval run.
eval/helper/evalsummary
Package evalsummary provides the evaluation summary command.
 Package evalsummary provides the evaluation summary command.
eval/helper/evalwrite
Package evalwrite provides the Git trace write-amplification evaluation command.
 Package evalwrite provides the Git trace write-amplification evaluation command.
eval/helper/git
Package git extracts deterministic commit-mutation traces from Git repos.
 Package git extracts deterministic commit-mutation traces from Git repos.
eval/helper/replay
Package replay defines write-amplification replay contracts and JSONL output.
 Package replay defines write-amplification replay contracts and JSONL output.
eval/helper/store
Package store provides per-system evaluation storage and write accounting.
 Package store provides per-system evaluation storage and write accounting.
eval/internal/baseline/indexedmap
Package indexedmap implements the canonical-path ordered baseline map semantic for eval comparison.
 Package indexedmap implements the canonical-path ordered baseline map semantic for eval comparison.
eval/internal/compat/hamt
Package hamt implements the Step interface for HAMT (Hash Array Mapped Trie).
 Package hamt implements the Step interface for HAMT (Hash Array Mapped Trie).
eval/internal/compat/implicit
Package implicit implements the Step interface for Merkle-DAG traversal.
 Package implicit implements the Step interface for Merkle-DAG traversal.
eval/internal/compat/implicit/codec
Package codec defines the IPLD codec interface for parsing blocks and extracting links.
 Package codec defines the IPLD codec interface for parsing blocks and extracting links.
eval/internal/compat/implicit/dagcbor
Package dagcbor implements the dag-cbor IPLD codec.
 Package dagcbor implements the dag-cbor IPLD codec.
eval/internal/compat/implicit/dagjson
Package dagjson implements the dag-json IPLD codec.
 Package dagjson implements the dag-json IPLD codec.
eval/internal/compat/implicit/dagpb
Package dagpb implements the dag-pb (Protocol Buffers) IPLD codec.
 Package dagpb implements the dag-pb (Protocol Buffers) IPLD codec.
eval/internal/eval/framework
eval/internal/eval/readbench
Package readbench provides read benchmark runners for MALT and IPLD UnixFS baselines.
 Package readbench provides read benchmark runners for MALT and IPLD UnixFS baselines.
eval/internal/eval/suites/casmodel
Package casmodel measures the local CAS mock under fixed latency models.
 Package casmodel measures the local CAS mock under fixed latency models.
eval/internal/eval/suites/configjson
Package configjson decodes evaluation suite config JSON consistently.
 Package configjson decodes evaluation suite config JSON consistently.
eval/internal/eval/suites/proofoverhead
Package proofoverhead measures local commit/prove/verify costs for semantic structures.
 Package proofoverhead measures local commit/prove/verify costs for semantic structures.
eval/internal/eval/suites/readquery
Package readquery adapts the read benchmark runner to the evaluation framework.
 Package readquery adapts the read benchmark runner to the evaluation framework.
eval/internal/eval/suites/storageoverhead
Package storageoverhead measures persisted and logical storage overhead.
 Package storageoverhead measures persisted and logical storage overhead.
eval/internal/eval/suites/writetrace
Package writetrace runs Git write-trace replay as an eval framework suite.
 Package writetrace runs Git write-trace replay as an eval framework suite.
eval/internal/eval/summary
Package summary converts framework raw result envelopes into figure CSVs.
 Package summary converts framework raw result envelopes into figure CSVs.
eval/malt-eval command
Package main provides the unified MALT evaluation CLI.
 Package main provides the unified MALT evaluation CLI.
eval/schemas
Package schemas embeds the evaluator JSON schemas for installed binaries.
 Package schemas embeds the evaluator JSON schemas for installed binaries.
internal/merkledagimport
Package merkledagimport imports local UnixFS data into an IPFS-style Merkle DAG.
 Package merkledagimport imports local UnixFS data into an IPFS-style Merkle DAG.
malt command
Package main provides the primary MALT CLI.
 Package main provides the primary MALT CLI.
Package config provides configuration loading and persistence for MALT.
 Package config provides configuration loading and persistence for MALT.
Package daemon hosts the shared local daemon bootstrap used by CLI entrypoints.
 Package daemon hosts the shared local daemon bootstrap used by CLI entrypoints.
querypath
Package querypath implements locked path canonicalization for stat and content APIs.
 Package querypath implements locked path canonicalization for stat and content APIs.
resolver
Package resolver implements the MALT explicit-arc resolution loop with prefix consumption.
 Package resolver implements the MALT explicit-arc resolution loop with prefix consumption.
resolver/step
Package step defines the Step interface for single-step resolution.
 Package step defines the Step interface for single-step resolution.
resolver/step/explicit
Package explicit implements the Step interface for MALT explicit arcs.
 Package explicit implements the Step interface for MALT explicit arcs.
writer
Package writer implements graph semantic mutations.
 Package writer implements graph semantic mutations.
layout
unixfs
Package unixfs implements a MALT-native UnixFS-style layout directly on top of the map and list structural semantics.
 Package unixfs implements a MALT-native UnixFS-style layout directly on top of the map and list structural semantics.
unixfs/internal/format
Package format defines UnixFS layout-specific format constants.
 Package format defines UnixFS layout-specific format constants.
unixfs/internal/manifest
Package manifest implements the locked directory manifest JSON used by the UnixFS application layout.
 Package manifest implements the locked directory manifest JSON used by the UnixFS application layout.
Package logger provides a structured logging interface for MALT.
 Package logger provides a structured logging interface for MALT.
runtime
arctable
Package arctable defines the Explicit Arc Table interface and implementations.
 Package arctable defines the Explicit Arc Table interface and implementations.
arctable/bloom
Package bloom provides Bloom Filter implementations for ArcTable.
 Package bloom provides Bloom Filter implementations for ArcTable.
arctable/overwrite
Package overwrite provides an ArcTable implementation with overwrite semantics.
 Package overwrite provides an ArcTable implementation with overwrite semantics.
arctable/versioned
Package versioned provides a versioned ArcTable implementation using a KVStore.
 Package versioned provides a versioned ArcTable implementation using a KVStore.
graph
Package runtimegraph provides concrete graph lifecycle management for MALT.
 Package runtimegraph provides concrete graph lifecycle management for MALT.
metrics
node
Package node provides the application-level API for MALT.
 Package node provides the application-level API for MALT.
semantic/list/tree
Package tree implements the stable-indexed list semantic using a tree-shaped fixed-slot layout.
 Package tree implements the stable-indexed list semantic using a tree-shaped fixed-slot layout.
semantic/list/tree/internal
semantic/mapping/radix
Package radix implements a digest-keyed radix-map semantic above the primitive index commitment backends.
 Package radix implements a digest-keyed radix-map semantic above the primitive index commitment backends.
sdk
client
Package client provides a thin HTTP client for the local MALT daemon.
 Package client provides a thin HTTP client for the local MALT daemon.
Package server provides the MALT daemon HTTP API.
 Package server provides the MALT daemon HTTP API.
storage
cas
Package cas provides Content Addressable Storage clients.
 Package cas provides Content Addressable Storage clients.
cas/httpapi
cas/ipfs
Package ipfs provides a CAS client that communicates with a local IPFS daemon via its API, supporting both read and write operations.
 Package ipfs provides a CAS client that communicates with a local IPFS daemon via its API, supporting both read and write operations.
cas/mock
Package mock provides a mock CAS implementation for testing.
 Package mock provides a mock CAS implementation for testing.
kv
Package kvstore defines a key-value store interface for MALT.
 Package kvstore defines a key-value store interface for MALT.
kv/badger
Package badger provides a BadgerDB implementation of kvstore.KVStore.
 Package badger provides a BadgerDB implementation of kvstore.KVStore.
kv/fs
Package fs provides a filesystem-based implementation of kvstore.KVStore.
 Package fs provides a filesystem-based implementation of kvstore.KVStore.
kv/memory
Package memory provides an in-memory implementation of kvstore.KVStore.
 Package memory provides an in-memory implementation of kvstore.KVStore.
wire
maltcid
Package maltcid defines MALT-specific multicodec constants and CID utilities.
 Package maltcid defines MALT-specific multicodec constants and CID utilities.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.