SPEC — omni export-git / omni import-git

Status: Accepted · implemented · reversible git round-trip of the omni content-addressed store.

1. Purpose

Let the whole omni store (or a chosen set of artifacts) be written into a real git repository and read back byte-identically. export-git and import-git are inverses:

import-git( export-git( store ) )  ==  store        # every blob + manifest identical

This gives git's history/distribution/tooling over omni's content without turning omni into git. It is opt-in and one-directional at call time; the live store stays a flat CAS (git-guard still applies to omni artifacts — the export git is a separate, ordinary repo).

2. What is exported — the STORE, not materialized artifacts

Export writes omni's objects and manifests verbatim, NOT a checked-out tree of the artifacts. This is deliberate: - It round-trips exactly (the bytes are the omni objects; import re-verifies each by sha256). - It sidesteps the "git can't hold a socket / loses xattrs / nests git-in-git" problems — special file types, modes, and xattrs live in the manifest, not as real filesystem objects in the tree. - Dedup and self-heal come for free: import goes through put_blob (re-verifies, heals rot).

Addressing is preserved by encoding omni's sha256 in the git tree path — no separate sha-map is required for round-trip. (A git-native omni-sha256 → git-oid map is intentionally not shipped — see §8/Q2: one ref per object would explode the ref namespace; if that lookup is ever wanted, it belongs in a single git-notes ref or an OMNI-STORE.json map, never N refs.)

3. On-git layout (the committed tree)

objects/<sha[:2]>/<sha[2:]>      # one git-tracked file per omni blob, byte-for-byte
manifests/<sha[:2]>/<sha[2:]>    # one git-tracked file per manifest, byte-for-byte
OMNI-STORE.json                  # export metadata: format version, store digest, counts, ts

- OMNI-STORE.json records {version, artifacts: [addr...], blob_count, manifest_count, store_digest} where store_digest = sha256 over the sorted list of contained shas — a single value that identifies the exported store state (used by verification + idempotency). - The tree mirrors omni's own objects//manifests/ fan-out, so git dedups identical blobs across commits by content, and a human can diff two store snapshots.

4. Commit model

- Each export is one commit on a ref (default branch omni-store), so history = store snapshots over time. export-git <dir> inits the repo if absent. - Deterministic tree ordering; commit message carries store_digest, artifact count, and blob/ manifest counts. Re-exporting an unchanged store is a no-op (same tree → git makes no commit). - Export is incremental: only new objects enter the pack; unchanged blobs are already present.

5. CLI

omni export-git <git-dir> [<addr> ...] [--branch NAME] [--pack]
    Export the whole store, or just the listed artifacts + the transitive closure of the blobs they
    reference, into the git repo at <git-dir> (init if absent) as a commit on --branch (default
    omni-store). --pack runs `git gc` at the end for a distribution-ready packed repo (default leaves
    loose objects — faster incremental re-export). Streams (peak mem ~OMNI_CHUNK_BYTES).

omni import-git <git-dir> [--ref REF]
    Reconstruct / merge into the current store ($OMNI_ROOT) from <git-dir> at --ref (default
    omni-store). Every object is VERIFIED on the way in (put_blob re-hashes); import is additive and
    idempotent — importing into an existing store dedups+heals. Streams.

Exit non-zero + a named error if the git tree references a blob it does not contain, or a blob fails its sha256 on import (never a silent partial store).

6. Reuse (do NOT rebuild)

- Enumerate: Store.manifest_shas(), blobs_referenced_by(manifest); whole-store closure = all manifests ∪ all objects under objects/. - Read/write streamed: get_blob_to_path(sha, dest) (blob → git working file) and put_file(path) / put_blob(data) (git file → store) — both already chunk at OMNI_CHUNK_BYTES, so multi-GB blobs round-trip within the RAM cap. - Record manifests via the store's _record_manifest path; run fsck after import as the integrity gate. - git plumbing via the existing _git() helper (hash-object/fast-import optional for the git-native map).

7. Acceptance / verification (each pin has a RED negative control)

1. Round-trip fidelity. Build a store with one of each kind (file, folder, tar.gz, git repo, image); export-git → fresh dir; import-git into a fresh $OMNI_ROOT; assert every artifact address resolves and pull is byte-identical to the original, and store_digest matches. Neg control: an export that omits one referenced blob → import fails loudly (missing-blob error), never a store that lists an address whose pull would refuse. 2. Verify-on-import. Flip one byte of an objects/** file in the git tree → import-git raises CorruptObject, nothing enters the store. Neg control: a naive import that trusts the filename stores the rot (proves the re-hash is load-bearing). 3. Idempotent / incremental. Re-export-git an unchanged store → no new commit and no new git objects; re-import-git → same store, object count unchanged. Neg control: an export that re-mints unchanged blobs grows the pack. 4. Bounded memory. Export+import of a ≥1 GiB blob run with peak RSS ~OMNI_CHUNK_BYTES. Neg control: a whole-file read path OOMs under a tight ulimit/MemoryMax. 5. git-guard interplay. The export dir is a normal git repo (git-guard does NOT flag it); an omni artifact directory still refuses git as before. Neg control: export writing an OMNI-ADDRESS.txt guard marker into the export repo (must not).

8. Resolved decisions

- Q1 — branch name: omni-store (kept, overridable via --branch). RFC-0003's <kind>/<id> rule (§ "git branches") is the RFC-0001 build-ticket convention for ephemeral ticket//landed/ branches. A store mirror is a persistent long-lived ref like main, not a per-work-item branch, so the <kind>/<id> rule does not apply; omni-store is valid kebab-case and descriptive. - Q2 — no git-native ref map (rejected for v1). One refs/omni/<sha256> per object would add thousands of refs and cripple git. Round-trip needs no map (the tree path IS the omni address). If address→git-oid lookup is ever wanted, use a single git-notes ref or an OMNI-STORE.json map — never N refs. - Q3 — selective export = artifact + transitive BLOB closure, not parents. export-git <addr> emits the manifest object plus every blob it references (blobs_referenced_by, recursive over BLOB_POINTER_KEYS). It does NOT pull in parent manifests that happen to reference the same blobs — those are separate artifacts, exported explicitly if wanted. omni manifests reference blobs (not sub-manifests), so the closure is self-contained (proven by the git-kind round-trip pin, the most complex manifest). - Q4 — loose by default, --pack opt-in. Default leaves loose objects (fast incremental re-export); --pack runs git gc at the end for a smaller, distribution-ready repo.

9. Non-goals

Not a live sync (it is a snapshot at call time). Not a replacement for omni serve/pull (those stay the live transport). Does not change omni addressing or the store format.