SPEC — omni export-git / omni import-git

Status: Draft · reversible git round-trip of the omni content-addressed store.

1. Purpose

Let the whole omni store (or a chosen set of artifacts) be written into a real git repository and read back byte-identically. export-git and import-git are inverses:

import-git( export-git( store ) )  ==  store        # every blob + manifest identical

This gives git's history/distribution/tooling over omni's content without turning omni into git. It is opt-in and one-directional at call time; the live store stays a flat CAS (git-guard still applies to omni artifacts — the export git is a separate, ordinary repo).

2. What is exported — the STORE, not materialized artifacts

Export writes omni's objects and manifests verbatim, NOT a checked-out tree of the artifacts. This is deliberate: - It round-trips exactly (the bytes are the omni objects; import re-verifies each by sha256). - It sidesteps the "git can't hold a socket / loses xattrs / nests git-in-git" problems — special file types, modes, and xattrs live in the manifest, not as real filesystem objects in the tree. - Dedup and self-heal come for free: import goes through put_blob (re-verifies, heals rot).

Addressing is preserved by encoding omni's sha256 in the git tree path — no separate sha-map is required for round-trip. (An optional refs/omni/<sha256> → git-oid mapping MAY be emitted for callers that want to resolve omni addresses to git objects directly; it is best-effort and never gates.)

3. On-git layout (the committed tree)

objects/<sha[:2]>/<sha[2:]>      # one git-tracked file per omni blob, byte-for-byte
manifests/<sha[:2]>/<sha[2:]>    # one git-tracked file per manifest, byte-for-byte
OMNI-STORE.json                  # export metadata: format version, store digest, counts, ts

- OMNI-STORE.json records {version, artifacts: [addr...], blob_count, manifest_count, store_digest} where store_digest = sha256 over the sorted list of contained shas — a single value that identifies the exported store state (used by verification + idempotency). - The tree mirrors omni's own objects//manifests/ fan-out, so git dedups identical blobs across commits by content, and a human can diff two store snapshots.

4. Commit model

- Each export is one commit on a ref (default branch omni-store), so history = store snapshots over time. export-git <dir> inits the repo if absent. - Deterministic tree ordering; commit message carries store_digest, artifact count, and blob/ manifest counts. Re-exporting an unchanged store is a no-op (same tree → git makes no commit). - Export is incremental: only new objects enter the pack; unchanged blobs are already present.

5. CLI

omni export-git <git-dir> [<addr> ...] [--branch NAME] [--ref-map]
    Export the whole store, or just the listed artifacts + the transitive closure of the blobs they
    reference, into the git repo at <git-dir> (init if absent) as a commit on --branch (default
    omni-store). --ref-map also emits refs/omni/<sha256> -> git-oid. Streams (peak mem ~OMNI_CHUNK_BYTES).

omni import-git <git-dir> [--ref REF]
    Reconstruct / merge into the current store ($OMNI_ROOT) from <git-dir> at --ref (default
    omni-store). Every object is VERIFIED on the way in (put_blob re-hashes); import is additive and
    idempotent — importing into an existing store dedups+heals. Streams.

Exit non-zero + a named error if the git tree references a blob it does not contain, or a blob fails its sha256 on import (never a silent partial store).

6. Reuse (do NOT rebuild)

- Enumerate: Store.manifest_shas(), blobs_referenced_by(manifest); whole-store closure = all manifests ∪ all objects under objects/. - Read/write streamed: get_blob_to_path(sha, dest) (blob → git working file) and put_file(path) / put_blob(data) (git file → store) — both already chunk at OMNI_CHUNK_BYTES, so multi-GB blobs round-trip within the RAM cap. - Record manifests via the store's _record_manifest path; run fsck after import as the integrity gate. - git plumbing via the existing _git() helper (hash-object/fast-import optional for the git-native map).

7. Acceptance / verification (each pin has a RED negative control)

1. Round-trip fidelity. Build a store with one of each kind (file, folder, tar.gz, git repo, image); export-git → fresh dir; import-git into a fresh $OMNI_ROOT; assert every artifact address resolves and pull is byte-identical to the original, and store_digest matches. Neg control: an export that omits one referenced blob → import fails loudly (missing-blob error), never a store that lists an address whose pull would refuse. 2. Verify-on-import. Flip one byte of an objects/** file in the git tree → import-git raises CorruptObject, nothing enters the store. Neg control: a naive import that trusts the filename stores the rot (proves the re-hash is load-bearing). 3. Idempotent / incremental. Re-export-git an unchanged store → no new commit and no new git objects; re-import-git → same store, object count unchanged. Neg control: an export that re-mints unchanged blobs grows the pack. 4. Bounded memory. Export+import of a ≥1 GiB blob run with peak RSS ~OMNI_CHUNK_BYTES. Neg control: a whole-file read path OOMs under a tight ulimit/MemoryMax. 5. git-guard interplay. The export dir is a normal git repo (git-guard does NOT flag it); an omni artifact directory still refuses git as before. Neg control: export writing an OMNI-ADDRESS.txt guard marker into the export repo (must not).

8. Open questions

- Q1: default branch name (omni-store) — RFC-0003 naming owns this; confirm. - Q2: --ref-map (git-native refs/omni/<sha256>) — ship in v1 or defer? (Round-trip does not need it.) - Q3: selective export closure — include the artifact's manifest set only, or also parent manifests that reference it? (Default: the artifact + the transitive blob closure it needs to pull.) - Q4: should export-git optionally git gc/pack for distribution, or leave loose (faster re-export)?

9. Non-goals

Not a live sync (it is a snapshot at call time). Not a replacement for omni serve/pull (those stay the live transport). Does not change omni addressing or the store format.