Repository-Backend Architecture
1. Backend
Backends implement an abstract view over a, conceptually, WORM 1 storage backend.
The backend divides storage into one or more segments, and each segment is further divided into one or more chunks.
Chunks are the fundamental unit of storage in a repository, and a referenced by a 32 byte key which, with very little exception, is an HMAC of the plain text of the contents of the chunk.
1.1. Chunks
The backend itself does’t have any knowledge of the contents or structure of chunks, it simply views them as list of bytes. The only knowledge a backend has of the chunks is their length, which segment they are in, and where they are in that segment.
1.1.1. Segments
Segments are groups of chunks stored on disk. Most often they will just be simple concatenations of msgpacked chunk structs. Implementations are allowed to have other data interspersed between chunks, so long as the chunks can be extracted from the file by simply taking a range of bytes from it.
Implementations will usually have multiple segments, but are not required to.
1.2. Manifest
The manifest is a special object in the backend containing the keys of archive objects. It is accessed through the Backend::Manifest trait.
1.3. Index
The index is another special object in the backend containing the locations and sizes of all the chunks in a repository. It has a notion of transactionality. Transactions are opened when a change is written without a transaction open, and closed with the close method.
Footnotes:
Write Once, Read Many