Asuran

An Archiver for the 2020s

Gitlab Codecov Gitlab pipeline status Crates.io Crates.io Matrix Blog

Notice: The author has very recently resumed work on this project (November 2021) after a hiatus, plans have changed and the information on this site may be out of date.

Asuran is a new security and performance focused deduplicating archiver, built to be able to stay just behind the cutting edge of deduplication technology.

Asuran has many features that users of borg, restic, or many similar tools will find familiar, and pairs them with blazing fast performance and cutting edge features that improve deduplication rates and flexibility.

Asuran is still in early development, and while it has a prototype that technically works, it has yet to be thoroughly tested, and the on-disk format has yet to be stabilized. Please feel free to test it out as an additional backup, but do not rely on it, in this early stage of development it WILL eat your laundry.

1. Mission Statement

The asuran archival format is designed to be, in order of importance

1.1. Suitable for long term archival

Asuran should be a format you should be able to keep your data in forever. Breaking changes to the format (once the release hits 0.1.0) should never lose data in the forward direction, always come with a statically linked binary utility that can convert archives back and forth between the two formats, and always come with through documentation about any structures/features that can not be preserved moving in the backwards direction.

Format versions should be well documented, with easily accessible plaintext documentation, such that a plaintext copy stored alongside an important repository should be sufficient to allow a future engineer to restore the repository without access to an existing asuran implementation.

Long term archival features like optional parity data to guard against bitrot and a built in for in place refreshing by rewriting every segment should be provided.

Operating system and hardware independence

Asuran should not make any design decisions that limit the ability to run on alternative or non-conventional hardware or operating systems. CI testing should extend to the big three (Windows, Linux, MacOS) on x86_64, as well as linux ARM64. At the moment we do not currently have a MacOS machine to run CI builds on. Other architectures and operating systems (so long as they are available as rust targets) should be extended support when practical.

1.2. Secure

Asuran should make good use of encryption and other cryptographic technologies to provide assurance of privacy to the user. Being hostable on untrusted storage, asuran can not hope to completely prevent data tampering, but it should, to the greatest extent possible, be immune to nondestructive tampering (i.e. addition of new files into an archive by an attacker), and be able to detect and reject archives that have been destructively tampered with (i.e. an attacker deleting or modifying files in a repository)

1.3. Flexible

Asuran should not place any arbitrary restrictions on the content or structure of data stored in the repository, and should not be limited to the traditional filesystem abstraction. Alternative data layouts, such as photo libraries, email inboxes, and SQL database dumps should enjoy first class citizen status in the Asuran ecosystem.

1.4. Fast

libasuran should be able to easily saturate a 1Gig ethernet port on a normal consumer grade desktop, or a 10Gig ethernet port on a mid to high tier server, with encryption and a reasonable level of compression turned on. This is assuming that libasuran does not outrun storage of course.

1.5. Easily Embeddable

The conical Asuran implementation (simply called Asuran) should eat its own dog food by directing all non-trivial repository operations through libasuran. libasuran should expose a well documented and consistent API for interacting with repositories, and should have a well maintained and thoroughly documented C FFI with bindings to, at very least, Python.

2. Documentation

3. Links