Advanced Storage Technology
Datto makes extensive use of ZFS locally and in the cloud for maximum data integrity protection, storage efficiencies, and performance.
The features of ZFS include protection against data corruption, support for high storage capacities, efficient data compression, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, and RAID-Z.
- Copy-on-write snapshots: Use write coalescing to create immutable snapshots that keep each recovery point fully constructed
- Data integrity: Checksums at all levels, even against DMA parity errors SSD acceleration: Enabling 6 second in-place recovery while providing production level storage performance
- Data reduction: Including inline data compression and inline block deduplication
A commitment to ZFS
Datto systems begin with a great hardware and expert build quality, but it requires great software to harness that potential. It starts with the OS and the file system, and Datto selected ZFS (originally stood for Zettabyte File System) as it is the most powerful, flexible, and trusted file system available, and Datto has committed to embedding ZFS into all of its systems. This includes Datto SIRIS, ALTO, and Datto NAS all utilize ZFS. The servers that Datto is manufacturing for its the 180+PB hyperscale Datto Cloud also all use ZFS.
With ZFS, Datto systems are capable of bringing enterprise IT capabilities to the masses but without the enterprise price tag. Datto BDRs compete with backup targets that cost several times more. But since Datto BDRs are a unified and converged solution, the benefits of ZFS are also enjoyed by the BMS, as well as the system which used for virtualizing protected servers in a DR scenario.
Rapid and intelligent directory comparison
The Backup Insights tool is extremely easy to use and can be accessed right from the user interface. In a matter of seconds you can simultaneously mount points and see all files broken down with an easy to read file tree. The reason this is uniquely fast on Datto BDR solutions is because each and every backup is always in a fully constructed state, thanks to Datto’s Inverse Chain and advanced storage technology. Flipping through backup “views” is immediate and fluid, whereas competitive solutions would have to reconstruct different views from chains of full and incremental backup files.
By default, Backup Insights intelligently excludes the Windows directory for any OS volume, as these directories can have a lot of changes and/or be very large. You can choose to have the restore rebuilt with that volume included by simply selecting the Windows folder. You’ll then be presented with a view of the changed files within the Windows directory.
ZFS is a log structured file system (LFS) (see here), where the storage is treated as a circular log, and ZFS and writes sequentially to the head of the log. When writing to disk, ZFS first buffers all updates (including metadata) in an in-memory segment; when the segment is full, it is written to disk in one long, sequential transfer to an unused part of the disk.
ZFS performance benefits can reduce backup windows by more than 2.5X, leading to improved RPO and RTOs. This is due to
- Integrates RAID intelligence with the file system for balanced and optimized use of storage drives.
- Only writes logs that have been coalesced into 128KB blocks to free locations, which both avoids overwrites (and RAID Read-Modify-Write)
- Write coalescing overcomes HDD limitations on operations per second by consolidating scattered writes into a few large sequential operations
- Inline compression and deduplication further reduce write IO transactions
One of the the greatest features of ZFS, and an enabler of Datto’s Inverse Chain Technology, is its built-in Copy-on-write snapshots. Since ZFS redirects all writes, Datto can hundreds of snapshots with effectively zero cost in terms of space, or computational resources to create those snapshots.
With “Copy-on-write” snapshots, each block (or atom) of data in each snapshot (or protected server recovery point) references back to the original atoms. It is also what enables a Datto BDR to instantly virtualize without destructively messing with a backup dependency chain, because with Datto every recovery point is always is a fully constructed state. With the “Copy on Write” capability, we can clone one of those snapshots, using copy-on-write feature to create that clone, and spin up VMs quickly.
NOTE: A Copy-on-write Snapshot has two different meanings. The intended ZFS context has already been explained. The other definition of “Copy-on-write” is from EMC Corporation in late 1990s called where, before an overwrite occurs, the original data is copied into a separate designated storage pool. Hence the name “copy-on-write”. Read requests to the snapshot volume of the unchanged data blocks are redirected to the “copied” blocks in the snapshot, while read requests to active data blocks that have been changed are directed to the original volume.
ZFS is designed with a focus on data integrity, a differentiator from other file systems. ZFS protects against silent data corruption caused by data degradation, current spikes, bugs in disk firmware, phantom writes (the previous write did not make it to disk), misdirected reads/writes (the disk accesses the wrong block), DMA parity errors between the array and server memory or from the driver (since the checksum validates data inside the array), driver errors (data winds up in the wrong buffer inside the kernel), accidental overwrites (such as swapping to a live file system), and so on.
With ZFS, each block of data is checksummed and the checksum value is then saved in the pointer to that block – rather than at the actual block itself. Next, the block pointer is checksummed, with the value being saved at its pointer. This checksumming continues all the way up the file system’s data hierarchy to the root node, which is also checksummed. In-flight data corruption or phantom reads/writes (the data written/read checksums correctly but is actually wrong) are undetectable by most filesystems as they store the checksum with the data. ZFS stores the checksum of each block in its parent block pointer so the entire pool self-validates. ZFS can also self-heal data integrity loss if the data if the storage pool provides data redundancy.
Even if the impossible were to happen, Datto’s Screenshot Verification also ensures there is no loss of data integrity.