Data loss happens to everyone

Most people will experience catastrophic data loss at some point. A family's photos were lost when a TV set-top box reformatted their external drive without proper warnings.

“There are two types of people: those who have suffered a catastrophic loss of data, and those who will.”

Single copies are a disaster waiting

Storing all important files in one location guarantees failure. Hard drives fail, bits rot in cold storage, SSDs leak electricity over time, and drives get stolen or destroyed.

Backups need to be copies, not mirrors

RAID 1 mirroring won't protect you from ransomware, accidental deletion, or scripts that overwrite everything. You need snapshots taken at intervals so you can restore to a previous point in time.

Recovery Point Objective sets snapshot frequency

RPO is how much data loss you can tolerate. Financial institutions need RPO under 30 seconds. Small businesses might accept 24 hours. Weekly snapshots mean losing up to 6 days and 23 hours of work.

Snapshots create a storage burden

Daily snapshots at 24-hour intervals create 365 snapshots per year. You can't keep them all, but you also can't discard recent ones. You need a rotation strategy.

Granularity changes over time

Keep daily snapshots for 14 days, weekly snapshots for 7 weeks, and monthly snapshots for 12 months. Recent data needs frequent snapshots. Old data needs less frequent ones.

GFS rotation adds complexity

Grandfather-Father-Son rotation combines daily, weekly, and monthly snapshots with different retention periods. It's efficient but requires careful management.

Deduplication saves space and bandwidth

Most files don't change between snapshots. Use hard links to reference existing files instead of storing duplicates. This cuts storage needs dramatically and reduces bandwidth costs.

Incremental backups store only changes

Instead of copying everything since the last full backup, store only the differences between adjacent snapshots using hard links. Tools like rsnapshot use this approach.

Docker containers complicate backups

Docker containers create root-owned files and run databases that flush to disk in batches. Backups can fail or restore corrupted data if you don't handle permissions and database dumps carefully.

Hardware failures demand redundancy

Certain hard drive models have notoriously high failure rates. Store backups on two different types of media to protect against hardware-specific failures.

The 3-2-1 backup rule

Keep 3 copies of your data, on 2 different types of media, with 1 copy offsite. This protects against power surges, floods, fires, and hardware failures.

Cloud storage requires tarballs

Object storage like Amazon S3 strips file metadata and charges heavily for many small files. Bundle files into 50MB chunks in tarballs to preserve metadata and control costs.

Use battle-tested tools instead

Building your own backup system is too complex. Use Borg or Restic, which handle encryption, chunk-level deduplication, checksums, and all the edge cases you'll encounter.

“It simply isn't worth the mental load to do all this.”

Test your restores regularly

A backup is worthless if you've never restored from it. Run restore tests every 6 months to verify your backups actually work when you need them.

Avoid the 2AM to 3AM cron trap

Don't schedule backups during daylight saving time transitions. When clocks skip an hour, cron jobs can fail or run twice, breaking your backup schedule.

Consider ZFS for snapshots

ZFS is a filesystem built around data integrity and snapshots. It handles many backup complexities automatically, making it worth considering for serious backup needs.

Consistency matters for databases

When snapshotting systems with databases, you risk inconsistent snapshots if the database state diverges from the application state. Freeze all I/O during snapshots or use the product's native snapshot tools.