09-15-2020, 10:20 AM
Incremental backups with Btrfs snapshots
<div><p>Snapshots are an interesting feature of Btrfs. A snapshot is a copy of a subvolume. Taking a snapshot is immediate. However, taking a snapshot is not like performing a <em>rsync</em> or a <em>cp</em>, and a snapshot doesn’t occupy space as soon as it is created. </p>
<p> <span id="more-31846"></span> </p>
<p>Editors note: From the <a href="https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Snapshots">BTRFS Wiki</a> – A snapshot is simply a subvolume that shares its data (and metadata) with some other subvolume, using Btrfs’s COW capabilities.</p>
<p>Occupied space will increase alongside the data changes in the original subvolume or in the snapshot itself, if it is writeable. Added/modified files, and deleted files in the subvolume still reside in the snapshots. This is a convenient way to perform backups.</p>
<h2>Using snapshots for backups</h2>
<p>A snapshot resides on the same disk where the subvolume is located. You can browse it like a regular directory and recover a copy of a file as it was when the snapshot was performed. By the way, a snapshot on the same disk of the snapshotted subvolume is not an ideal backup strategy: if the hard disk broke, snapshots will be lost as well. An interesting feature of snapshots is the ability to send them to another location. The snapshot can be sent to an external hard drive or to a remote system via SSH (the destination filesystems need to be formatted as Btrfs as well). To do this, the commands <em>btrfs send</em> and <em>btrfs receive</em> are used.</p>
<h2>Taking a snapshot</h2>
<p>In order to use the <em>send</em> and the <em>receive</em> commands, it is important to create the snapshot as read-only, and snapshots are writeable by default.<mark class="annotation-text annotation-text-yoast" id="annotation-text-4d3fc109-ee5a-4154-a85e-c5e6303a0c71"></mark></p>
<p>The following command will take a snapshot of the <em>/home</em> subvolume. Note the <em>-r</em> flag for readonly. </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume snapshot -r /home /.snapshots/home-day1</div>
</div>
<p>Instead of day1, the snapshot name can be the current date, like <em>home-$(date +%Y%m%d)</em>. Snapshots look like regular subdirectories. You can place them wherever you like. The directory <em>/.snapshots</em> could be a good choice to keep them neat and to avoid confusion.</p>
<p>Editors note: Snapshots will not take recursive snapshots of themselves. If you create a snapshot of a subvolume, every subvolume or snapshot that the subvolume contains is mapped to an empty directory of the same name inside the snapshot.</p>
<h2>Backup using btrfs send</h2>
<p>In this example the destination Btrfs volume in the USB drive is mounted as <em>/run/media/user/mydisk/bk</em> . The command to send the snapshot to the destination is: </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs send /.snapshots/home-day1 | sudo btrfs receive /run/media/user/mydisk/bk</div>
</div>
<p> This is called initial bootstrapping, and it corresponds to a full backup. This task will take some time, depending on the size of the <em>/home</em> directory. Obviously, subsequent incremental sends will take a shorter time.</p>
<h2>Incremental backup</h2>
<p>Another useful feature of snapshots is the ability to perform the send task in an incremental way. Let’s take another snapshot. </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume snapshot -r /home /.snapshots/home-day2</div>
</div>
<p>In order to perform the send task incrementally, you need to specify the previous snapshot as a base and this snapshot has to exist in the source and in the destination. Please note the <em>-p</em> option. </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs send -p /.snapshot/home-day1 /.snapshot/home-day2 | sudo btrfs receive /run/media/user/mydisk/bk</div>
</div>
<p> And again (the day after): </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume snapshot -r /home /.snapshots/home-day3</div>
</div>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs send -p /.snapshot/home-day2 /.snapshot/home-day3 | sudo btrfs receive /run/media/user/mydisk/bk</div>
</div>
<h2>Cleanup</h2>
<p>Once the operation is complete, you can keep the snapshot. But if you perform these operations on a daily basis, you could end up with a lot of them. This could lead to confusion and potentially a lot of used space on your disks. So it is a good advice to delete some snapshots if you think you don’t need them anymore.</p>
<p>Keep in mind that in order to perform an incremental send you need at least the last snapshot. This snapshot must be present in the source and in the destination. </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume delete /.snapshot/home-day1</div>
</div>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume delete /.snapshot/home-day2</div>
</div>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume delete /run/media/user/mydisk/bk/home-day1</div>
</div>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume delete /run/media/user/mydisk/bk/home-day2</div>
</div>
<p>Note: the day 3 snapshot was preserved in the source and in the destination. In this way, tomorrow (day 4), you can perform a new incremental <em>btrfs send</em>.</p>
<p>As some final advice, if the USB drive has a bunch of space, you could consider maintaining multiple snapshots in the destination, while in the source disk you would keep only the last one.</p>
</div>
https://www.sickgaming.net/blog/2020/09/...snapshots/
<div><p>Snapshots are an interesting feature of Btrfs. A snapshot is a copy of a subvolume. Taking a snapshot is immediate. However, taking a snapshot is not like performing a <em>rsync</em> or a <em>cp</em>, and a snapshot doesn’t occupy space as soon as it is created. </p>
<p> <span id="more-31846"></span> </p>
<p>Editors note: From the <a href="https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Snapshots">BTRFS Wiki</a> – A snapshot is simply a subvolume that shares its data (and metadata) with some other subvolume, using Btrfs’s COW capabilities.</p>
<p>Occupied space will increase alongside the data changes in the original subvolume or in the snapshot itself, if it is writeable. Added/modified files, and deleted files in the subvolume still reside in the snapshots. This is a convenient way to perform backups.</p>
<h2>Using snapshots for backups</h2>
<p>A snapshot resides on the same disk where the subvolume is located. You can browse it like a regular directory and recover a copy of a file as it was when the snapshot was performed. By the way, a snapshot on the same disk of the snapshotted subvolume is not an ideal backup strategy: if the hard disk broke, snapshots will be lost as well. An interesting feature of snapshots is the ability to send them to another location. The snapshot can be sent to an external hard drive or to a remote system via SSH (the destination filesystems need to be formatted as Btrfs as well). To do this, the commands <em>btrfs send</em> and <em>btrfs receive</em> are used.</p>
<h2>Taking a snapshot</h2>
<p>In order to use the <em>send</em> and the <em>receive</em> commands, it is important to create the snapshot as read-only, and snapshots are writeable by default.<mark class="annotation-text annotation-text-yoast" id="annotation-text-4d3fc109-ee5a-4154-a85e-c5e6303a0c71"></mark></p>
<p>The following command will take a snapshot of the <em>/home</em> subvolume. Note the <em>-r</em> flag for readonly. </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume snapshot -r /home /.snapshots/home-day1</div>
</div>
<p>Instead of day1, the snapshot name can be the current date, like <em>home-$(date +%Y%m%d)</em>. Snapshots look like regular subdirectories. You can place them wherever you like. The directory <em>/.snapshots</em> could be a good choice to keep them neat and to avoid confusion.</p>
<p>Editors note: Snapshots will not take recursive snapshots of themselves. If you create a snapshot of a subvolume, every subvolume or snapshot that the subvolume contains is mapped to an empty directory of the same name inside the snapshot.</p>
<h2>Backup using btrfs send</h2>
<p>In this example the destination Btrfs volume in the USB drive is mounted as <em>/run/media/user/mydisk/bk</em> . The command to send the snapshot to the destination is: </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs send /.snapshots/home-day1 | sudo btrfs receive /run/media/user/mydisk/bk</div>
</div>
<p> This is called initial bootstrapping, and it corresponds to a full backup. This task will take some time, depending on the size of the <em>/home</em> directory. Obviously, subsequent incremental sends will take a shorter time.</p>
<h2>Incremental backup</h2>
<p>Another useful feature of snapshots is the ability to perform the send task in an incremental way. Let’s take another snapshot. </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume snapshot -r /home /.snapshots/home-day2</div>
</div>
<p>In order to perform the send task incrementally, you need to specify the previous snapshot as a base and this snapshot has to exist in the source and in the destination. Please note the <em>-p</em> option. </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs send -p /.snapshot/home-day1 /.snapshot/home-day2 | sudo btrfs receive /run/media/user/mydisk/bk</div>
</div>
<p> And again (the day after): </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume snapshot -r /home /.snapshots/home-day3</div>
</div>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs send -p /.snapshot/home-day2 /.snapshot/home-day3 | sudo btrfs receive /run/media/user/mydisk/bk</div>
</div>
<h2>Cleanup</h2>
<p>Once the operation is complete, you can keep the snapshot. But if you perform these operations on a daily basis, you could end up with a lot of them. This could lead to confusion and potentially a lot of used space on your disks. So it is a good advice to delete some snapshots if you think you don’t need them anymore.</p>
<p>Keep in mind that in order to perform an incremental send you need at least the last snapshot. This snapshot must be present in the source and in the destination. </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume delete /.snapshot/home-day1</div>
</div>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume delete /.snapshot/home-day2</div>
</div>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume delete /run/media/user/mydisk/bk/home-day1</div>
</div>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">sudo btrfs subvolume delete /run/media/user/mydisk/bk/home-day2</div>
</div>
<p>Note: the day 3 snapshot was preserved in the source and in the destination. In this way, tomorrow (day 4), you can perform a new incremental <em>btrfs send</em>.</p>
<p>As some final advice, if the USB drive has a bunch of space, you could consider maintaining multiple snapshots in the destination, while in the source disk you would keep only the last one.</p>
</div>
https://www.sickgaming.net/blog/2020/09/...snapshots/