zigford.org

About | Links | Scripts
Sharing linux/windows scripts and tips

Transfer BTFS snapshots with netcat

July 19, 2020 — Jesse Harris

Netcat, the swiss army knife of TCP/IP can be used for many tasks. Today, I'll breifly demonstrate sending btrfs snapshots between computers with it's assistance.


The setup

I use a raspberry pi computer as a small NAS. It doesn't have a high performance requirment and just has an 8Tb spinning rust drive connected over usb 3. Ocasionally, I want to back it up to another set of drives across my local LAN.

Both the raspberry pi, and my receiving desktop computer run Gentoo and thus installing netcat can be achieved by the following command:

    emerge net-analyzer/netcat

Gentoo also offers the OpenBSD variant:

    emerge net-analyzer/openbsd-netcat

These two versions will happily work together, but have slightly different syntax and behaviors

Version differences

The main difference I have noticed between the two versions is that the openbsd variant will automatically quit when it receives and EOF signal, while the original version requires the -q parameter.

On the receiving end

I tend to setup my recieving end of the netcat tunnel first. I beleive the sending end could be setup first but this seems more natural to my mind. In this case I want my btrfs snapshot stored in another btrfs filesystem. This allows easy restoration of single file.

    cd /mnt/btrfs/backups
    nc -l 10004 | btrfs receive .

The -l 10004 parameter specifies that netcat should listen on port 10004. You could as easily pick any other port not in use.

Next, a pipe to btrfs receive and . says to store the snapshot here.

After you press enter on this command, netcat will be listening, but as no data has passed through the pipe yet, the btrfs command will not have recieved anything and will just be waiting indefinitly for data.

On the sending end

From the source machine, if this is the first snapshot ever send:

    cd /mnt/btrfs/snapshots/
    btrfs send root-2020-07-19-01:00:01 | nc -q 1 192.168.11.203 10004

If it is not the first:

    cd /mnt/btrfs/snapshots/
    btrfs send -p root-2020-07-19-01:00:01 \
        root-2020-07-20-01:00:01 | nc -q 1 192.168.11.203 10004

The -p can tell btrfs that the following subvolume is a parent of the snapshot we are sending and therefore only send the difference. The -q 1 tells that non-openbsd varient of netcat to quit 1 second after receiving and EOF message from the upstream program.

After pressing enter on this command, btrfs send will begin passing data through the pipe to netcat which will connect to the destination computer on the port specified. You should begin to see informational output from btrfs at this stage.

If you hadn't thought to specify -q parameter, not to worry. You can monitor that data is coming through the pipe using a tool like nettop or iotop. After data transfer has ceased, simply hit Ctrl+C to close the pipe on both ends at once.

Summing up

And that is about all there is to sending btrfs snapshots with netcat. You can transfer your snapshots as fast as your disks and LAN will allow. Depending on your CPU you may also get better performance if you pipe through gzip and gunzip.

Tags: btrfs, netcat, linux