LTO Tapes
I bought a Tandberg LTO-5 external tape drive. Here's some notes.
Hardware
There's a variety of LTO tape drive manufacturers. They aren't all equal. They'll all do the same job, insofar as they all work with *mt*, certainly. But the software used to inspect more interesting metadata stored on the drive seems to be manufacturer specific.
Tape drive
I'm working with an external LTO-5 drive branded Tandberg Data. Though Tandberg appears to have a relationship with HPE. I wouldn't be at all surprised if the internals were identical to a HPE product. It works well, and was around 200GBP used. The following information was pulled from HPE's Library and Tape Tools:
- Product ID : TANDBERG LTO-5 HH - Format : LTO-5 (w/r), LTO-4 (w/r), LTO-3 (r) - Firmware : Z629/OEM (Ver 018.393, Date 2014/02/03-16:36) - SKU number : AQ281W#390_ - Serial number : HUJ4472C93 - Type : SAS - OBDR : Supported - WORM : Supported - Interface - Drive Capability : SAS 6 Gbps - Head life : Great margin, 99% life remaining - Reposition life : Great margin, 99% life remaining - Load/unload life : Great margin, 99% life remaining - Power on time : 668 days, 12 hours, 35 minutes - Tape Pulled : 32.0 full volume equivalents - Tape Metres Pulled : 2068938 metres - Load/unload cycles : 297 (0% of expected life) - Duty cycle : 1% - Power cycles : 38 - Cleans : 2
SAS Card
The drive presents a single Mini-SAS port. My workstation, lacking such a port, required an additional PCI-E card. Thankfully, having dealt with Dell's Powervault products that also present Mini-SAS, I knew I'd be able to get away with any old Dell PERC H card. What I ended up buying was a Dell Perc H200E, part number 12DNW-HP, which is supported by upstream Ubuntu 22.04 with the mpt3sas kernel module (oddly, EL9 seemingly no longer support the card). See below:
#lspci 04:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03) # dmesg | grep LSI mpt2sas_cm0: LSISAS2008: FWVersion(07.15.08.00), ChipRevision(0x03), BiosVersion(07.11.10.00)
Cables
Being a SAS drive, connecting to a SAS controller, it requires a SAS cable. Specifically, SFF 8088 to 8088. Again, because I had a previous reference to work with, I went with Dell part number 038-003-787.Tapes
No tapes, no use for a tape drive, right? I bought a few, from a few different manufacturers. Tapes do wear over time. Both with reads, and writes. But given some companies opt for writing a new tape each session, using the relative low cost of tape as a medium to justify buying more, before shoving them in a cupboard until they go bankrupt, a lot of the time used tapes will be perfectly fine to use. To date, I've had no issues with new HPE tapes, new Sony tapes, and used Fujifilm tapes. There are stories online about how some brands are more likely to cause head-wear. That is, wear to the drive's tapehead. It stands to reason there'd be some sort of difference, although likely negligable. I'm not writing terrabytes a day (so why buy tapes you fool?), so I don't expect that to be an issue. Although it does raise an interesting question of how useful is a medium that lasts thirty years, but built on a standard that seems to only cover reading from the last two standards? LTO-1 came out in 2000, and can be read by (at most) LTO-3 drives that came out in 2005. They can't be read by LTO-4 tapes that came out in 2007. So presumably once you're in, you want to have a stash of drives and appropriate controllers (as the connector itself changed from SCSI to SFF 8088 with LTO-4).Software
I was surprised to find out IBM's xTalk, documented by the fantastic Ben Cox wouldn't work with my drive. Thankfully, unsurprisingly, I had other options:
HP LTT
A rather clunky CUI application that provides drive health checking, tape health checking, and automatic ticket submission to HPE. I'm not an HPE customer, so the latter is rather useless. But the information earlier in this post was gathered by the hpe_ltt binary, which is distributed here. Do note, that HPE do not distribute Debian packages, so you'll need to either extract everything by hand, or convert it with alien. I wont distribute the package I generated here, as I'm not sure of the licensing terms.
tar
tar was made for tapes. I've always known that, since it's the first line of the man page. But actually using tar for its intended purpose (rather than just being a step needed to push numerous files through gzip), is novel to me. There are a few options I found that I'd not have ever considered without messing around.
Firstly, blocking. -b512. By that I mean grouping stuff up into blocks. I understand the general idea behind it - we increase the blocking factor to 512 to increase resulting record length (512*512). Thus tar pushes larger (262144 byte) chunks to the tape drive. Specifying the same blocking factor when extracting from the tarball is essential.
Secondly, labels. --label="flac-2024-01-11". Not strictly necessary, but useful if you aren't going to keep good records of your tapes. The GNU Tar manual documents it well enough -- it's metadata at the front of the archive you can read.
mt
The remote control for the tape deck. It handles rewinding, fast forwarding, ejecting, and printing out current position on the tape. I'll illustrate it alongside dd.
dd
dd does what dd does. It's the hammer of the data world. Naturally, it works for tapes too:
# writing to a tape, with (symmetric) passphrase encryption > dd if=/file1 bs=512k of=/dev/nst0 # demarcating the end of the first file (albeit perhaps unnecessary) > mt -f /dev/nst0 weof 1 # rewinding the tape > mt -f /dev/nst0 rewind # reading the first file from a tape > dd if=/dev/nst0 bs=512k of=file1 # rewinding and ejecting the tape > mt -f /dev/nst0 offline
mbuffer
I don't keep all my files on my desktop, so I need to copy them over the wire. But if my desktop has less space than my NAS, I can't keep them on disk, and certainly not in memory. That isn't a dealbreaker, it just means I'll make the tape drive stopping, starting, and jumping around regularly as it exceeds the speed of my local network and needs to wait for data to come in. This is where mbuffer comes in. very sensible name.
# Archive the contents of an NFS mount # Pipe the resulting tar into mbuffer, which will, in order of arguments: # 1. Buffer up to 64 gigabytes in memory (as I have 128GB to use in my desktop) # 2. Wait until the buffer is 80% full before beginning to write # 3. Write in 512kb blocks. # 4. To /dev/nst0, which is my tape drive. > tar -b512 -cvf - /mnt/H2C/flac 2> | mbuffer -m 64G -P 80 -s 262144 -o /dev/nst0
This results in full speed writes, until the buffer is exhausted (although mbuffer will continue to refill even as it is being drained). Were my network slower, or the LTO standard newer, this would be much more important. As it is, this process simply means I don't need to listen to the whirr of the tape mechanism speed up and slow down and speed up and slow down.
age
Finally, something new to me. age. I'm slightly cautious about it, as everyone should be when a new encryption tool comes along. But it really is quite fantastic. Here's an easy example showing how to add passphrase encryption to a directory written to a tape:
# Writing a large file to a tape > dd if=bigfile.bin bs=512k | age -p > /dev/nst0 # Rewinding > mt -f /dev/nst0 rewind # And extracting again, but decrypted > dd if=/dev/nst0 bs=512k | age -d > bigfile.bin
Frankly, I'm sure this is possible with gnupg. Maybe. But age fits a niche that I like - small tools that work well with pipes. I'd like to see a full audit in the future, but for now, given its history (coming out of the Golang development team), I'm comfortable using it.