I’ve recently been playing with Debian’s iSCSI support, and it’s pretty neat.
It was a little esoteric to set things up, so I figured I’d write up a quick blog post of exactly what I did both for my own future-self’s sake and for the sake of anyone else trying to do something similar.
The most “followable” guide I found was https://www.certdepot.net/rhel7-configure-iscsi-target-initiator-persistently/ (which the below is probably really similar to).
The exact details of what I was trying to accomplish are as follows:
- 100GB “sparse” file on
my-desktop - presented as an iSCSI target
- mounted on
my-rpi3as/var/lib/docker(preferably withdiscardenabled so the file onmy-desktopstays sparse)
On my-desktop, I used the targetcli-fb package to configure my iSCSI target:
$ sudo apt install targetcli-fb
$ # create the sparse file
$ mkdir -p /home/tianon/iscsi
$ truncate --size=100G /home/tianon/iscsi/my-rpi3-docker.img
$ # launch "targetcli" to configure the iSCSI bits
$ sudo targetcli
# create a "fileio" object connected to the new sparse file
/> /backstores/fileio create name=my-rpi3-docker file_or_dev=/home/tianon/iscsi/my-rpi3-docker.img
# enable "emulated TPU" (enable TRIM / UNMAP / DISCARD)
/> /backstores/fileio/my-rpi3-docker set attribute emulate_tpu=1
# create iSCSI storage object
/> /iscsi create iqn.1992-01.com.example.my-desktop:storage:my-rpi3-docker
# create "LUN" assigned to the "fileio" object
/> /iscsi/iqn.1992-01.com.example.my-desktop:storage:my-rpi3-docker/tpg1/luns create /backstores/fileio/my-rpi3-docker
# create an ACL for my-rpi3 to connect
/> /iscsi/iqn.1992-01.com.example.my-desktop:storage:my-rpi3-docker/tpg1/acls create iqn.1992-01.com.example:node:my-rpi3
# and set a CHAP username and password, for security
/> /iscsi/iqn.1992-01.com.example.my-desktop:storage:my-rpi3-docker/tpg1/acls/iqn.1992-01.com.example:node:my-rpi3 set auth userid=rpi3 password=holy-cow-this-iscsi-password-is-so-secret-nobody-will-evvvvvvvvver-guess-it
Additionally, I’ve been experimenting with firewalld on my-desktop, so I had to add the iscsi-target service to my internal zone to allow the traffic from my-rpi3.
On my-rpi3, I used the open-iscsi package to configure my iSCSI initiator:
$ sudo apt install open-iscsi
$ # update "InitiatorName" to match the value from our ACL above
$ sudo vim /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1992-01.com.example:node:my-rpi3
$ # update "node.startup" and "node.session.auth.*" for our CHAP credentials from above
$ sudo vim /etc/iscsi/iscsid.conf
...
node.startup = automatic
...
node.session.auth.authmethod = CHAP
node.session.auth.username = rpi3
node.session.auth.password = holy-cow-this-iscsi-password-is-so-secret-nobody-will-evvvvvvvvver-guess-it
...
# restart iscsid so all that takes effect (especially the InitiatorName change)
$ sudo systemctl restart iscsid
$ sudo iscsiadm --mode discovery --type sendtargets --portal my-desktop-ip-address
$ sudo iscsiadm --mode node --targetname iqn.1992-01.com.example.my-desktop:storage:my-rpi3-docker --portal my-desktop-ip-address --login
$ lsblk --scsi
NAME HCTL TYPE VENDOR MODEL REV TRAN
sda 0:0:0:0 disk LIO-ORG my-rpi3-docker 4.0 iscsi
$ sudo fdisk /dev/sda
...
$ sudo mkfs.ext4 -T news -L my-rpi3-docker /dev/sda1
...
$ lsblk | grep my-rpi3-docker
... UUID="xxx" ...
$ sudo vim /etc/fstab
...
UUID="xxx" /var/lib/docker ext4 noatime,discard,_netdev 0 0
...
$ sudo systemctl stop docker
$ sudo mount /var/lib/docker
$ sudo systemctl start docker
$ # yay, profit (and should auto-remount properly on boot and everything, too)
(Obviously, replace iqn.1992-01.com.example with an appropriate IQN for your own domain as described on Wikipedia, and other values as appropriate like the username/password, hostnames, IPs, etc.)
As for speed, I was able to get the following result from a very simplified dd-based speed test – YMMV:
$ dd if=/dev/zero of=/var/lib/docker/testfile bs=100M count=10 oflag=direct
10+0 records in
10+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 97.9608 s, 10.7 MB/s