Installing and using iSCSI-SCST with scstadmin ============================================== 1. Download, build and install iSCSI-SCST. Start with downloading the iSCSI-SCST source code. You can either download the released version from the following URL: http://scst.sourceforge.net/downloads.html or you can download the latest development version by running the following command in a shell: svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk scst While iSCSI-SCST works fine with an unpatched Linux kernel with the same or better speed as other open source iSCSI targets, including IET, if you want even better performance you have to patch and rebuild the kernel. Select the patch according to your kernel version (2.6.38.x in the example below): cd /usr/src/kernels/linux-2.6.38.8 patch -p1 < $HOME/scst/iscsi-scst/kernel/patches/put_page_callback-2.6.38.patch patch -p1 < $HOME/scst/scst/kernel/scst_exec_req_fifo-2.6.38.patch make clean Next, build and install the kernel: make && make modules make modules_install && make install For some Linux distributions (e.g. Ubuntu) the above steps do not rebuild the initial RAM disk. Make sure you regenerate the RAM disk before rebooting. Please look up the documentation of the update-initrd, update-initramfs or the mkinitrd command supplied with your distribution. Now reboot your system such that the newly built kernel is loaded. Once the system is again up and running the next step is to build and install SCST, iSCSI-SCST and scstadmin. With most distros (including RHEL 6) you can do that by running the following command from the SCST directory: make scst scst_install iscsi iscsi_install scstadm scstadm_install When using RHEL 5 or CentOS 5, use these commands instead: make enable_proc make scst scst_install iscsi iscsi_install scstadm scstadm_install iSCSI-SCST includes the following components: * iscsi-scstd - daemon * man pages * Obsolete configuration files located under iscsi-scst/etc/obsolete, need to be copied to /etc if you want to use them. In iSCSI-SCST 2.0 it is recommended to use scstadmin and SCST sysfs interface for that: * initiators.allow - used for assigning specific initiators to targets * initiators.deny - used for assigning specific initiators to targets * iscsi-scstd.conf - list of targets and their properties 2. Set up /etc/iscsi/initiatorname.iscsi The most convenient way to set up this file is to install the open-iscsi package provided by your Linux distro first and then to run the shell commands shown below. Verify the contents of the generated file. /etc/init.d/open-iscsi stop { echo "InitiatorName=$(if [ -e /usr/sbin/iscsi-iname ]; then /usr/sbin/iscsi-iname; else /sbin/iscsi-iname; fi)"; echo "InitiatorAlias=$(hostname)"; } >/etc/iscsi/initiatorname.iscsi /etc/init.d/open-iscsi start cat /etc/iscsi/initiatorname.iscsi 3. Set up /etc/scst.conf The scst.conf configuration file is loaded at system boot time by the shell script /etc/init.d/scst. The target, device and LUN configuration is defined in this file. Here is the sample config file. Copy it in /etc/scst.conf HANDLER vdisk_fileio { DEVICE disk01 { filename /dev/ram0 nv_cache 1 } DEVICE disk02 { filename /dev/ram1 nv_cache 1 } } TARGET_DRIVER iscsi { enabled 1 TARGET iqn.2006-10.net.vlnb:tgt { LUN 0 disk01 LUN 1 disk02 enabled 1 } } Note that for each SCSI device LUN 0 must exist. This is required by the SCSI protocol and is also documented in SCST's README file. Then run the following command: /etc/init.d/scst start You should have the working iSCSI-SCST now. Do the following steps if you need to automatically configure it upon reboot. 4. Create soft links in /etc/init.d This will make SCST to start automatically upon system boot. Use either chkconfig or update-rc.d, depending on the Linux distribution you are using. An example for Debian systems such as Ubuntu: update-rc.d scst defaults On openSUSE you can let SCST start automatically through the following commands: insserv scst 5. Restart SCST and iSCSI-SCST If you don't want to reboot now, you can make the above settings effective via the following commands: /etc/init.d/scst restart Installing and using iSCSI-SCST without using scstadmin via /sys interface ========================================================================== First repeat steps 1 and 2 from the first section but leave out "scstadm scstadm_install" from the make command in step 1. Next load the scst_disk and scst_vdisk kernel modules as follows: modprobe scst_disk modprobe scst_vdisk modprobe iscsi-scst iscsi-scstd Creating targets ---------------- echo "add_target iqn.2007-05.com.example:storage.iscsi-scst-1" > /sys/kernel/scst_tgt/targets/iscsi/mgmt This will add target iqn.2007-05.com.example:storage.iscsi-scst-1. Changing targets' parameters ---------------------------- echo "8192" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/MaxRecvDataSegmentLength This will change MaxRecvDataSegmentLength parameter of target iqn.2007-05.com.example:storage.iscsi-scst-1 to 8192. You can read it then by: # cat /sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/MaxRecvDataSegmentLength 8192 [key] The mark "[key]" shows high level management tools, like scstadmin, that this attribute has not default value. You can ignore it. Add vdisk devices ----------------- dd if=/dev/zero of=/disk1 bs=1M count=1024 This will create a backend file /disk1 for our future virtual device disk1. echo "add_device disk1 filename=/disk1; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt This will add new FILEIO device disk1 with backend file /disk1 and NV_CACHE option enabled. Selecting devices to be used by SCST ------------------------------------ You can see the list of available devices: # ls -1 /sys/kernel/scst_tgt/devices 0:0:0:0 1:0:0:0 3:0:0:0 3:0:0:1 3:0:0:2 3:0:0:3 3:0:0:4 3:0:0:5 3:0:0:6 3:0:0:7 2:0:0:0 4:0:0:0 4:0:0:5 4:0:0:6 4:0:0:7 4:0:0:8 4:0:0:9 4:0:1:0 4:0:1:1 4:0:1:2 4:0:1:3 4:0:1:4 5:0:0:0 5:0:0:5 5:0:0:6 5:0:0:7 5:0:0:8 5:0:0:9 5:0:1:0 5:0:1:1 5:0:1:2 5:0:1:3 5:0:1:4 disk1 Defining LUN masking -------------------- In order to associate specific LUNs with target iqn.2007-05.com.example:storage.iscsi-scst-1, do the following: echo "add 4:0:0:8 0" > /sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/luns/mgmt This will assign the LUN 0 for pass-through device 4:0:0:8 to the target iqn.2007-05.com.example:storage.iscsi-scst-1. Note, you must have LUN 0 for each group. That's a SCSI requirement (documented in SCST's README). echo "add disk1 1 read_only=1" > /sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/luns/mgmt This will assign the read only LUN 1 for virtual device disk1 to the target iqn.2007-05.com.example:storage.iscsi-scst-1. Deleting a LUN from a group --------------------------- Run the following command: echo "del X" > /sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/luns/mgmt This will remove the LUN X from the target iqn.2007-05.com.example:storage.iscsi-scst-1. For more information about LUN masking, refer to SCST README, section "Access and devices visibility management (LUN masking)". Enabling targets ---------------- After you created and configured target, you should enable it: echo 1 >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/enabled Then enable the iSCSI-SCST driver: echo 1 >/sys/kernel/scst_tgt/targets/iscsi/enabled Assigning targets to specific initiators ---------------------------------------- In order to assign targets to specific initiators, you should use LUN masking defined above. If an initiator assigned to an empty security group of a target, the initiator will not see this target in the SendTargets discovery and will not be able to connect to it. Additionally, you can make the target be visible only via selected portals using allowed_portal sysfs attribute. Read iSCSI-SCST's README file for more details about this facility. Installing and using iSCSI-SCST without using scstadmin via /proc interface =========================================================================== First repeat steps 1 and 2 from the previous section but leave out "scstadm scstadm_install" from the make command in step 1. Next set up /etc/iscsi-scstd.conf You can do this by e.g. running the following shell commands: echo "Target $(sed -n 's/InitiatorName=//p' /etc/iscsi/initiatorname.iscsi):storage" >/etc/iscsi-scstd.conf cat /etc/iscsi-scstd.conf Next load the scst_disk kernel module as follows: modprobe scst_disk The shell script /etc/init.d/iscsi-scst can now be used to start/stop/restart or check the status of iSCSI-SCST. Selecting devices to be used by SCST ------------------------------------ You can see the list of available devices: # cat /proc/scsi_tgt/scsi_tgt Device (host:ch:id:lun or name) Device handler 0:0:0:0 dev_disk 1:0:0:0 dev_disk 3:0:0:0 dev_disk 3:0:0:1 dev_disk 3:0:0:2 dev_disk 3:0:0:3 dev_disk 3:0:0:4 dev_disk 3:0:0:5 dev_disk 3:0:0:6 dev_disk 3:0:0:7 dev_disk 2:0:0:0 none 4:0:0:0 none 4:0:0:5 dev_disk 4:0:0:6 dev_disk 4:0:0:7 dev_disk 4:0:0:8 dev_disk 4:0:0:9 dev_disk 4:0:1:0 dev_disk 4:0:1:1 dev_disk 4:0:1:2 dev_disk 4:0:1:3 dev_disk 4:0:1:4 dev_disk 5:0:0:0 none 5:0:0:5 dev_disk 5:0:0:6 dev_disk 5:0:0:7 dev_disk 5:0:0:8 dev_disk 5:0:0:9 dev_disk 5:0:1:0 dev_disk 5:0:1:1 dev_disk 5:0:1:2 dev_disk 5:0:1:3 dev_disk 5:0:1:4 dev_disk LUN masking ----------- SCST defines security groups. For each group, you can add LUNs. If you want to have all LUNs available for all targets, just add them to the "Default" group: # echo "add 5:0:0:8 0" >/proc/scsi_tgt/groups/Default/devices # cat /proc/scsi_tgt/groups/Default/devices Device (host:ch:id:lun or name) Virtual lun Options 5:0:0:8 0 Now, the LUN "5:0:0:8" was added to the "Default" group as LUN #0. Note, that you must have LUN 0. That's a SCSI requirement (documented in SCST's README). Defining LUN masking -------------------- In order to associate specific LUNs with specific targets, do the following: * Create a group for the target: echo "add_group Default_iqn.2007-05.com.example:storage.iscsi-scst-1" >/proc/scsi_tgt/scsi_tgt * Add LUNs to the group: echo "add 4:0:0:8 0" > /proc/scsi_tgt/groups/Default_iqn.2007-05.com.example\:storage.iscsi-scst-1/devices This will assign the LUN "4:0:0:8" to the target iqn.2007-05.com.example:storage.iscsi-scst-1. Again, you must have LUN 0 for each group. Deleting a LUN from a group --------------------------- Run the following command: echo "del 4:0:0:8 0" > /proc/scsi_tgt/groups/Default_iqn.2007-05.com.example\:storage.iscsi-scst-1/devices This will remove the LUN "4:0:0:8" from the target iqn.2007-05.com.example:storage.iscsi-scst-1. Deleting a group ---------------- Run the following command: echo "del_group Default_iqn.2007-05.com.example:storage.iscsi-scst-1" >/proc/scsi_tgt/scsi_tgt For more information about LUN masking, refer to SCST README, section "Access and devices visibility management (LUN masking)". Creating targets using iscsi-scstd.conf --------------------------------------- The easiest way to create targets is to define them in /etc/iscsi-scstd.conf. An example can be found in etc/iscsi-scstd.conf. You need to have this file under /etc/ before starting iSCSI-SCST. Assigning targets to specific initiators ---------------------------------------- In order to assign targets to specific initiators, you need to have /etc/initiators.allow and /etc/initiators.deny. You can find example files in etc/initiators.allow and etc/initiators.deny. Note that all targets are allowed to all initiators by default, so if you want to use /etc/initiators.allow, you will need to have /etc/initiators.deny that looks like this: ALL ALL This will deny all initiators expect for those defined in /etc/initiators.allow. Useful examples of iscsi-scst-adm: Show a specific target: # iscsi-scst-adm --op show --tid=1 QueuedCommands=0 Show the configured parameters for a specific target: # iscsi-scst-adm --op show --tid=1 --sid=0 InitialR2T=No ImmediateData=Yes MaxConnections=1 MaxRecvDataSegmentLength=2097152 MaxXmitDataSegmentLength=131072 MaxBurstLength=2097152 FirstBurstLength=262144 DefaultTime2Wait=2 DefaultTime2Retain=0 MaxOutstandingR2T=1 DataPDUInOrder=Yes DataSequenceInOrder=Yes ErrorRecoveryLevel=0 HeaderDigest=None DataDigest=None OFMarker=No IFMarker=No OFMarkInt=Reject IFMarkInt=Reject Bart Van Assche Erez Zilber Vladislav Bolkhovitin