Annvix
Personal tools
/Documentation/iSCSI

From Annvix

iSCSI

Contents

What is it?

iSCSI is a protocol designed to allow block devices to be treated as local devices through a TCP/IP connection. This means that systems, such as databases or mail servers, that need local storage can take advantage of pooled storage.

Essentially there will be a server that makes storage available and clients that use the storage. The server side provides iSCSI targets, and the client connects to those targets using an iSCSI initiator, but this is all usually software, so you don't have to worry about buying anything.

iSCSI is known to work well with Linux and Windows. It can work with BSD and AIX and others, but Your Mileage May Vary.

Why

Who hasn't wanted to add a drive to a machine but either didn't have the hardware to expand or just didn't want to open up the box? iSCSI is a way to add "drives" over a TCP/IP network.

NAS vs iSCSI

NAS, or Network Attached Storage typically is available as either NFS or CIFS (or SMB for older systems.) iSCSI differs from traditional network storage in the way the information is shared. iSCSI is block level storage, NAS is file level storage. iSCSI works well for things like databases and pst files. With iSCSI the operating system can just work with the portion of a file that it is changing rather than copying the entire file into temporary storage as is typical of NAS. For this reason it is possible to have entire operating systems installed in iSCSI.

Potential Drawbacks

There is one very serious consideration that cannot be ignored when using iSCSI. Just as two machines cannot easily share a single hard drive, neither can two systems share a single iSCSI share. It is technically possible but extremely dangerous since any changes made by one might apply to both, without the second being aware of the changes. Using a single iSCSI target for two systems is likely to result in data corruption. If using read only access, it may be possible, but the documentation on this is sparse. Where possible it is recommended to provide a separate iSCSI target for each Operating System. (One iSCSI system can provide many iSCSI targets so this is rarely a significant concern.)

Virtual Machines with iSCSI

iSCSI is ideal with Virtual Machines such as those with Xen or VMWare. With a virtual machine stored in an iSCSI target the hosting operating system can make the target available to the virtual machine which allows for more efficient use of storage. An especially attractive option is then to run the virtual machine from a different host when the original is unavailable.

iSCSI with LVM

When using LVM to provide the partitions that become iSCSI targets, new options become available. It is possible to create multiple snapshots of a base system and then the common portions of the systems are shared, but every difference created by each individual system is still all that individual system sees. As a backup, snapshots also make it possible to completely undo recent changes. This means that you could potentially create a good server configuration and then take a snapshot and not have to worry about doing something risky (like running a global distribution upgrade) and be able to restore the original system exactly as it was. LVM itself provides this functionality but when combined with iSCSI it adds the convenience of centralized storage. Snapshots also make it possible to clone an active system.

Multipath

One additional potential benefit is the ability to multipath iSCSI targets. If one network becomes unavailable, a second can be used to access the same iSCSI target without interruption of service. This is not supported by all implementations, but is supported by current Windows initators.

iSCSI is cheap

The standard high performance SANs have used Fiber Channel for drive access. Fiber Channel access is indeed very fast, about the equivilent of 2Gb ethernet, but it is also very expensive.

iSCSI uses usually free software and existing ethernet networking to provide the same type of storage. It is often completely free of cost to implement. Even implementing iSCSI with specifically purchased network hardware is usually less expensive than implementing fiber channel.

iSCSI can be fast

It is possible to access iSCSI shares with minimal access speeds and preferable over NAS because the OS is able to copy only the changed and read blocks instead of files over the network. It is also possible to use very high speed networking to achieve even higher performance. With the proliferation of Gigabit ethernet and multi-pathing and bonding it is possible to approach and possibly exceed speeds offered by Fiber Channel. Multiple targets can be combined with different networks to then be combined in software RAID, making it possible to utilize two or more Gigabit ethernet connections at the same time. With 10 Gigabit ethernet now available, it is possible to exceed Fiber Channel data transmission speeds, although keeping the drive capability ahead of the capacity of the network to transfer data becomes a challenge at those rates.

Creating an iSCSI Enterprise Target

Author's note: this is a one time install and configuration for me. Please let me know how your tests go and any changes that should be made to this document in the talk page.


There are two main projects that bring the ability to provide an iSCSI target to Linux. The first is the Open iSCSI project which is integrated and works well with Xen, CentOS and RedHat. The second is the Enterprise iSCSI project which I had no trouble getting to work on Annvix.

Preparation

My testing was done an installation of 2.0 while it was still in beta. Your installation should always be current so start off by making sure it is:

  1. apt-get update
  2. apt-get upgrade
  3. apt-get install kernel kernel-source
  4. Update grub if necessary
  5. reboot

Pre-Configuration

  1. Download the source that matches your kernel installation (2.6.14 kernel or higher probably required) from the sourceforge site.
  2. Create a link so that modules that are installed by the iSCSI system will end up where they belong:
    ln -s /lib/modules/2.6.16.37-6656avx /lib/modules/2.6.16.37-6656avxcustom

Installation

  1. Decompress as necessary
  2. Move to the correct directory
  3. Compile and install according to the directions included
    Something like
    export KSRC=/usr/src/linux
    make
    make install

Post-Installation Configuration

  1. Edit /etc/modprobe.preload and add:
    iscsi_tcp
    scsi_transport_iscsi
  2. Edit /etc/ietd.conf
    • This file is heavily commented and should not present too much of an obstacle to the experienced Linux user. It is recommended that for testing purposes CHAP not be used.
  3. Start the daemon as a supervised service or with the included daemon at /etc/init.d/iscsi-target start
    Note that you may need to reboot before this will work

Use

Connecting

  • Linux: Many modern Linux distributions come with iSCSI initiators built in. It may be necessary to configure them. For RHEL use the package "iscsi-tools"
    • Once installed, you will need to create a configuration file. With iscsi-tools there is a tool called iscsiadm which will do most of the work for you.
  2. Windows: search for and install the Microsoft iSCSI initator. Point the discovery to the machine providing the targets then configure the individual target as displayed.
  3. AIX (not thoroughly tested): iscsi is already available from AIX 5.2 newer. Just configure the file at /etc/iscsi.conf then add the drive with smit and/or configmgr

Preparing the drive for use

  • Linux: Simply run fdisk against the new drive, usually /dev/sd[abcdef], partition and format
  • Windows:
    1. Right click the My Computer icon and then choose Manage.
    2. From the left menu open the storage sub menu and drive management tools
    3. Right-click the new drive and initalize it, then format it. (This may be done in the Wizard if it is available)
    4. Format the new partition(s) and then assign drive letters or mount points as desired
  • AIX: testing incomplete
Retrieved from "http://annvix.org/Documentation/iSCSI"



Sponsors: Flights - Car Insurance - United Specialties - Mortgages