Configuring Dell EqualLogic PS6500 Array to work with RedHat Linux 6 EL

I decided to write the following post after having some problems getting iSCSI connectivity to work correctly with a Dell EqualLogic PS6500 Array with RedHat 6.

It seems that Dell have done a great job in getting the Host Integration Tools to work with RH5.5 and ESXi and also the MEM for MPIO (Multipath) and I love the product, but I encountered some problems when trying to install a RH6 server and connect it all up following the docs on the Dell site. The main issue I had straight way that I was predominantly using SuSE SLES as my preferred Enterprise distro, but the RPMs provided by Dell for the HIT tools and MPIO had a load of dependencies that needed to be resolved before it was even possible to start using the extended functionality that these tools offer on SuSE.

So I downloaded a version of RH6.1 to test the new tools on, only to find that this didnt work as intended. After a lengthly support call, I was told that although the documentation said that it supported RH6 what this actually meant was 6.0. – So I downloaded 6.0 and had exactly the same issues…

So I did some digging around and debugging of the installation procedure and docs. The first thing that the docs tell you to do is make sure that you are at the latest supported release of 6 and to perform a yum_update. – This in itself is a problem, as it effectively takes the 6.0 installation and makes it 6.1. I tested this once I had a working 6.0 installation by doing the yum_update, and it completely broke the ability to address the iSCSI devices properly using the Dell tools. I found this a bit worrying, particularly if you are sharing your admin roles with someone else, they could inadvertently break the entire configuration.

The instructions for actually formatting and adding the device to the system seem to be a bit flaky at best, and I couldnt help feeling that somewhere there must be a ‘missing manual’

The documentation(for RedHat MPIO on EQL) states as a final step that you need to do the following(this is entirely absent from the HIT tools docs):

mke2fs –j –v /dev/mapper/rhel-test

this doesnt work at all using the HIT tools and tells you that the process could not read block 0, and gives you all sorts of wonderful messages about not being happy with the superblocks on the device. – This doesnt happen if you dont use the tools, but just use Linux native MPIO.

The only approach to this that worked, which was not resolved by my support experience, but by trial and error on my part (which was very time consuming) was as follows. Here’s my checklist:

 

Configure SCSI on the local onboard NICS on a dedicated DELL R710 and set 4 paths (edit the /etc/equallogic/equallogic.conf file and change the [MPIO] section to contain the maximum amount of path (NICS) you intend to use) – If you dont do this you could have problems in the future as for each NIC path you are given a corresponding /dev/sdX device which sits behind the mapper tool.

Some issues with the Dell HIT tool for EQL it doesnt work well on 6.1.

  • Yum update from 6.0 breaks a valid config as it effectively takes the server to 6.1!!!
  • Ignore the doc, dont do a localinstall of DKMS,EQL and iscsi-initiators. Also dont grab the latest version of DKMS as it is broken even though its available on the Dell site. – at the time of posting I used dkms-2.1.1.2-1.noarch.rpm, but had problems with dkms-2.2.0.0-1.noarch.rpm.

Grab kernel-devel and gcc, gcc-headers using yum
Configure iscsi.conf and CHAP. – enter in here the CHAP password you set on your EqualLogic
Enable iscsi logins in /etc/rc.local
Run eqltune -v and run through checklist and make the necessary alterations to the kernel config,etc. – This tool is your friend!

Run ehcmcli -d to show valid paths and sessions (you should see 4 paths here if you set your MPIO settings in /etc/equallogic/equallogic.conf  to 4).

Run rswcli -E -network <ip> -mask 255.255.255.0 to exclude the public NICS from accepting broadcast traffic from the EQL tool.
Use iscsiadm to discover the new LUNS and to log in the EQL LUN.
Next look in /dev/mapper for the eql-xxxxxxxx-volname LUN id. Make a note of its true dm-X number
To format the disk do not use the MPIO device. Its necessary to format the sdX devices that are available on each session. for this reason as well it means you need to set the config ahead of time at its maximum. It wont create more sessions as they are needed (this will break the LUN!!!).
do this in /etc/equalogic/eql.conf

Run ehcmcli to find out which sdX devices are underneath the MPIO layer.
next run:
kpartx -a /dev/dm-X (where dm-X is the dm id associated with the eql LUN and vol name)
sfdisk /dev/sdXX (do this for each sdX)
kpartx -a /dev/dm-X
Next: mkfs.ext3 /dev/mapper/eql-lun-vol-part
mount /dev/mapper/eql-lun-vol-part

I’ve also included my working multipath.conf file from my attempts to use native MPIO on RH6.1 and not the Dell tools. You will note that the way that the scsi tools (on RH6.1) interrogate the device has changed slightly and it now expects to see SEQLOGIC not EQLLOGIC, which could explain why the yum update breaks the tool.

There is also the suggestion to do the following in the lvm.conf file, but this renders you install unbootable if you are using LVM on your main filesystem.

filter = [ r|/dev/mapper/eql-[-0-9a-fA-F]*_[a-z]$| ]

/etc/multipath.conf

defaults {

        udev_dir                /dev

        polling_interval        10

        path_selector           “round-robin 0”

        path_grouping_policy    multibus

        getuid_callout          “/lib/udev/scsi_id –whitelisted –device=/dev/%n”

        prio                    const

        path_checker            directio

        rr_min_io               100

        flush_on_last_del       no

        max_fds                 8192

        rr_weight               priorities

        failback                immediate

        no_path_retry           fail

        queue_without_daemon    no

        user_friendly_names     yes

        mode                    644

        uid                     0

        gid                     disk

}

blacklist {

        devnode “^sd[ab]$”

        devnode “^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*”

        devnode “^hd[a-z][[0-9]*]”

}

#blacklist_exceptions {

#       devnode “^dasd[c-d]+[0-9]*”

#       wwid    “IBM.75000000092461.4d00.34”

#}

multipaths {

        multipath {

                wwid                    36090a0a890d37ea5f1df5414000040ac

                alias                   MyRedHatVol

        }

}

devices {

 device {

        vendor “SEQLOGIC”

        product “100E-00”

        path_grouping_policy multibus

        getuid_callout “/sbin/scsi_id -p 0x83 -gus /block/%n”

        features “1 queue_if_no_path”

        path_checker readsector0

        failback immediate

        path_selector “round-robin 0”

        rr_min_io 10

        rr_weight properties

}

}

 

I’m sure theres other ways to sort this out, but the above checklist is what worked for me..

Check out the EqualLogic Family here, and grab yourself one!

Matt Palmer 03 August 2011

One thought on “Configuring Dell EqualLogic PS6500 Array to work with RedHat Linux 6 EL”

  1. I received the following email as feedback from a Technical source at Dell, which I thought I would share in the interests of promoting the development and success of the HIT tools for EqualLogic on the Linux platform.

    Hello,

    My name is John Harrigan and I work at Dell EqualLogic in the Technical
    Marketing organization. I have authored a Technical Report which covers
    usage of the Dell EqualLogic Host Integration Tools for Linux (HIT/LE)
    software. The Technical Report can be downloaded here:
    http://www.equallogic.com/resourcecenter/assetview.aspx?id=10311

    I encourage you to continue to work with Dell EqualLogic Customer
    Support. But I thought it would also be helpful for me to respond to
    your post regarding your experiences with HIT/LE.
    I have organized my responses into these areas:

    1) HIT/LE installation and configuration
    – DKMS v2.2 regression
    There was a problem with the v2.2.0.0 release of DKMS, but we worked
    with the maintainers to get a patch out quickly (v2.2.0.1). In
    parallel to working with the DKMS maintainers, the Dell support
    organization was notified of the DKMS v2.2.0.0 issue and provided
    a workaround procedure for customers. The currently posted version
    of DKMS v2.2 resolves the problem.

    – Manually editing the session counts
    It’s not necessary to manually edit the session counts. The default
    values work fine in most customer environments. Can you provide more
    info on why you felt it was necessary to change these defaults in
    your environment?

    – SLES not supported
    We are looking at supporting additional Linux distributions in future
    releases of HIT/LE.

    – RHEL 6.1 not supported
    When HIT/LE was released, it was tested against RHEL5.5 and RHEL 6
    which were available at the time. While we haven’t repeated the
    formal QA cycle, our in-house usage confirms that HIT/LE functions
    properly on subsequent RHEL releases (e.g. RHEL5.6).

    – RHEL 6.1 multipath.conf issues
    HIT/LE does not access or use the multipath.conf file. I am not sure
    how you ran into these issues. In order to configure Linux
    native multipath I/O, the user must manually make several edits to
    the multipath.conf file. The default multipath.conf file included
    in the RHEL distribution does not include a ‘device’ entry for
    EqualLogic. If you are seeing problems due to an incorrect device
    identifier ‘SEQLOGIC’, you should check with whoever you got your
    modified multipath.conf from.

    – eqltune is your friend
    Thank you for your kind words. We agree 🙂

    2) Difficulty creating a filesytem
    It sounds as though you may be attempting to use the Linux native MPIO
    devices in conjunction with HIT/LE. We have a specific set of
    instructions in our release notes on how to migrate from a standard
    multipathing configuration to HIT/LE (you can even do this without
    rebooting or other interruption on a Linux system). When using HIT/LE
    you should never directly access the individual MPIO devices (sdX).
    Rather, you should access the dm-switch device which is found in the
    /dev/mapper directory (eql-xxxxxx-volname).

    3) LVM exclusion filter affects bootability
    Would you please clarify if you are trying to use HIT/LE on a SAN
    booted system? That functionality is unsupported in HIT/LE as noted
    in the Release Notes.

    Several of my responses refer to HIT/LE documentation. The HIT/LE
    installation procedure copies the product documentation to this
    directory: /usr/share/doc/equalllogic-host-tools-1.0.0.0/doc

    The HIT/LE package and documentation are available at this URL:
    https://www.equallogic.com/support/download.aspx?id=10235

    NOTE: Access to this link requires a Dell EqualLogic customer support
    account.

    Thank you for choosing Dell!

    ———————————————————————–
    John Harrigan (w) 603.589.5507 [email protected]
    Dell EqualLogic – Technical Marketing Engineering – Linux