Softpanorama

Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
May the source be with you, but remember the KISS principle ;-)
Bigger doesn't imply better. Bigger often is a sign of obesity, of lost control, of overcomplexity, of cancerous cells

Procedure for installing Qlogic SAN cards in Suse 10 & 11

News

Linux Disk Management

Linux Networking

Recommended Links

QLogic cards

installing QLE2460

LVM
Manage Your Disk By UUID System Configuration Configuring multipath-tools Enabling the Multipath Deamon Querying the Status Tuning the Failover Managing IO
Software RAID udev Trunking / Bonding Solaris Multipathing Humor Etc

  1. Unpack the cards and write down the service tags of cards from Dell boxes.
     
  2. Install the cards on the server . Connect them to SAN disk storage
     
  3. Create LUN drives and reboot. Check if cards are recognized
    # cd /proc
    # find . -name "qla*"
    ./irq/69/qla2xxx (rsp_q)
    ./irq/68/qla2xxx (default)
    ./irq/67/qla2xxx (rsp_q)
    ./irq/66/qla2xxx (default)	
  4. Use rescan-scsi-bus.sh. Allocated disks should now be visible
     
  5. Fully patch the system.  For Suse 11 it should be SP1 with latest patches. The alternative is to follow instruction from Qlogic installation disk and install the latest drivers for the cards
     
  6. Reboot the system
     
  7. On versions Suse before Suse 10 SP2 detect UUID for /boot partition and change it in /etc/fstab
     
  8. Enable two multipath daemons boot.multipath and multipathd:
    chkconfig multipathd on
    chkconfig boot.multipath on
  9. Exclude local drives (sd* or cciss* ) from multipath daemon in /etc/multipath.conf. This is pretty tricky and you can make system unbootable if you screw the  /etc/multipath.conf 
  10. Change driver order in /etc/sysconfig/kernel and insert dm-multipath if necessary and move qla2xxx to the last position of the list, for example:
    INITRD_MODULES="piix megaraid_sas mptspi siimage processor thermal fan jbd ext3 dm_mod edd dm-multipath qla2xxx"
  11. Recompile initrd
    /sbin/mkinitrd -f mpath

For servers with root partition on LVM

  1. Uncheck boot.evms
    chconfig boot.evms off
    chkconfig boot.lvm on
  2. Modify /etc/lvm/lvm.conf
old: filter = [ "r|/dev/.*/by-path/.*|", 
		"r|/dev/.*/by-id/.*|", "a/.*/" ]
new: filter = [ "a|/dev/disk/by-id/.*|", "r|.*|" ]

Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Nov 10, 2011] Adding and Removing SAN Disks from SUSE Device Manager

January 23, 2009 | Novell User Communities

Remove a disk

      echo 1 > /sys/block/sdX/device/delete
      echo 1 > /sys/block/sdY/device/delete	  

You should now have all traces removed, you can run multipath -ll and cat /proc/scsi/scsi to cross check. You can now remove the mapping from the SAN and delete the logical volume if required.

[Nov 10, 2011] How to setup - use multipathing on SLES

SLES 9 information -- outdated

The boot scripts will only detect MPIO devices if the modules for the respective controllers are loaded at boot time. To achieve this, simply add the needed driver module to the variable INITRD_MODULES within the file /etc/sysconfig/kernel.

Example:

Your system contains a RAID controller that is accessed by the cciss driver and you are using ReiserFS as a filesystem. The MPIO devices will be connected to a Qlogic controller accessed by the driver qla2xxx, which is not yet configured to be used on this system. The mentioned entry within /etc/sysconfig/kernel will then probably look like this:

INITRD_MODULES="cciss reiserfs"

Using an editor, you would now change this entry:

INITRD_MODULES="cciss reiserfs qla2xxx"

When you have applied this change, you will need to recreate the INITRD on your system to reflect it. Simply run this command:

mkinitrd

When you are using GRUB as a bootmanager, you do not use to make any further changes. Upon the next reboot the needed driver will be loaded within the INITRD. If you are using LILO as bootmanager, please remember to run it once to update the boot record.

  • Configuring multipath-tools

    If your system is one of those listed above, no further configuration should be required.

    You might otherwise have to create /etc/multipath.conf (see the examples under /usr/share/doc/packages/multipath-tools/) and add an appropriate devices entry for your storage subsystem.

    One particularly interesting option in the /etc/multipath-tools.conf file is the "polling_interval" which defines the frequency of the path checking that can be configured.

    Alternatively, you might choose to blacklist certain devices which you do not want multipath-tools to scan.

    You can then run:

    multipath -v2 -d

    to perform a 'dry-run' with this configuration. This will only scan the devices and print what the setup would look like.

    The output will look similar to:

    3600601607cf30e00184589a37a31d911
    [size=127 GB][features="0"][hwhandler="1 emc"]
    \_ round-robin 0 [first]
      \_ 1:0:1:2 sdav 66:240  [ready ]
      \_ 0:0:1:2 sdr  65:16   [ready ]
    \_ round-robin 0
      \_ 1:0:0:2 sdag 66:0    [ready ]
      \_ 0:0:0:2 sdc  8:32    [ready ]
    

    showing you the name of the MPIO device, its size, the features and hardware handlers involved, as well as the (in this case, two) priority groups (PG). For each PG, it shows whether it is the first (highest priority) one, the scheduling policy used to balance IO within the group, and the paths contained within the PG. For each path, its physical address (host:bus:target:lun), device nodename and major:minor number is shown, and of course whether the path is currently active or not.

    Paths are grouped into priority groups; there's always just one priority group in active use. To model an active/active configuration, all paths end up in the same group; to model active/passive, the paths which should not be active in parallel will be placed in several distinct priority groups. This normally happens completely automatically on device discovery.

  • Enabling the MPIO components

    Now run

    /etc/init.d/boot.multipath start
    /etc/init.d/multipathd start

    as user root. The multipath devices should now show up automatically under /dev/disk/by-name/; the default naming will be the WWN of the Logical Unit, which you can override via /etc/multipath.conf to suit your tastes.

    Run

    insserv boot.multipath multipathd

    to integrate the multipath setup into the boot sequence.

    From now on all access to the devices should go through the MPIO layer.

  • Querying MPIO status

    To query the current MPIO status, run

    multipath -l

    This will output the current status of the multipath maps in a format similar to the command already explained above:

    3600601607cf30e00184589a37a31d911
    [size=127 GB][features="0"][hwhandler="1 emc"]
    \_ round-robin 0 [active][first]
      \_ 1:0:1:2 sdav 66:240  [ready ][active]
      \_ 0:0:1:2 sdr  65:16   [ready ][active]
    \_ round-robin 0 [enabled]
      \_ 1:0:0:2 sdag 66:0    [ready ][active]
      \_ 0:0:0:2 sdc  8:32    [ready ][active]

    However, it includes additional information about which priority group is active, disabled or enabled, as well as for each path whether it is currently active or not.

  • Tuning the fail-over with specific HBAs

    HBA timeouts are typically setup for non-MPIO environments, where longer timeouts make sense - as the only alternative would be to error out the IO and propagate the error to the application. However, with MPIO, some faults (like cable failures) should be propagated upwards as fast as possible so that the MPIO layer can quickly take action and redirect the IO to another, healthy path.

    For the QLogic 2xxx family of HBAs, the following setting in /etc/modprobe.conf.local is thus recommended:

    options qla2xxx qlport_down_retry=1 ql2xfailover=0 ql2xretrycount=5
  • Managing IO in error situations

    In certain scenarios, where the driver, the HBA or the fabric experiences spurious errors,it is advisable that DM MPIO is configured to queue all IO in case of errors leading loss of all paths, and never propagate errors upwards.

    This can be achieved by setting

    defaults {
    		default_features "1 queue_if_no_path"
    }

    in /etc/multipath.conf.

    As this will lead to IO being queued forever, unless a path is reinstated, make sure that multipathd is running and works for your scenario. Otherwise, IO might be stalled forever on the affected MPIO device, until reboot or until you manually issue a

    dmsetup message 3600601607cf30e00184589a37a31d911 0 fail_if_no_path				

    (substituting the correct map name), which will immediately cause all queued IO to fail. You can reactivate the queue if no path feature by issueing

    dmsetup message 3600601607cf30e00184589a37a31d911 0 queue_if_no_path
    

    You can also use these two commands to switch between both modes for testing, before committing the command to your /etc/multipath.conf.

    4. Using the MPIO devices

    • Using the whole MPIO devices directly

      If you want to use the whole LUs directly (if for example you're using the SAN features to partition your storage), you can simply use the /dev/disk/by-name/xxx names directly for mkfs, /etc/fstab, your application, etc.

    • Using LVM2 on top of the MPIO devices

      To make LVM2 recognize the MPIO devices as possible Physical Volumes (PVs), you will have to modify /etc/lvm/lvm.conf. You will also want to modify it so that it does not scan and use the physical paths, but only accesses your MPIO storage via the MPIO layer.

      Thus, change the "filter" entry in lvm.conf as follows and add the types extension to make LVM2 recognize them:

      filter = [ "a|/dev/disk/by-name/.*|", "r|.*|" ]
      types = [ "device-mapper", 1 ] 

      This will allow LVM2 to only scan the by-name paths and reject everything else. (If you are also using LVM2 on non-MPIO devices, you will of course need to make the necessary adjustments to suit your setup.)

      You can then use pvcreate and the other LVM2 commands as usual on the /dev/disk/by-name/ path.

    • Partitions on top of MPIO devices

      It is not currently possible to partition the MPIO devices themselves. However, if the underlying physical device is partitioned, the MPIO device will reflect those partitions and the MPIO layer will provide /dev/disk/by-name/>name<p1 ... pN devices so you can access the partitions through the MPIO layer.

      So you will have to partition the devices prior to enabling MPIO; if you change the partitioning in the running system, MPIO will not automatically detect this and reflect the changes; you will have to reinit MPIO, which in a running system, with active access to the devices, will likely imply a reboot.

      Thus, using the LUNs directly or via LVM2 is recommended.


  • Recommended Links

    Google matched content

    Softpanorama Recommended

    Top articles

    Sites

    Internal

    External

    SUSE Linux Enterprise Desktop 10 Release Notes

    Multipath-usage.txt File for Red Hat Enterprise Linux 4 Update 3

    Fun with your SAN and Multi-path " SUSE Linux Enterprise in the Americas

    Linux Multipath Howto (RHAS4)

    Device Mapper Resource Page
    List of sources and related packages, etc...
    How do I setup device-mapper multipathing in Red Hat Enterprise Linux 4?
    Redhat Knowledge Base, article id: 7170
    Basically the same as this page, but more generic.
    How do I make device mapper multipath ignore my local disks when generating the multipath maps in Red Hat Enterprise Linux 4?
    Redhat Knowledge Base, article id: 7319
    How can I add more products into the multipathing database?
    Redhat Knowledge Base, article id: 8131
    Device Mapper Resource Page
    List of sources and related packages, etc...
    How do I setup device-mapper multipathing in Red Hat Enterprise Linux 4?
    Redhat Knowledge Base, article id: 7170
    Basically the same as this page, but more generic.
    How do I make device mapper multipath ignore my local disks when generating the multipath maps in Red Hat Enterprise Linux 4?
    Redhat Knowledge Base, article id: 7319
    How can I add more products into the multipathing database?
    Redhat Knowledge Base, article id: 8131



    Etc

    Society

    Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

    Quotes

    War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

    Bulletin:

    Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

    History:

    Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

    Classic books:

    The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Haterís Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

    Most popular humor pages:

    Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

    The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D


    Copyright © 1996-2018 by Dr. Nikolai Bezroukov. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) in the author free time and without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

    FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

    This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

    You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info

    Disclaimer:

    The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

    The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

    Last modified: October 03, 2017