Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-) Softpanorama Search

AIX Logical Volume Manager

LVM stands for Logical Volume Manager and JFS is a Journaled File System.

The role of LVM is to present a simple logical view of underlying physical storage space, ie. harddrive(s). LVM manages individual physical disks - or to be more precise, the individual partitions present on them (for a short glossary of terms, look at the end of the article). LVM hides the numbers, size and location of physical partitions from users. Instead it presents the concept of logical volume. A logical volume may correspond to a physical partition (but that obviously almost defeats the purpose of LVM) but it doesn't have to. One volume may be composed of several partitions located on multiple physical disks. Not only that, the volumes can even be extended (not shrunk - people usually want more space, not less). They can even be extended while the OS is running and the filesystem is being accessed! Of course, most home and SOHO users don't have the hardware required for this.

The more experienced readers are now probably wondering how 'traditional' file systems like FAT or HPFS could be extended at runtime. The answer is, they can't. To take full advantage of LVM, it is necessary to use a filesystem designed for it. This file system is of course JFS. JFS is not really tied to LVM, both LVM and JFS can exist separately, but only when working in concert both can reach their full potential.

Physical View of the LVM

The following is a list of the common features in the LVM and VxVM methods of storage management:

• Online backup of file systems
• Snapshot backup of file systems
• Quick resynchronization of changed partitions after online or snapshotbackup
• Migration of data with active volumes
• Transparent data stream switch after mirrored disks fail
• Ability and limits to operate in a multinode concurrent configuration
• Commands to replace dead or failing drives
• Hot spare, standby disks

Both VxVM and LVM enable file systems to be expanded while the file system is mounted and in use. Both volume managers are tied closely to specific file systems. VxVM only works with the Veritas File System, and LVM only works with the JFS or JFS2 (enhanced JFS) file system found on AIX. The one difference between VxVM and LVM is the philosophy on the concept of file system shrinkage. LVM/JFS/JFS2 does not allow the user the ability to shrink the file system. The reasoning behind this is that sometimes portions of files are spread out over the file system for performance reasons. And when a file system needs to be shrunk, the file system must go out and gather up the "pieces" of a file and relocate them to a portion of the file system that will not be eliminated. This method is time-consuming and not very efficient for performance. Thus on AIX, file system reduction is not permitted on JFS and JFS2.

VxVM takes a different approach to this. It does allow the reduction of a file system size, but it is very simplistic and has drawbacks. They literally "chop" off the end of the logical volume and file system up to the point where the user wants the file system reduced. Warnings are included in the VxVM product regarding this practice; if the user has data at the end of the file system, the data may be corrupted or lost. Thus, for the purposes of this paper, the ability of VxVM to reduce file system size is considered limited.

Terminology:

Physical Volume (PV)

Synonym for "hard disk". A single physical hard drive.

Volume Group (VG)

A set of one or more PVs which form a single storage pool. You can define multiple VGs on each AIX system.

Physical Partition (PP)

The smallest allocation unit in the LVM. All PPs within a VG are the same size, usually 4 or 8 MB.

Logical View of the LVM

Terminology:

Volume Group (VG)

A set of one or more PVs which form a single storage pool. You can define multiple VGs on each AIX system.

Logical Partition (LP)

A logical mapping to one or more PPs within the same VG, regardless of whether they're on the same PV or not.

Logical Volume (LV)

A set of one or more LPs within the same VG which form a usable unit of disk space. LVs are used analogously to partitions on PCs or slices under Solaris: they usually contain filesystems or paging spaces ("swap").

Big Picture of the LVM

Advantages:

• Filesystems can span disks, so size is not limited by capacity of physical disks.
• Can expand filesystems on the fly.
• Can add additional disks to existing pool of disk space (VG).
• Can mirror important data on multiple physical disks for redundancy.
• Can "export" an entire VG so that all disks in the VG can be easily physically disconnected, moved to another machine, and "imported".

Limitations:

• Must reducevg when removing disk.
• When a single disk in a VG dies, the whole VG is affected.
• "Brick Wall" between VGs - LVs can't cross VG boundary.
• Cannot shrink filesystems.

• Configuring for performance or flexibility vs. reliability or fault-tolerance.
• Strategy: Define adequate but not huge filesystems, leaving pool of free storage for later assignment.
• Seperate out data categorically onto different VGs, so that an OS upgrade will not touch user data, or so that user data can be easily moved to another machine.

Filesystems

• /etc/filesystems - AIX's version of /etc/fstab. Uses stanza format, * is comment character.
• Useful commands:
  • crfs - Creates logical volume, makes filesystem on logical volume, adds entry to /etc/filesystems. Does not mount the new filesystem for you!
  • chfs - Changes settings of filesystem, logical volume, or /etc/filesystems entry.
  • rmfs - Removes filesystem, logical volume, entry from /etc/filesystems, and optionally the mount point.
• AIX itself is always installed in a VG called rootvg
• The jfs filesystem is a journaling filesystem, meaning that it keeps a summary of filesystem update information on a special LV of type jfslog

Example: Adding a Disk

1. Physically attach the disk to the system. Make sure there are no SCSI ID conflicts.
2. Run cfgmgr to detect new device. Device will be installed as hdiskn, where n is some integer.
3. Run extendvg vgname hdiskn to add the disk to a VG named vgname.
4. To create a new filesystem on this VG, use crfs -v jfs -g vgname -A yes -a size=262144. (Note that the size parameter indicates the number of 512-byte blocks, so 262144 means 128MB.)
5. To add 128MB to an existing filesystem on this VG, use chfs -a size=+262144 /filesystem/name.

Old News ;-)

[Apr 20, 2009] AIX Support Tips

File system - allocate more space

• chfs -a size='+500' /usr/acct (To enlarge file system)
• Use chlv -x 256 <lv-name> if you get an errorwhen running the above command (To expand beyond maximum LV's)

[Apr 15, 2009] [PDF] Logical Volume Manager Basics

Good SHARE presentation

[Apr 14, 2009] IBM developerWorks Wikis - AIX - LVM

Logical Volumes striping

• From AIX 4.3.3 (not backward compatible) LVM support striping
• Often called fine striping (i.e. not PP level striping)
• Recommend 32K, 64K or 128K stripe width = 4 to 16 times I/O size most databases do 4K or 8K I/O
• Excellent for balancing serial and random I/O across disks
• Excellent read ahead serial I/O boost

Using LVM commands (with smit)

• Find unused disks: lspv
• Add them to a VG or create a VG: smitty lvm + VG + Add VG
• Check unused space in VG: lsvg <vgname>
• Create a LV: smitty lvm + LV + Add LV
• Create a Filesystem (JFS)smitty jfs + Add jfs + standard + which VG (see below)

If the command changes the VG, PV or LV then the VG is locked i.e. one command at a time.
This is needed to allow recovery of the change - if the system fails during the update.

Creating Stripes Tips:

• Stripes LV must contain a whole number of items, that is the number of PPs must divide evenly into the number of disks
• Or put another way: each disk have the same number of PPs
• Hint: use multiples of No. of disks to make this easier

LVM Create Mirrors

There are three ways to create a mirror:

1. One Step = create LV with mirror
   • These can be slow as you can have to wait until mirror complete before creating next lv
2. Create LV and later add mirror synchronise=yes
   • again slow as you can have to wait for silvering
     #Create LV and later add mirror synchronise=no and then sychronise all mirrors together
   • Use this when creating lots of lv because all mirrors can be created at the same time

So (with smit)

1. create lv's
2. add mirrors sync=no
3. syncvg -l <lvname>
   to remove "stale" partitions = resilver
   Warning:

• resilvering hurts performance
• often forgotten when recommending mirror breaking for backup purposes

LVM Higher Level Management Commands

Disown disks ready to connect them to another system

• varyoffvg <vgname>
• exportvg <vgname>
  or smitty lvm

Activate newly attached disks

• cfgmgr – finds disks
• importvg hdiskN – any one of the disks
• varyonvg <vgname> – mounts file systems

Moving LVs

migratelv (not striped)

• Use smit lvm
• Can migrate LV on a disk to another disk
• Can empty whole disk to remove it later or replac a fault disk
  Disks can be added and removed from a VG
• extendvg and reducevg
• Use smit lvm recommended

LVM rootvg good practice

Small rootvg - 1 or 2 disks (for mksysb backup)
Mirror rootvg for AIX disk protection

• Add disk hdisk1
• chvg -Qn rootvg
• For hd1 to hd8 and hd9var copy the LV
• mklvcopy hd{1,2,3,4,5,6,7,8,9var} 2 hdisk1
• syncvg -v rootvg
• bosboot -a
• bootlist -m normal hdisk0 hdisk1
  Or
• mirrorvg -s rootvg
• this second option is recommended 

TIP: This command will show you formatted list of active Volume groups

for i in $(lsvg -o);do lsvg $i;done \
| awk ' BEGIN { printf("%-10s\t%10s\t%10s\t%10s\t%10s\n","VG","Total(MB)","Free","USED","Disks") };/VOLUME GROUP:/ \
{ printf("%-10s\t", $3) };/TOTAL PP/ { B=index($0,"(") + 1;E=index($0," megaby");D=E-B;printf("%10s\t", substr($0,B,D) );};/FREE PP/ \
{ B=index($0,"(") + 1;E=index($0," megaby");D=E-B;printf("%10s\t", substr($0,B,D) );};/USED PP/ \
{ B=index($0,"(") + 1;E=index($0," megaby");D=E-B;printf("%10s\t", substr($0,B,D) );};/ACTIVE PV/ { printf("%10s\t\n", $3) } '
 

Put other disks into other volume groups

• use min PP size ( 8MB or 16 MB)
• use 16 to 32 disks per volume group
• use different vg for data that might be moved elsewhere
  Avoid hot disks
• Spread data across disk with LVs or RAID5
• Mirrors add protection and performance
• Use striping whenever possible
• Think spindles not disk size
  reorgvg
• this will move physical partitions (PP) around in an attempt to get PPs in the right place
  migratepv
• this can be used to spread PPs across move disks
• also to remove all PPs from a suspect disk before replacement (due to errors being reported)

AVOIDING Disk Crashes

1. Rule One: RAID5 or mirror everything
2. Rule Two: monitor error logs. Make sure you know when a disk failed
3. Rule Three: call hardware support. That is what they are for
4. Rule Four: Don't meddle.  Only try, if you know what you are doing
5. Rule Five: Read and practise. Get the Redbooks and try it safely

Recommended Links

http://www.networktechnologist.com/tips-aix-lvm.html
http://web.utanet.at/mario/exam/5129fm.htm#tocbrvH1 - old (AIX 4.3 but good description) http://www.redbooks.ibm.com/redbooks/SG246584/ 

Chapter 5. Device management

5.1 Overview

5.2.2 Managing device drivers

5.2.3 Configuring a device

5.2.4 Adding a new device to a SCSI bus

5.2.5 Remove a SCSI device

5.3 Device management in AIX 5L Version 5.1

5.3.1 Listing devices

5.3.3 Removing a device

5.3.4 Changing a device

5.4 Quick reference

Chapter 6. Logical Volume Manager and disk management

6.1 Logical volume management overview

6.2 Introducing the logical volume solutions

6.2.1 Solaris Solstice DiskSuite: Introduction

6.2.2 VERITAS Volume Manager: Introduction

6.2.3 AIX 5L Version 5.1 LVM: Introduction

6.3 Working with logical volume manager

6.3.1 Volume groups

6.3.2 Working with logical volumes

6.3.3 Working with physical disks

6.3.4 Additional features: Hotspare disks

6.4 Quick reference

Chapter 7. File system management

7.1 Overview

7.1.1 Solaris file systems types and commands

7.1.2 AIX file systems types and commands

7.2 Formatting and partitioning a disk (Solaris only)

7.3 Creating a file system

Laurent Vanel, Ronald van der Knaap, Dugald Foreman (2000). AIX Logical Volume Manager from A to Z: Introduction and Concepts .

IBM Redbooks | AIX Logical Volume Manager from A to Z ...

AIX Logical Volume Manager, from A to Z: Introduction and Concepts

Rosetta Stones

• Compares: AIX Logical Volume Manager and Veritas Volume Manager (PDF)
• Compares: AIX 5.1 and Solaris 8 (PDF)
• Compares: AIX, FreeBSD, HP-UX, Redhat, Solaris, Tru64 (PDF)
• Compares: Everything and the kitchen sink (PDF)
  

 Naming conventions for LVM Use Table 6-5 to find out the names that LVM uses for each of its components. Table 6-5   Naming conventions for LVM 6.3  Working with logical volume manager In this section, we will describe the way to list, create, remove, and change characteristics for volume groups, logical volumes, and physical disks. We mainly use VxVM and LVM, because not all the features in these two products are available in Solaris DiskSuite. 6.3.1  Volume groups As defined earlier, a volume group is a collection of physical disks that are related. Let us review the main and most important ways to work with them. Listing a volume group: LVM In AIX 5L Version 5.1, we use lsvg to list a volume group. If you do not specify an option, it will show you all the volume groups defined in the system. Some of its useful flags are: -o Shows only the active volume groups. -p <vg_name> Shows all the physical volumes that belong to the requested volume group (vg_name).

 Logical Volume Manager (LVM) Commands for AIX

Glossary

Command Summary

Sample LVM Procedures:

Filesystem Procedures

• See below the procedure for creating a logical volume and a filesystem using JFS:

1. "df" to see the filesystem, it's current size, % utilization and the name of it's logical volume
2. "lslv <logical_volume>" to show information about the logical volume including it's volume group name.
3. "lsvg <volume_group>" to show information about the volume group, including number of free pp's and the pp size
4. If there are not enough free pp's then see below for procedure to add a disk to a volume group.
   
5. "chfs -a size= +4194304 <MOUNT_POINT>" to grow the filesystem by 2 GB (4194304=2*1024*1024*1024/512)
   • NOTE:  Growing the file system will automatically grow the logical volume
6. df" shows the file system's current size is 2 GB more than before.

• Error Message:  0516-787 extendlv: Maximum allocation for logical volume <LV_Name> is 512.
  • Maximum number of LPs for the logical volume has been exceeded - must increase the allocation
  • Calculate the number of LPs needed = LV Size in MB / LP size in MB
  • chlv -x <new_max_lps> <logical_volume>

1. Unmount the filesystem
   
2. Remove the logical volume "rmlv <lv_name>"
3. Remove the filesystem information from /etc/filesystems

1. Create the file system
   1. crfs -v jfs -m /usr/sharenew -g rootvg -a size=8192
   2. this makes a logical volume in the root volume group of 4MB that uses jfs
2. Mount the volume
   1. mount /usr/sharenew
3. Move the files from the old file system (/usr/shareold)
   1. cd /usr/shareold
   2. tar cf - | (cd /usr/sharenew; tar xvf -)
   3. cd
4. Unmount the file systems
   1. umount /usr/sharenew
   2. umount /usr/shareold
5. Remove the old file system and it's logical volume
   1. rmfs /usr/shareold
6.   
   1. chfs -m /usr/shareold /usr/sharenew
7. Mount the new filesystem
   1. mount /usr/shareold
8. Delete the temporary mount point
   1. rmdir /usr/share

Logical Volume Procedures

1. lsvg to determine the size of the PP
2. lslv in similar logical volumes to determine if mirroring is in effect
   
3. Calculate the number of PPs needed for the logical volume
   
   1. bc
   2. scale=2
   3. <size of lv in MB>/<size of PP in MB>
   4. quit
      
4. mklv -y  "<LV_NAME>" <VG_NAME> <# of LPS>  --> creates the logical volume
5. crfs -v jfs -d <LV_NAME> -m /<MOUNTPOINT> -A yes   --> makes the filesystem, creates the mountpoint and puts it in /etc/filesystems
6. mount /<MOUNTPOINT>  --> mounts the new fileystem
7. df /<MOUNTPOINT>  --> verifies the mount and the size of the new filesystem
8. Check the ownership and permissions of the new mount point
   • ls -ld <mountpoint>
   • chown owner:group <mountpoint>
   • chmod XXX <mountpoint>
     
9. If mirroring is in effect, then mirror this logical volume to another disk (original and 1 mirror):
   • mklvcopy -s y <LV_NAME> 2
     

• lsvg -l

•  mklvcopy -s y <LV_NAME> 2

Volume Group Procedures

1. lsdev -C -c disk  -> lists available disks (and the hdisk#) on the server
2. mkvg -y "<VG_NAME>" hdisk#  --> creates the volume group on the named hard disk
3. varyonvg <VG_NAME>  --> activates the volume group

• extendvg <vg> <disk#>
  • Verify the disk has been successfully added to the vg
• lsvg -p <vg>

1. lspv  --> determine the hdisk#
   
2. extendvg rootvg hdisk<number>  --> add the hdisk to the volume group
   
3. lspv  -->  verify that the hdisk has been successfully added to the volume group
   
4. chvg -Q 'n' rootvg  -->  change the quorum so that the vg will stay active if one of the mirrors fail
   
5. mirrorvg -S -c 2 rootvg  --> mirror all of the logical volumes in the volume group
   
6. lsvg -l rootvg  --> verify successful mirroring (pps will appear "stale" until synchronization is complete).
   
7. bosboot -a  -->  update the boot image information
   
8. bootlist -m normal -o hdisk0 hdisk1  --> create a new bootlist
   
9. bootlist -m normal -o  --> verify the bootlist is correct
   

1. "lsvg <volume_group>" to show the total PP's available in the volume group =1250
2. "lsvg -l <volume_group>" to show the total PP's used in all logical volumes in that volume group (showed sys1log, the jfs log was using 2 PP's)
3. "chlv -x 1248 <logical_volume>" to change the maximum number of LP's from 1100 to 1248 (1250 PP's in the volume group - 2 PP's used by the jfs log  = 1248 available)

Physical Disk Procedures

• lsvpcfg
  • This will show disks/vpaths and the volume group they are allocated to
• lspv|grep None
  • This will show pvs and whether they are asssociated with a volume group
  • Note:  For vpaths, the hdisks will show as none, but they may be allocated to a vpath - you must grep each hdisk with the lsvpcfg

• Allocate the new lun on the SAN
• Run "cfgmgr"
• Verify the new vpatch/hdisk by running "lsvpcfg"
  • There should be a new vpath and it should be available with no volume group - if not, rerun cfgmgr

Procedure to list the PVs in a volume group:

• lsvg -p <volume group>