Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-) Softpanorama Search

AIX Logical Volume Manager

LVM stands for Logical Volume Manager and JFS is a Journaled File System.

LVM and JFS didn't originate on OS/2. They were created for AIX and used for OS/2.  

The role of LVM is to present a simple logical view of underlying physical storage space, ie. harddrive(s). LVM manages individual physical disks - or to be more precise, the individual partitions present on them (for a short glossary of terms, look at the end of the article). LVM hides the numbers, size and location of physical partitions from users. Instead it presents the concept of logical volume. A logical volume may correspond to a physical partition (but that obviously almost defeats the purpose of LVM) but it doesn't have to. One volume may be composed of several partitions located on multiple physical disks. Not only that, the volumes can even be extended (not shrunk - people usually want more space, not less). They can even be extended while the OS is running and the filesystem is being accessed! Of course, most home and SOHO users don't have the hardware required for this.

The more experienced readers are now probably wondering how 'traditional' file systems like FAT or HPFS could be extended at runtime. The answer is, they can't. To take full advantage of LVM, it is necessary to use a filesystem designed for it. This file system is of course JFS. JFS is not really tied to LVM, both LVM and JFS can exist separately, but only when working in concert both can reach their full potential.

JFS volume structure

• capacity - JFS allows much larger file and volume sizes than HPFS. Basically JFS is a 64-bit file system while HPFS structures are at most 32 bits large.
• recovery - thanks to the journaling techniques employed by JFS (described in more detail later), CHKDSK times for JFS are significantly faster than for equivalent HPFS volumes. Roughly speaking, where HPFS checkdisk after a crash takes minutes, JFS takes seconds.

• the superblock
• the i-nodes
• the data blocks
• the allocation groups

The entire file system space is divided into logical blocks that contain file or directory data. For JFS, the logical blocks are always 4096 bytes (4K) in size, but can be optionally subdivided into smaller fragments (512, 1024 or 2048 bytes).

An i-node is a logical entity that contains information about a file or directory. There is a 1:1 relationship between i-nodes and files/directories. An i-node contains file type, access permissions, user/group ID (UID/GID - unused on OS/2), access times and points to actual logical blocks where file contents are stored. The maximum file size allowed in JFS is 2TB (HPFS and FAT allow 2GB max). It should be noted that the number of i-nodes is fixed. It is determined at file system creation (FORMAT) time and depends on fragment size (which is user selectable). In theory users could run out of i-nodes, meaning that they would be unable to create more files even if there was enough free space. In practice this is extremely rare.

Fragments were already briefly mentioned in the discussion of logical blocks. The JFS logical block size is fixed at 4K. This is a reasonable default but it means that the file system cannot allocate less than 4K for file storage. If a file system stores large amounts of small files (< 2K), the disk space waste becomes significant. We've all got to know and hate this problem from FAT (cluster size of 32K leads to massive waste of space, in some cases over 50%). JFS attacks this by allowing fragmentation of logical blocks into smaller units, as small as 512 bytes (this is sector size on harddrives and it is not possible to read or write less than 512 bytes from/to disk). However users should be careful because fragmentation incurs additional overhead and hence slows down disk access. I would recommend using fragments smaller than 4K only when the users know for sure that they will store very large amounts of small files on the file system.

The entire JFS volume space is subdivided into allocation groups. Each allocation group contains i-nodes and data blocks. This enables the file system to store i-nodes and their associated data in physical proximity (HPFS uses a very similar technique). The allocation group size varies from 8MB to 64MB and depends on fragment size and number of fragments it contains.

Journaling

JFS uses a special log device to implement circular journal. On AIX, several JFS volumes can share single log device. I'm not sure this is possible on OS/2, I believe each JFS volume (corresponding to a drive letter) has its own 'inline' log located inside the JFS volume - its size is  selectable at FORMAT time.

It is important to note that JFS does not log (or journal) everything. It only logs all changes to file system meta-data. Simply speaking, the log contains a record of changes to everything in the file system except actual file data, ie. changes to the superblock, i-nodes, directories and allocation structures. It is clear that there must be some overhead here and indeed, performance may suffer when applications are doing lots of synchronous (uncached) I/O or creating and/or deleting many files in short amount of time. The performance loss is however not noticeable in most cases and is well worth the increased security.

The log (or journal) occupies a dedicated area on disk and is written to immediately when any meta-data change occurs. When the disk becomes idle, the actual file system structure is updated according to the log. After a crash, all it usually takes to restore the file system to full consistency is replaying the log, ie. performing the recorded transactions. Of course, if a process was in the middle of writing a file when the system crashed or power died, the file could be inconsistent (the app might not be able to read it again),  but you will not lose this file nor other files, as is often the case with other file systems.

OS/2 considerations

LVM

FDISK.COM has been replaced by LVM.EXE and FDISKPM.EXE has been
replaced by LVMGUI.CMD.  Please use one of these utilities.

It should be noted here that LVMGUI is a GUI app (as the name implies) and requires Java, while LVM is a VIO app and can be run from a command line boot. It looks and feels similar to FDISK, but it presents two views: logical and physical. FDISK didn't differentiate between the two. These views corresponds to the concepts described at the beginning of this article. Basically the physical view shows physical disks and lets users manage partitions while logical view presents volumes. One important concept must be introduced here, and that is a compatibility volume.  A compatibility volume corresponds to old FDISK partitions. During WSeB installation, the installer automatically converts all existing partitions to compatibility volumes. This conversion technically means that the installer writes a special block of LVM data to the sector following the partition table. OSes other than WSeB won't see any difference at all. It is however necessary to manage all partitions/volumes exclusively with LVM after this conversion.

All FAT, HPFS, FAT32 etc. partitions can reside on either compatibility or LVM volumes, however other OSes will only be able to access them on compatibility volumes.  JFS on the other hand must be created on LVM volumes. Those were already described above and enjoy all the flexibility of LVM, such as spanning multiple physical disks or online expansion.

Each volume, compatibility or LVM, represents a single drive letter on an OS/2 system. LVM however is significantly more flexible than FDISK because the drive letters are not assigned by a fixed algorithm. Instead, users can assign arbitrary drive letters to volumes. The drive letters can even be changed at runtime, but users have to understand the dangers before doing that. If you reassign the drive letter of the boot volume, it doesn't require a genius to understand that a system crash will be the most likely result.

JFS

1) JFS stores file and directory names in Unicode. This allows JFS to always maintain proper sort order, regardless of active codepage.
2) This is not a permanent limitation. Only no one wrote a JFS micro- and mini-IFS yet.

It might perhaps interest some users that JFS also seems to have built-in support for DASD limits. I have however never tried to use this feature. DASD limits, aka Directory Limits feature of LAN Server allows administrators to control how much space a directory can take, effectively enabling them to limit disk space usage of users. Previously this feature only worked on HPFS386 volumes. Obviously this is of no use to home users who have all their disk space for themselves but it can be very useful for system administrators.

JFS Utilities

• DEFRAGFS - can be used to defragment and reorganize a JFS volume. It is similar in spirit to equivalent FAT or HPFS utilities. It should be noted that just like HPFS, JFS tries not to fragment files. However especially on nearly full volumes, this is not always possible. In addition to defragmenting files, DEFRAGFS will try to rearrange internal JFS structures by placing certain pieces of data physically close to each other to speed up disk access. DEFRAGFS is designed to be run in the background.
• EXTENDFS - after enlarging a LVM volume, this utility must be used to tell the JFS file system that it should take up all the extra space now available.
• CACHEJFS - not documented in Command Reference, this utility can be used to query the settings of the JFS cache and set its lazy writer parameters.
• CHKLGJFS - again undocumented. This is a diagnostic tool and will show a formatted log of the last (or one before last) checkdisk process. Not very useful to normal users.

• LOGDUMP - as the name suggests, this tool dumps formatted contents of the current JFS log (journal) to a file.
• CSTATS - lists current statistics of the JFS cache.
• XPEEK - perhaps the most useful of the bunch, this one is the closest thing to a JFS disk editor I've seen. This utility lets users dump and optionally modify various internal JFS structures. It has a very crude interface but it worked for me. Needless to say, this utility is extremely dangerous and you can easily destroy your data if you don't know exactly what you're doing.

Conclusion

Parting note: Everything said here about WSeB will equally apply to eCS.
 

Glossary of Terms:

• Partition - a portion of physical hard disk space. A hard disk may contain one or more partitions. Partitions are defined by PC BIOS and described by partition tables stored on a harddrive. Every PC OS understands partitions.
• Volume - a logical concept which hides the physical organization of storage space. A compatibility volume directly corresponds to a partition while LVM volume may span more than one partition on one or more physical disks. A volume is seen by users as a single drive letter. Only WSeB and eCS understand LVM volumes.
• DASD - Direct Access Storage Device. A term often used by IBM instead of 'hard disk' to confuse mere mortals.

Physical View of the LVM

The following is a list of the common features in the LVM and VxVM methods of storage management:

• Online backup of file systems
• Snapshot backup of file systems
• Quick resynchronization of changed partitions after online or snapshotbackup
• Migration of data with active volumes
• Transparent data stream switch after mirrored disks fail
• Ability and limits to operate in a multinode concurrent configuration
• Commands to replace dead or failing drives
• Hot spare, standby disks

Both VxVM and LVM enable file systems to be expanded while the file system is mounted and in use. Both volume managers are tied closely to specific file systems. VxVM only works with the Veritas File System, and LVM only works with the JFS or JFS2 (enhanced JFS) file system found on AIX. The one difference between VxVM and LVM is the philosophy on the concept of file system shrinkage. LVM/JFS/JFS2 does not allow the user the ability to shrink the file system. The reasoning behind this is that sometimes portions of files are spread out over the file system for performance reasons. And when a file system needs to be shrunk, the file system must go out and gather up the "pieces" of a file and relocate them to a portion of the file system that will not be eliminated. This method is time-consuming and not very efficient for performance. Thus on AIX, file system reduction is not permitted on JFS and JFS2.

VxVM takes a different approach to this. It does allow the reduction of a file system size, but it is very simplistic and has drawbacks. They literally "chop" off the end of the logical volume and file system up to the point where the user wants the file system reduced. Warnings are included in the VxVM product regarding this practice; if the user has data at the end of the file system, the data may be corrupted or lost. Thus, for the purposes of this paper, the ability of VxVM to reduce file system size is considered limited.

Terminology:

Physical Volume (PV)

Synonym for "hard disk". A single physical hard drive.

Volume Group (VG)

A set of one or more PVs which form a single storage pool. You can define multiple VGs on each AIX system.

Physical Partition (PP)

The smallest allocation unit in the LVM. All PPs within a VG are the same size, usually 4 or 8 MB.

Logical View of the LVM

Terminology:

Volume Group (VG)

A set of one or more PVs which form a single storage pool. You can define multiple VGs on each AIX system.

Logical Partition (LP)

A logical mapping to one or more PPs within the same VG, regardless of whether they're on the same PV or not.

Logical Volume (LV)

A set of one or more LPs within the same VG which form a usable unit of disk space. LVs are used analogously to partitions on PCs or slices under Solaris: they usually contain filesystems or paging spaces ("swap").

Big Picture of the LVM

Advantages:

• Filesystems can span disks, so size is not limited by capacity of physical disks.
• Can expand filesystems on the fly.
• Can add additional disks to existing pool of disk space (VG).
• Can mirror important data on multiple physical disks for redundancy.
• Can "export" an entire VG so that all disks in the VG can be easily physically disconnected, moved to another machine, and "imported".

Limitations:

• Must reducevg when removing disk.
• When a single disk in a VG dies, the whole VG is affected.
• "Brick Wall" between VGs - LVs can't cross VG boundary.
• Cannot shrink filesystems.

• Configuring for performance or flexibility vs. reliability or fault-tolerance.
• Strategy: Define adequate but not huge filesystems, leaving pool of free storage for later assignment.
• Seperate out data categorically onto different VGs, so that an OS upgrade will not touch user data, or so that user data can be easily moved to another machine.

Filesystems

• /etc/filesystems - AIX's version of /etc/fstab. Uses stanza format, * is comment character.
• Useful commands:
  • crfs - Creates logical volume, makes filesystem on logical volume, adds entry to /etc/filesystems. Does not mount the new filesystem for you!
  • chfs - Changes settings of filesystem, logical volume, or /etc/filesystems entry.
  • rmfs - Removes filesystem, logical volume, entry from /etc/filesystems, and optionally the mount point.
• AIX itself is always installed in a VG called rootvg
• The jfs filesystem is a journaling filesystem, meaning that it keeps a summary of filesystem update information on a special LV of type jfslog

Example: Adding a Disk

1. Physically attach the disk to the system. Make sure there are no SCSI ID conflicts.
2. Run cfgmgr to detect new device. Device will be installed as hdiskn, where n is some integer.
3. Run extendvg vgname hdiskn to add the disk to a VG named vgname.
4. To create a new filesystem on this VG, use crfs -v jfs -g vgname -A yes -a size=262144. (Note that the size parameter indicates the number of 512-byte blocks, so 262144 means 128MB.)
5. To add 128MB to an existing filesystem on this VG, use chfs -a size=+262144 /filesystem/name.

Recommended Links

Chapter 5. Device management

5.1 Overview

5.2.2 Managing device drivers

5.2.3 Configuring a device

5.2.4 Adding a new device to a SCSI bus

5.2.5 Remove a SCSI device

5.3 Device management in AIX 5L Version 5.1

5.3.1 Listing devices

5.3.3 Removing a device

5.3.4 Changing a device

5.4 Quick reference

Chapter 6. Logical Volume Manager and disk management

6.1 Logical volume management overview

6.2 Introducing the logical volume solutions

6.2.1 Solaris Solstice DiskSuite: Introduction

6.2.2 VERITAS Volume Manager: Introduction

6.2.3 AIX 5L Version 5.1 LVM: Introduction

6.3 Working with logical volume manager

6.3.1 Volume groups

6.3.2 Working with logical volumes

6.3.3 Working with physical disks

6.3.4 Additional features: Hotspare disks

6.4 Quick reference

Chapter 7. File system management

7.1 Overview

7.1.1 Solaris file systems types and commands

7.1.2 AIX file systems types and commands

7.2 Formatting and partitioning a disk (Solaris only)

7.3 Creating a file system

7.4 Mounting and unmounting a file system

7.5 Checking file system consistency

7.6 Changing file system attributes

7.7 Removing a file system

7.8 Displaying a file system information

7.8.1 Displaying defined file systems

7.8.2 Displaying the file systems mount table

7.8.3 Displaying the available file system space

7.9 Back up and restore file systems

7.10 File system logging

7.11 Compression and defragmentation (AIX only)

7.11.1 Compressed journaled file system

7.11.2 Defragmentation

7.12 Paging space management

7.12.1 Monitoring paging space resources

7.12.2 Adding and activating a paging space

7.12.3 Changing attributes of a paging space (AIX only)

7.12.4 Removing a paging space

7.13 Quick reference

Chapter 8. Backup and restore

Laurent Vanel, Ronald van der Knaap, Dugald Foreman (2000). AIX Logical Volume Manager from A to Z: Introduction and Concepts.

IBM Redbooks | AIX Logical Volume Manager from A to Z ...

AIX Logical Volume Manager, from A to Z: Introduction and Concepts

Rosetta Stones

• Compares: AIX Logical Volume Manager and Veritas Volume Manager (PDF)
• Compares: AIX 5.1 and Solaris 8 (PDF)
• Compares: AIX, FreeBSD, HP-UX, Redhat, Solaris, Tru64 (PDF)
• Compares: Everything and the kitchen sink (PDF)

 Logical Volume Manager (LVM) Commands for AIX

Glossary

Command Summary

Sample LVM Procedures:

Filesystem Procedures

• See below the procedure for creating a logical volume and a filesystem using JFS:

1. "df" to see the filesystem, it's current size, % utilization and the name of it's logical volume
2. "lslv <logical_volume>" to show information about the logical volume including it's volume group name.
3. "lsvg <volume_group>" to show information about the volume group, including number of free pp's and the pp size
4. If there are not enough free pp's then see below for procedure to add a disk to a volume group.
   
5. "chfs -a size= +4194304 <MOUNT_POINT>" to grow the filesystem by 2 GB (4194304=2*1024*1024*1024/512)
   • NOTE:  Growing the file system will automatically grow the logical volume
6. df" shows the file system's current size is 2 GB more than before.

• Error Message:  0516-787 extendlv: Maximum allocation for logical volume <LV_Name> is 512.
  • Maximum number of LPs for the logical volume has been exceeded - must increase the allocation
  • Calculate the number of LPs needed = LV Size in MB / LP size in MB
  • chlv -x <new_max_lps> <logical_volume>

1. Unmount the filesystem
   
2. Remove the logical volume "rmlv <lv_name>"
3. Remove the filesystem information from /etc/filesystems

1. Create the file system
   1. crfs -v jfs -m /usr/sharenew -g rootvg -a size=8192
   2. this makes a logical volume in the root volume group of 4MB that uses jfs
2. Mount the volume
   1. mount /usr/sharenew
3. Move the files from the old file system (/usr/shareold)
   1. cd /usr/shareold
   2. tar cf - | (cd /usr/sharenew; tar xvf -)
   3. cd
4. Unmount the file systems
   1. umount /usr/sharenew
   2. umount /usr/shareold
5. Remove the old file system and it's logical volume
   1. rmfs /usr/shareold
6.   
   1. chfs -m /usr/shareold /usr/sharenew
7. Mount the new filesystem
   1. mount /usr/shareold
8. Delete the temporary mount point
   1. rmdir /usr/share

Logical Volume Procedures

1. lsvg to determine the size of the PP
2. lslv in similar logical volumes to determine if mirroring is in effect
   
3. Calculate the number of PPs needed for the logical volume
   
   1. bc
   2. scale=2
   3. <size of lv in MB>/<size of PP in MB>
   4. quit
      
4. mklv -y  "<LV_NAME>" <VG_NAME> <# of LPS>  --> creates the logical volume
5. crfs -v jfs -d <LV_NAME> -m /<MOUNTPOINT> -A yes   --> makes the filesystem, creates the mountpoint and puts it in /etc/filesystems
6. mount /<MOUNTPOINT>  --> mounts the new fileystem
7. df /<MOUNTPOINT>  --> verifies the mount and the size of the new filesystem
8. Check the ownership and permissions of the new mount point
   • ls -ld <mountpoint>
   • chown owner:group <mountpoint>
   • chmod XXX <mountpoint>
     
9. If mirroring is in effect, then mirror this logical volume to another disk (original and 1 mirror):
   • mklvcopy -s y <LV_NAME> 2
     

• lsvg -l

•  mklvcopy -s y <LV_NAME> 2

Volume Group Procedures

1. lsdev -C -c disk  -> lists available disks (and the hdisk#) on the server
2. mkvg -y "<VG_NAME>" hdisk#  --> creates the volume group on the named hard disk
3. varyonvg <VG_NAME>  --> activates the volume group

• extendvg <vg> <disk#>
  • Verify the disk has been successfully added to the vg
• lsvg -p <vg>

1. lspv  --> determine the hdisk#
   
2. extendvg rootvg hdisk<number>  --> add the hdisk to the volume group
   
3. lspv  -->  verify that the hdisk has been successfully added to the volume group
   
4. chvg -Q 'n' rootvg  -->  change the quorum so that the vg will stay active if one of the mirrors fail
   
5. mirrorvg -S -c 2 rootvg  --> mirror all of the logical volumes in the volume group
   
6. lsvg -l rootvg  --> verify successful mirroring (pps will appear "stale" until synchronization is complete).
   
7. bosboot -a  -->  update the boot image information
   
8. bootlist -m normal -o hdisk0 hdisk1  --> create a new bootlist
   
9. bootlist -m normal -o  --> verify the bootlist is correct
   

1. "lsvg <volume_group>" to show the total PP's available in the volume group =1250
2. "lsvg -l <volume_group>" to show the total PP's used in all logical volumes in that volume group (showed sys1log, the jfs log was using 2 PP's)
3. "chlv -x 1248 <logical_volume>" to change the maximum number of LP's from 1100 to 1248 (1250 PP's in the volume group - 2 PP's used by the jfs log  = 1248 available)

Physical Disk Procedures

• lsvpcfg
  • This will show disks/vpaths and the volume group they are allocated to
• lspv|grep None
  • This will show pvs and whether they are asssociated with a volume group
  • Note:  For vpaths, the hdisks will show as none, but they may be allocated to a vpath - you must grep each hdisk with the lsvpcfg

• Allocate the new lun on the SAN
• Run "cfgmgr"
• Verify the new vpatch/hdisk by running "lsvpcfg"
  • There should be a new vpath and it should be available with no volume group - if not, rerun cfgmgr
    

• lsvg -p <volume group>