Logical Volume Manager (LVM) using HP-UX

hp-ux民工

Glossary
Term
Definition

Journaled File System (JFS)
Unix filesystem that uses a journaled log to record changes to the filesystem.  In the event of a failure the log can be used to recover.

Logical Extents
Logical extents are used to create the logical volume - they are the same size as physical extents

Logical Interchange Format (LIF)
HP Storage format used for interchange of files between HP systems.

Logical Volumes
Allocates the space inside a volume group, the logical volume gets the file system

Mirroring
Mapping a logical extent to more than one physical extent.  Logical volumes are mirrored, not disks.

Physical Extent (PE)
Unit used for allocating disk space to logical volumes, default size is 4MB.  It is a set of contiguous disk blocks.

Physical Partitions
The physical volume is organized into blocks called physical partitions (same as physical extent??)

Physical Volume
Physical disk that is being managed by LVM

Striping
A logical volume that is configured to write across more than one disk in a stripe.

Volume
A device used for storage - either an entire disk or a partition.

Volume Group
Contain one or more physical volumes, a physical volume can only be in one volume group (pool of partitions)



Commands
Command Function
lvcreate
Command used to create logical volumes
lvdisplay -v /dev/<VG>/<LV>  Command used to display information about logical volumes
pvchange
Changes the attributes of existing physical volumes.

pvcreate /dev/rdsk/cxtxdx
pvcreate -f /dev/rdsk/cxtxdx
Creates a physical volume by creating the LVM data structures on the physical disk called physical extents.  The -f option will force the creation of the PV - potentially destroying old PV information.

pvdisplay /dev/dsk/cxtxdx
pvdisplay -v /dev/dsk/cxtxdx
Command used to display information about physical volumes
vgchange
Command used to change information about the volume group, such as to activate an inactive vg

vgcreate
Command used to create the volume groups
vgdisplay [-v] [VG_NAME}
Command used to display information about volume groups, -v is the verbose option which will list lvs and pvs.

vgexport


vgimport





Files
File Description

/dev/dsk/cxtxdx Block device for disk

/dev/rdsk/cxtxdx Character device for disk

/dev/<volumegroup>/group
Character device file (created with the mknod command) that allows the lvm kernel and the lvm commands to communicate
/etc/lvmrc
Script that starts each volume group based upon the contents of /etc/lvmtab
/etc/lvmtab This file has the device file associated with each disk in a volume group.  Access it with the strings command.

/etc/vpath.cfg If IBM SDD is being used this file contains the mapping of vpath to the corresponding disk-target-lun.  Access it with the strings command.



Procedures
Filesystem Procedures:
Create a new filesystem:
(A new filesystem will require a new logical volume)

Create the Logical Volume (Procedure Below)

Create the File System
Using newfs:

newfs -F <filesystem_type> <path_to_lvol_devicefile>
newfs -F vxfs /dev/vg02/lvol1
Using mkfs:
mkfs -F <FILESYSTEM TYPE> -o bsize=<OPTIONS>,<OPTIONS> <PATH TO LV>
-F is filesystem type, such as vxfs for the Veritas File System or JFS
-o are options, such as bsize (Block Size) and largefiles.
Path to LV is of the form /dev/VGNAME/LVNAME
For example:  "mkfs -F vxfs -o bsize=8192,largefiles /dev/bgbill/lvbill"
Create the Mount Point
mkdir -p <full path to mountpoint>
-p will make any subdirectories in the path that do not already exist
Set the proper ownership and permissions on the mount point
chown owner:group /<mount point>
chmod XXX /<mount point>

Mount the File System
Edit /etc/fstab using vi and add the proper entry
<LV PATH> <MOUNTPOINT PATH> <FILESYSTEM> <OPTIONS> <BACKUP FREQUENCY> <PASS NUMBER>
For example:  "/dev/vgbill/lvbill  /maindir/subdir/share vxfs rw,suid,largefiles,delaylog,datainlog 0 2"
"mount /<mountpoint>" to mount the file system

Test
Do a "bdf <mount point>".    This command will show if the file system is mounted, and it's size.
Do an "ls -ld <mount point>" to verify that the proper permissions are there.

Have the user test the filesystem, and report back with any problems
Extending a VxFS Filesystem (Online JFS)

Verify that there is enough space in the volume group
Check the volume group for enough space

vgdisplay <VG Name>
Check the logical volume to determine if mirroring or striping is in effect
lvdisplay /dev/vg_name/lv_name

If there is not enough space then the volume group must be extended - see procedure below.

First lvextend the logical volume - 2 Options for lvextend:
lvextend -L <New Size in MB> /dev/VG_NAME/LV_NAME

lvextend -l <NEW Size in # of LEs> /dev/VG_NAME/LV_NAME
fsadm -F vxfs  -b <new size of filesystem in K> <MOUNT POINT>
NOTE:  extendfs will NOT work with Online JFS, you must use fsadm
NOTE:  new size of the filesystem = <# of LEs> * <LE Size in MB> * 1024
NOTE:  If lvextend -L was used simply multipy size in MB by 1024

NOTE:  If this FAILS with errno 28, then the filesystem is 100% full and must be reduced to less than 100%.
NOTE:  If this FAILS with "write failure at block XXXXXXXX : No such device or address" then the new size may be too big, use a smaller number

Extending a VxFS Filesystem (No Online JFS)

Verify that there is enough space in the volume group
vgdisplay <VG Name>
If there is not enough space then the volume group must be extended - see procedure below.
First lvextend the logical volume - 2 Options for lvextend:
lvextend -L <New Size in MB> /dev/VG_NAME/LV_NAME

lvextend -l <NEW Size in # of LEs> /dev/VG_NAME/LV_NAME
Unmount the filesystem
extendfs -F vxfs /dev/<VG_Name>/r<LV_Name>
Note:  extendfs uses the raw logical volume name (ie rlvol1)



How to determine if you have Online JFS

swlist -l fileset | grep -i advanced
How to determine the filesystem type

grep <mount point> /etc/fstab
Results should show the filesystem type (vxfs or hfs) in the return string
Logical Volume Procedures:
Creating a Logical Volume:

Determine if an existing volume group has enough spare capacity for a new logical volume.
Is mirroring or striping required (if not known, then see if other logical volumes in that volume group use it)?

vgdisplay   <This will list existing volume groups
Spare capacity is indicated by the number of free PEs
Multiply the "Free PE" by "PE Size (MBytes)" to get spare capacity in MB of the volume group
Note:  If mirroring or striping is required then there must be enough space available on multiple disks

Is there a volume group with enough spare capacity?
If yes, the continue to "Create the Logical Volume"
If no, then the Volume Group must be extended or a new Volume Group must be created - see procedures below.

Create the logical volume:

lvcreate -1 <size in LEs> -n <LV Name> -r N <VG Directory path> OR

-l is size in Logical Extents (same as PE size)
-n is the name of the Logical Volume
-r N disable bad block allocation
lvcreate -L <size in MBs> -n <LV Name> -r N <VG Directory path>
lvcreate -L 100 /dev/vg01
Logical volume will be 100MB in size
Name of the logical volume default name will be lvolx
lvcreate -L 100 -n oracle /dev/vg01
This creates a logical volume called oracle of 100 MB
Verify the lv was created properly "vgdisplay -v <volume group>"

Removing a Logical Volume

Unmount the filesystem

umount /<Path to Mount Point>

Remove the volume group
lvremove /dev/<VG_NAME>/<LV_NAME>
Clean up
Delete the mount point
Remove the mount from /etc/fstab


Troubleshooting (just an idea, not sure if this really works??)
ONCTRA01# lvchange -a n /dev/vg_wte_test/lv_wte_test
Logical volume "/dev/vg_wte_test/lv_wte_test" has been successfully changed.
Volume Group configuration for /dev/vg_wte_test has been saved in /etc/lvmconf/vg_wte_test.conf

Unmirroring a Logical Volume

lvreduce -m 0 /dev/<VG_NAME>/<LV_NAME>


Volume Group Procedures:
Create a Volume Group
Determine the list of disk devices that will make up the volume group (procedure below)
Create PV disks for each of the disks (procedure below)

Create the volume group directory
mkdir /dev/vg01
Create the group file
Determine the minor number to use
ls -l /dev/*/group | sort +5
Choose the next unused number

mknod /dev/vol_group_directory/group c Major_number Minor_number
mknod /dev/vg01/group c 64 0x010000
The group file is used to allow communications between the lvm commands and the lvm kernel

Create the volume group

vgcreate -l 255 -p 64 -s 16 /dev/<vg id>/dev/dsk/<disk id>  <-- create the volume group with initial disk devices
-l sets the maximum number of logical volumes for the volume group
-p sets the maximum number of physical volumes for the volume group
-s sets the physical extent size

/dev/<vg id> is the volume group directory
/dev/dsk/diskid are the physical volumes to be added to the volume group
vgcreate /dev/vg01 /dev/dsk/c1t0d0 /dev/dsk/c1t1d0
vgextend /dev/vgxx /dev/dsk/diskid  <-- adds additional disk devices to the volume group
Extending a Volume Group

A volume group is extended by adding additional PVs to it with vgextend
vgextend /dev/vgxx /dev/dsk/diskid
Note:  If there are no available PVs, then disk space (LUNS) must be added to the system and PVs must be created - see procedures below.

Activating a Volume Group

vgchange -a y /dev/<vg name>
mount -a  ==>  This may indicate that a filesystem check needs to be run (fsck)
fsck -m /dev/<vg name>/<lv name>  ==> this is a sanity check, it is very quick, but may indicate that a complete fsck needs to be run
fsck /dev/<vg name>/<lv name>  ==> this is the full file system check
Deactivating a Volume Group

umount all of the logical volumes in the volume group
Determine all of the mounted logical volumes by "bdf | grep vg_name"

vgchange -a n /dev/<vg name>

Adding mirroring to the root vg - vg00

pvcreate -B /dev/rdisk/c2t2d0
Create the PV on the second disk so that LVM can manage it
-B makes it a bootable volume

mkboot /dev/rdsk/c2t2d0
Installs the boot files on the second disk

mkboot -a "hpux-lq (;0)/stand/vmunix" /dev/rdsk/c2t2d0
Creates an autoboot file on the disk, with information on how to boot

vgextend /dev/vg00 /dev/dsk/c2t2d0
extends the volume group to include this second disk.

vgdisplay -v vg00
To verify that the second disk is now part of the volume group
lvlnboot -v
To verify that the system thinks this disk can boot
lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c2t2d0
This extends the logical volume to the mirrored disk, effectively creating the mirrored copy.
This command needs to be run for each logical volume in the volume group


Removing a Volume Group

To remove a volume group using vgreduce and vgremove:

Remove all Logical Volumes (see procedure above)

remove all disks except on using vgreduce <VG_NAME> </dev/dsk/DISK>
This will need to be done for each disk in the volume group, except for the last disk

remove the final disk in the volume group with vgremove <VG_NAME>
To remove a volume group using vgexport:

Deactive the volume group

vgexport -m <mapfile> -v -f <devicefile> <VG_NAME>
PV Procedures:
How to recognize a newly added LUN:

Check for the new hardware

ioscan -fnC disk | more  <-- the new devices will have a hardware path, but no device file associated with it.

Create the device file for the hardware path
insf
If using vpaths then create the vpath association
/opt/IBMdpo/bin/cfgvpath
/opt/IBMdpo may not be the path of the sdd software, "whereis cfgvpath" may need to be run to find it if it is not in path.

Verify the new devices/vpaths are there:
ioscan -fnC disk
strings /etc/vpath.cfg
/opt/IBMdpo/bin/showvpath
Create the PV
For each disk device (not vpath) issue a "pvcreate /dev/rdsk/cxtxdx"
Note for vpaths this information can be found in /etc/vpath.cfg

How to create Physical Volumes:

3 Methods of Using pvcreate:
pvcreate /dev/rdsk/cxtxdx
Note this command uses the "r" version of the device
pvcreate -f /dev/rdsk/cxtxdx
Note:  the -f option will overwrite existing pv data on the disk device
Note this command uses the "r" version of the device
pvcreate -B /dev/rdsk/cxtxdx
Makes the PV boot capable
Note this command uses the "r" version of the device

How to determine available disks to be used in a Volume Group:

"ioscan -funC disk"  will list all of the disk devices
Some of these devices will be allocated, some will not
"vgdisplay -v" will list all of the PVs and their devices for all of the existing volume groups
This is a list of the devices that are in use
Any devices that are on the ioscan, but are NOT on the vgdisplay are available for use
Possible strategy to automate this process using sed, awk and grep
create a file that has all of the disks that can be used (in this example HITACHI)

cat wte_disk_ioscan | sed 1,2d | grep -v -e TOSHIBA -e c2t0d0 | xargs -n 10 | grep HITACHI | grep -vi subsystem > wte_hitachi_disk
cat will output the file that contains the disk ioscan (ioscan -fnC disk)  
sed is used to delete the first 2 header lines of the file.
Next grep is used to print any lines that DO NOT include TOSHIBA or c2t0d0
xargs is used to group the output into groups of 10
Next grep finds all of the lines with HITACHI in them
All of the lines that have HITACHI in them are saved to a file
Refine the ioscan of HITACHI disks to include just the disk devices, sorted - this is a list of all HITACHI disks on the system

awk '{ print $9 }' wte_hitachi_disk | sort -u > wte_hitachi_sorted_u
awk prints just the 9th field of the file
sort - u sorts the file and surpresses any duplicates
This is saved to a sorted file
Print a list of all the disks that are currently being used (a list of PVs)

vgdisplay -v | grep  "PV Name" > wte_pvdisk_used
vgdisplay -v prints a verbose listing of all volume groups
grep only prints lines that contain PV Name
The list of PVs is saved to a file
Refine the list of disks that are being used

awk '{ print $3 }' wte_pvdisk_used | sort -u > wte_pvdisk_sorted_u
awk prints on the 3rd field (the disk device)
sort will sort the list, surpressing any duplicate entries
the results are saved to a file
Compare the 2 files - the list of all Hitachi disks on the system with the list of all disks being used

diff wte_hitachi_sorted_u wte_pvdisk_sorted_u
diff compares the 2 files and prints out any differences.  The difference will be a disk that the system sees, but that is not being used by LVM
How to remove PVs

Identify the hardware path of the disk to remove
ioscan -fnC disk
Remove the special device file
rmsf -H <HW Path from ioscan>

How to perform an non-destructive test of a disk:
dd if=/dev/rdsk/cxxxxxx of=/dev/null bs=1024k

gesture

也给俺贴点VxVM的东东啊，不要高了，‘民工’级的就成啊，厚厚~ 谢谢