IBM AIX - Mirror Inconsistency Issue with a Logical Volume
Update on mazor Issue Scenario:
There had been a mirror inconsistency issue with a logical volume.
The issue was that there was a logical volume that had mirror copies, but one LP had 3 copies as against 2.
Problem Identification:
[servotsm01:root:/home/sysadmin:] lsvg -l Serverstgpvg
Serverstgpvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
0516-1147 : Warning - logical volume Serverstgplv may be partially mirrored.
Serverstgplv jfs2 4017 8035 30 open/syncd /tsm/yar1/stg
loglv01 jfs2log 1 2 2 open/syncd N/A
The logical Volume Serverstgplv has 4017 LPs mapped to 8035 PP (which ideally should have been 8034 [4017*2]).
The LV has 1 extra PP to it which implies, 1 LP has an additional copy (the 3rd copy) hence the giving the warning "logical volume Serverstgplv may be partially mirrored".
Problem Diagnosis / approach:
Step 1.
Identify which LP has the 3rd copy.
[servotsm01:root:/home/sysadmin:] lslv -m Serverstgplv
Serverstgplv:/tsm/yar1/stg
LP PP1 PV1 PP2 PV2 PP3 PV3
0001 0801 vpath62 0059 vpath41 0814 vpath63
0002 0060 vpath41 0060 vpath7
0003 0061 vpath41 0061 vpath7
0004 0062 vpath41 0062 vpath7
0005 0063 vpath41 0063 vpath7
0006 0064 vpath41 0064 vpath7
0007 0065 vpath41 0065 vpath7
0008 0066 vpath41 0066 vpath7
0009 0067 vpath41 0067 vpath7
.......... ................................ ..............................
......... ............................... ..............................
The first logical partition itself has 2 mirror copies. The master copy is at the 801st location of vpath62 ,
2nd mirror copy is at the 59th location of vpath41 and third mirror copy at the 814th location of vpath63.
Now the requirement is to remove any one of these copies to make it consistent again. we would not want to touch the primary copy on vpath62,
so anyone of vpath41 or vpath63 will do it for us. I choose vpath41 because for some future activities I need a physical partition free from this vpath41.
Step 2.
Run the following command to get the exact ODM entries of location of the physical copies of a logical partition.
lquerylv -L `getlvodm -l Serverstgplv` -r > /tmp/mapfile
sample o/p of the command lquerylv -L `getlvodm -l Serverstgplv` -r..
[servotsm01:root:/home/sysadmin:] lquerylv -L `getlvodm -l Serverstgplv` -r
000163bfb78bc82d 801 1
000163bfb78b9af5 814 1
0001647501aab194 60 2
0001647502f032c1 60 2
0001647501aab194 61 3
0001647502f032c1 61 3
0001647501aab194 62 4
0001647502f032c1 62 4
.......... ......... .........................................
..............................................................
Check it stores the info on the following format ....
PVID PP-copy LP no.
location
and I took the o/p of the command in /tmp/mapfile to have backup for the same.
Check the first three lines of the /tmp/mapfile...
[servotsm01:root:/home/sysadmin:] cat /tmp/mapfile1 | head
000163bfb78bc82d 801 1
0001647501aab194 59 1
000163bfb78b9af5 814 1
0001647501aab194 60 2
0001647502f032c1 60 2
0001647501aab194 61 3
0001647502f032c1 61 3
0001647501aab194 62 4
0001647502f032c1 62 4
0001647501aab194 63 5
LP1 is having three pp copies on the 801st location of the pv 000163bfb78bc82d(vpath61), 59th location of the pv 0001647501aab194(vpath41) and on the 814th location of the pv 000163bfb78b9af5(vpath63)
we have to delete 0001647501aab194 59 1 from the ODM as I don't want to keep a copy at vpath41. See the next step.
Step 3:
Now edit your mapfile and delete all good entries and leave only the entries that you don't require , so the map file will look like this..
[servotsm01:root:/home/sysadmin:] cat /tmp/mapfile
0001647501aab194 59
Now we have to delete the entry from the odm. Check the below step..
Step 4:
Use the following command to delete it from odm and read the data from /tmp/mapfile.
lreducelv -l `getlvodm -l Serverstgplv` -s 1 /tmp/mapfile
This command will delete the physical partition copy from ODM permanently. Now check the below o/p ..
[servotsm01:root:/home/sysadmin:] lslv -m Serverstgplv | head
Serverstgplv:/tsm/yar1/stg
LP PP1 PV1 PP2 PV2 PP3 PV3
0001 0801 vpath62 0814 vpath63
0002 0060 vpath41 0060 vpath7
0003 0061 vpath41 0061 vpath7
0004 0062 vpath41 0062 vpath7
0005 0063 vpath41 0063 vpath7
0006 0064 vpath41 0064 vpath7
0007 0065 vpath41 0065 vpath7
0008 0066 vpath41 0066 vpath7
Now, no third copy exists for logical partition 1.
Verification:
Check the below o/p to get confirmation for the same..
[servotsm01:root:/home/sysadmin:] lsvg -l Serverstgpvg
Serverstgpvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
Serverstgplv jfs2 4017 8034 30 open/syncd /tsm/yar1/stg
loglv01 jfs2log 1 2 2 open/syncd N/A
Assumptions and Warnings.
Don't use these lower level commands in AIX if you don't have confidence and knowledge about the same.
If you don't know consult the experts or try to understand and learn their exact activity first. Double check and be sure everything is all right from all ends before running those, as a moment of carelessness can cause a real disaster in the production environment.
There had been a mirror inconsistency issue with a logical volume.
The issue was that there was a logical volume that had mirror copies, but one LP had 3 copies as against 2.
Problem Identification:
[servotsm01:root:/home/sysadmin:] lsvg -l Serverstgpvg
Serverstgpvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
0516-1147 : Warning - logical volume Serverstgplv may be partially mirrored.
Serverstgplv jfs2 4017 8035 30 open/syncd /tsm/yar1/stg
loglv01 jfs2log 1 2 2 open/syncd N/A
The logical Volume Serverstgplv has 4017 LPs mapped to 8035 PP (which ideally should have been 8034 [4017*2]).
The LV has 1 extra PP to it which implies, 1 LP has an additional copy (the 3rd copy) hence the giving the warning "logical volume Serverstgplv may be partially mirrored".
Problem Diagnosis / approach:
Step 1.
Identify which LP has the 3rd copy.
[servotsm01:root:/home/sysadmin:] lslv -m Serverstgplv
Serverstgplv:/tsm/yar1/stg
LP PP1 PV1 PP2 PV2 PP3 PV3
0001 0801 vpath62 0059 vpath41 0814 vpath63
0002 0060 vpath41 0060 vpath7
0003 0061 vpath41 0061 vpath7
0004 0062 vpath41 0062 vpath7
0005 0063 vpath41 0063 vpath7
0006 0064 vpath41 0064 vpath7
0007 0065 vpath41 0065 vpath7
0008 0066 vpath41 0066 vpath7
0009 0067 vpath41 0067 vpath7
.......... ................................ ..............................
......... ............................... ..............................
The first logical partition itself has 2 mirror copies. The master copy is at the 801st location of vpath62 ,
2nd mirror copy is at the 59th location of vpath41 and third mirror copy at the 814th location of vpath63.
Now the requirement is to remove any one of these copies to make it consistent again. we would not want to touch the primary copy on vpath62,
so anyone of vpath41 or vpath63 will do it for us. I choose vpath41 because for some future activities I need a physical partition free from this vpath41.
Step 2.
Run the following command to get the exact ODM entries of location of the physical copies of a logical partition.
lquerylv -L `getlvodm -l Serverstgplv` -r > /tmp/mapfile
sample o/p of the command lquerylv -L `getlvodm -l Serverstgplv` -r..
[servotsm01:root:/home/sysadmin:] lquerylv -L `getlvodm -l Serverstgplv` -r
000163bfb78bc82d 801 1
000163bfb78b9af5 814 1
0001647501aab194 60 2
0001647502f032c1 60 2
0001647501aab194 61 3
0001647502f032c1 61 3
0001647501aab194 62 4
0001647502f032c1 62 4
.......... ......... .........................................
..............................................................
Check it stores the info on the following format ....
PVID PP-copy LP no.
location
and I took the o/p of the command in /tmp/mapfile to have backup for the same.
Check the first three lines of the /tmp/mapfile...
[servotsm01:root:/home/sysadmin:] cat /tmp/mapfile1 | head
000163bfb78bc82d 801 1
0001647501aab194 59 1
000163bfb78b9af5 814 1
0001647501aab194 60 2
0001647502f032c1 60 2
0001647501aab194 61 3
0001647502f032c1 61 3
0001647501aab194 62 4
0001647502f032c1 62 4
0001647501aab194 63 5
LP1 is having three pp copies on the 801st location of the pv 000163bfb78bc82d(vpath61), 59th location of the pv 0001647501aab194(vpath41) and on the 814th location of the pv 000163bfb78b9af5(vpath63)
we have to delete 0001647501aab194 59 1 from the ODM as I don't want to keep a copy at vpath41. See the next step.
Step 3:
Now edit your mapfile and delete all good entries and leave only the entries that you don't require , so the map file will look like this..
[servotsm01:root:/home/sysadmin:] cat /tmp/mapfile
0001647501aab194 59
Now we have to delete the entry from the odm. Check the below step..
Step 4:
Use the following command to delete it from odm and read the data from /tmp/mapfile.
lreducelv -l `getlvodm -l Serverstgplv` -s 1 /tmp/mapfile
This command will delete the physical partition copy from ODM permanently. Now check the below o/p ..
[servotsm01:root:/home/sysadmin:] lslv -m Serverstgplv | head
Serverstgplv:/tsm/yar1/stg
LP PP1 PV1 PP2 PV2 PP3 PV3
0001 0801 vpath62 0814 vpath63
0002 0060 vpath41 0060 vpath7
0003 0061 vpath41 0061 vpath7
0004 0062 vpath41 0062 vpath7
0005 0063 vpath41 0063 vpath7
0006 0064 vpath41 0064 vpath7
0007 0065 vpath41 0065 vpath7
0008 0066 vpath41 0066 vpath7
Now, no third copy exists for logical partition 1.
Verification:
Check the below o/p to get confirmation for the same..
[servotsm01:root:/home/sysadmin:] lsvg -l Serverstgpvg
Serverstgpvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
Serverstgplv jfs2 4017 8034 30 open/syncd /tsm/yar1/stg
loglv01 jfs2log 1 2 2 open/syncd N/A
Assumptions and Warnings.
Don't use these lower level commands in AIX if you don't have confidence and knowledge about the same.
If you don't know consult the experts or try to understand and learn their exact activity first. Double check and be sure everything is all right from all ends before running those, as a moment of carelessness can cause a real disaster in the production environment.
Comments