On the road to VCAP5-DCA (Aug 2015) , I thought I will blog stuff as I go along. Here’s Objective 1.1 Understand and apply LUN masking using PSA-related commands. LUN masking gives you control over which hosts see which LUNs. Typically this is done on the SAN-end but in cases you would want to make LUNs available to all the hosts and perform the masking on ESXi end.
My lab (storage) is configured using active:active iSCSi virtual adapters (MPIO), meaning two vNICs on each hosts are binded to vmkernel ports exclusively for storage traffic.
Step1. – Identify LUN to be masked
In order to mask the LUN, ssh into the hosts on which the LUN will be masked and list LUNs (which contain VMFS datastores)
1 2 3 4 5 6 |
~ # esxcfg-scsidevs -m naa.60003ff44dc75adca1f1c655bd658363:1 /vmfs/devices/disks/naa.60003ff44dc75adca1f1c655bd658363:1 543217c6-8560f93c-4cbe-000c2956461b 0 DatastoreA mpx.vmhba1:C0:T1:L0:1 /vmfs/devices/disks/mpx.vmhba1:C0:T1:L0:1 556edb65-2ba42e84-6963-000c2956461b 0 locala naa.60003ff44dc75adc918e8c05f005c032:1 /vmfs/devices/disks/naa.60003ff44dc75adc918e8c05f005c032:1 5459f2d5-b4bedaf6-4def-000c2956461b 0 DatastoreC naa.60003ff44dc75adcaffb68e97f5457cd:1 /vmfs/devices/disks/naa.60003ff44dc75adcaffb68e97f5457cd:1 5432356c-0b33aed4-6ff0-000c2956461b 0 DatastoreB naa.60003ff44dc75adcaf965c0ff08dc0d1:1 /vmfs/devices/disks/naa.60003ff44dc75adcaf965c0ff08dc0d1:1 554fcb0a-168f6e2f-15bb-000c29ab807a 0 Ndatastore |
In my case I have created new datastore Ndatastore for this exercise, therefore the device id I’m after is: naa.60003ff44dc75adcaf965c0ff08dc0d1
Step2. Check the path(s) to this LUN
As we have iSCSi MPIO I expect to see 2 paths.
1 2 3 |
~ # esxcfg-mpath -L | grep naa.60003ff44dc75adcaf965c0ff08dc0d1 vmhba33:C1:T0:L3 state:active naa.60003ff44dc75adcaf965c0ff08dc0d1 vmhba33 1 0 3 NMP active san iqn.1998-01.com.vmware:vhosta-7c3fb590 00023d000002,iqn.1991-05.com.microsoft:win-sc4kipek4vv-datastore-target,t,1 vmhba33:C0:T0:L3 state:active naa.60003ff44dc75adcaf965c0ff08dc0d1 vmhba33 0 0 3 NMP active san iqn.1998-01.com.vmware:vhosta-7c3fb590 00023d000001,iqn.1991-05.com.microsoft:win-sc4kipek4vv-datastore-target,t,1 |
Highlighted paths are in bold.
Step3. Check if the claim rules are already defined if not define it.
1 2 3 4 5 6 7 8 9 10 11 |
~ # esxcli storage core claimrule list Rule Class Rule Class Type Plugin Matches ---------- ----- ------- --------- --------- --------------------------------- MP 0 runtime transport NMP transport=usb MP 1 runtime transport NMP transport=sata MP 2 runtime transport NMP transport=ide MP 3 runtime transport NMP transport=block MP 4 runtime transport NMP transport=unknown MP 101 runtime vendor MASK_PATH vendor=DELL model=Universal Xport MP 101 file vendor MASK_PATH vendor=DELL model=Universal Xport MP 65535 runtime vendor NMP vendor=* model=* |
As we can see no rules appear to have vmhba33:C1:T0:L3 or C0:T0:L3
Let’s define the rules:
1 |
~ # esxcli storage core claimrule add -r 500 -t location -A vmhba33 -C 0 -T 0 -L 3 -P MASK_PATH |
(In this case I recommend adding one path at the time, I had issues adding rules for both paths, seeing error:(Unable to unclaim path vmhba33:C1:T0:L3 on device naa.60003ff44dc75adcaf965c0ff08dc0d1. Some paths may be left in an unclaimed state. You will need to claim them manually using the appropriate commands or wait for periodic path claiming to reclaim them automatically.) Adding the rule for each path at the time fixes the issue.
Step4. Load the rule and unclaim all the paths, try to reclaim all
Load the rules:
1 |
~ # esxcli storage core claimrule load |
Unclaim and reclaim the paths based on the rules:
1 2 |
~ # esxcli storage core claiming reclaim -d naa.60003ff44dc75adcaf965c0ff08dc0d1 Unable to unclaim path vmhba33:C1:T0:L3 on device naa.60003ff44dc75adcaf965c0ff08dc0d1. Some paths may be left in an unclaimed state. You will need to claim them manually using the appropriate commands or wait for periodic path claiming to reclaim them automatically. |
(This error is obvious and correct – No panic 😀 as there are 2 paths and one is still not included in the rules)
Check the paths to LUN:
1 2 |
~ # esxcfg-mpath -L | grep naa.60003ff44dc75adcaf965c0ff08dc0d1 vmhba33:C1:T0:L3 state:active naa.60003ff44dc75adcaf965c0ff08dc0d1 vmhba33 1 0 3 NMP active san iqn.1998-01.com.vmware:vhosta-7c3fb590 00023d000002,iqn.1991-05.com.microsoft:win-sc4kipek4vv-datastore-target,t,1 |
Great result, we can no longer see path: vmhba33:C0:T0:L3
Step5. Repeat step 3 and 4 for the second path:
1 2 3 |
~ # esxcli storage core claimrule add -r 501 -t location -A vmhba33 -C 1 -T 0 -L 3 -P MASK_PATH ~ # esxcli storage core claimrule load ~ # esxcli storage core claiming reclaim -d naa.60003ff44dc75adcaf965c0ff08dc0d1 |
(No output means good!)
Step5. Verify rules list, verify LUN paths, check for the datastore in vSphere client
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
~ # esxcli storage core claimrule list Rule Class Rule Class Type Plugin Matches ---------- ----- ------- --------- --------- ---------------------------------------- MP 0 runtime transport NMP transport=usb MP 1 runtime transport NMP transport=sata MP 2 runtime transport NMP transport=ide MP 3 runtime transport NMP transport=block MP 4 runtime transport NMP transport=unknown MP 101 runtime vendor MASK_PATH vendor=DELL model=Universal Xport MP 101 file vendor MASK_PATH vendor=DELL model=Universal Xport MP 500 runtime location MASK_PATH adapter=vmhba33 channel=0 target=0 lun=3 MP 500 file location MASK_PATH adapter=vmhba33 channel=0 target=0 lun=3 MP 501 runtime location MASK_PATH adapter=vmhba33 channel=1 target=0 lun=3 MP 501 file location MASK_PATH adapter=vmhba33 channel=1 target=0 lun=3 MP 65535 runtime vendor NMP vendor=* model=* |
Check the paths to LUN:
1 2 |
~ # esxcfg-mpath -L | grep naa.60003ff44dc75adcaf965c0ff08dc0d1 ~ # |
Lun masking successfully completed.
To remove it please follow:
1 2 3 4 5 6 |
esxcli storage core claimrule remove -r 500 esxcli storage core claimrule remove -r 501 esxcli storage core claimrule load esxcli storage core claiming unclaim -t location -A vmhba33 -C 1 -T 0 -L 3 esxcli storage core claiming unclaim -t location -A vmhba33 -C 0 -T 0 -L 3 esxcli storage core adapter rescan -A vmhba35 |
References:
http://www.valcolabs.com/2012/05/12/objective-1-1-implement-and-manage-complex-storage-solutions/
https://blogs.vmware.com/vsphere/2011/08/vsphere-50-storage-features-part-12-iscsi-multipathing-enhancements.html
Hope that helps!