iSCSIADM: Cannot see iSCSI luns on server side

On the storage side , the LUNs are created. However you cannot see them on the server side. This post explains the steps needed to debug and fix the issue.



Issue list command to see if you can see the LUNs on the server side:

• iscsiadm list target -vS

If there are no LUNs listed, verify which discovery method is enabled

 

root@dat01:/# iscsiadm list discovery

Discovery:

        Static: disabled

        Send Targets: enabled

        iSNS: disabled

 

Verify which address is enabled in send targets discovery

 

 iscsiadm list discovery-address

Discovery Address: 10.129.195.209:3260

 

 

If the discovery-address is correct and you  you cannot still see the luns, do the following:

 

Check /var/adm/messages for any error message

Restart the iscsi service on storage side

Readd the address to send targets discovery as under:

iscsiadm remove discovery-address  10.129.195.209

root@dat01:/var/adm# iscsiadm list discovery-address

root@dat01:/var/adm# iscsiadm add discovery-address 10.129.195.209

root@dat01:/var/adm# iscsiadm list discovery-address

Discovery Address: 10.129.195.209:3260

FC: Which storage/controller is HBA connected to

• Get info about the HBA

The output below indicates that I have 2 HBAs and both of them are online.

root@dat01:~# fcinfo hba-port  | ggrep "HBA Port" root@dat01:~# fcinfo hba-port  | ggrep "State"

HBA Port WWN: 21000024ff30b222

HBA Port WWN: 21000024ff30b223

• What remote device is the HBA connected to. Does it have SCSI target.
  
  
  The output below indicates that both the HBAs have SCSI targets.

root@dat01:~# fcinfo remote-port -p  21000024ff30b222 Remote Port WWN: 20360080e5249ad6
        Active FC4 Types:
        SCSI Target: yes
        Port Symbolic Name:
        Node WWN: 20060080e5249ad6
root@dat01:~# fcinfo remote-port -p 21000024ff30b223
Remote Port WWN: 20370080e5249ad6
        Active FC4 Types:
        SCSI Target: yes
        Port Symbolic Name:
        Node WWN: 20060080e5249ad6


The first HBA is connected to remote device 20360080e5249ad6 and the second HBA is connected to remote device 20370080e5249ad6

Both the remote devices have the same node WWN (20060080e5249ad6) which indicates that the remote device is probably a dual controller device.

•  Get detailed info about the SCSI targets (Optional)

Detailed info about the SCSI targets can be obtained by following 2 commands:

• fcinfo remote-port -sl -p 21000024ff30b223
•  fcinfo lu -v 

• Get WWNs from the storage

Storage has following WWN:

Node WWN: 20:06:00:80:E5:24:9A:D6

Array WWN: 60:08:0E:50:00:24:A7:7C:00:00:00:00:4E:CC:74:CF

It has two controllers:
Both the controllers have 4 ports out of which only 2 ports are enabled:

Controller A: port 1:

Port WWN:	20:36:00:80:E5:24:9A:D6
Node WWN:	20:06:00:80:E5:24:9A:D6

Controller A: port 2

Port WWN:	20:46:00:80:E5:24:9A:D6
Node WWN:	20:06:00:80:E5:24:9A:D6

Controller B: Port 1

Port WWN:	20:37:00:80:E5:24:9A:D6
Node WWN:	20:06:00:80:E5:24:9A:D6

Controller B: Port 2

Port WWN:	20:47:00:80:E5:24:9A:D6
Node WWN:	20:06:00:80:E5:24:9A:D6

Note that the node WWN for all ports is identical: 20060080E5249AD6 (which is the node WWN of the storage)

• Find out which controller is connected to which HBA

Combining output from storage and fcinfo remote-port, we conclude that controller A port 1 is connected to first HBA and Controller B port 1 is connected to second HBA

•  Additional info:

http://www.sunsolarisadmin.com/hardware/how-to-find-the-wwn-world-wide-name-in-sun-solaris/
http://docs.oracle.com/cd/E19082-01/819-2240/fcinfo-1m/index.html
http://blogs.warwick.ac.uk/peggleton/entry/disks_cfgadm_fcinfo/

        State: online

        State: online

NFS setup for Oracle install

• On the client side:

mkdir /test1
Edit the /etc/vfstab file and add the following line to mount the file system as NFS

10.129.195.224:/export/test1    -       /test1  nfs     -       yes     rw,bg,hard,nointr,rsize=32768,wsize=32768,proto=tcp,noac,forcedirectio,vers=3,suid

or
zfssaIp:/fs    -    /clientfs    nfs    -    yes    {mount options}

Refer to following metalink doc for mount option info
Mount Options for Oracle files when used with NFS on NAS devices [ID 359515.1]

You can unmount the filesystem with the following command
umount /test1

• On the storage side:

Create a storage pool (Configuration ->Storage).
Create a new project (Shares->Project)
Create a FS under that project (Shares->Shares)

• Issue resolution:

• Direct NFS: Failed to set socket buffer size.wtmax=[1048576] rtmax=[1048576], errno=-1

You have to increase tcp_buf_size. Kindly set the value of the tcp buffer to a value higher than wtmax and rtmax specified above. Setting to the same value does not resolve the issue.

You can issue the following commands to set and display the values:

# /usr/sbin/ndd -set /dev/tcp tcp_max_buf 1056768
# /usr/sbin/ndd -set /dev/tcp tcp_xmit_hiwat 1056768
#  /usr/sbin/ndd -set /dev/tcp tcp_recv_hiwat 1056768
#
# /usr/sbin/ndd /dev/tcp tcp_max_buf
1056768
# /usr/sbin/ndd /dev/tcp tcp_xmit_hiwat
1056768
# /usr/sbin/ndd /dev/tcp tcp_recv_hiwat
1056768

Refer to  MOS ID 1352886.1 for details.

• Additional info:

https://blogs.oracle.com/taylor22/entry/nfs_root_access_on_sun
http://robpetti.com/?p=85 : NFS mounts as nobody:nobody

iSCSI setup

This blog describes the steps to setup S7420 storage as iSCSI device.

Verify that the iscsi initiator service is enabled on the Solaris initiator. If not, enable the service.


root@dat02:~# svcs | grep initiator
online         Jun_08   svc:/system/fcoe_initiator:default
online         Jun_08   svc:/network/iscsi/initiator:default


Log on to Solaris initiator and find out the IQN value of the server and the number of configured sessions

root@dat02:~# iscsiadm list initiator-node
Initiator node name: iqn.1986-03.com.sun:01:002128f52ca4.4fb2e431
Initiator node alias: dat02
        Login Parameters (Default/Configured):
                Header Digest: NONE/-
                Data Digest: NONE/-
                Max Connections: 65535/-
        Authentication Type: NONE
        RADIUS Server: NONE
        RADIUS Access: disabled
        Tunable Parameters (Default/Configured):
                Session Login Response Time: 60/-
                Maximum Connection Retry Time: 180/-
                Login Retry Time Interval: 60/-
        Configured Sessions: 1



Solaris initiator can use the target only after configuring a discovery method (a way to identify targets on the network). Solaris supports three discovery methods

• static discovery
• SendTargets
• iSNS

The following commands list which discovery method is enabled

root@dat02:~# iscsiadm list discovery
Discovery:
        Static: disabled
        Send Targets: enabled
        iSNS: disabled

The following commands enables the sendTarget discovery method

SendTargets:

Enable the send target discovery method and add the IP address of the target (storage) to the discovery address.

root@dat02:~# iscsiadm modify discovery --sendtarget enable
root@dat02:~# iscsiadm add discovery-address 10.129.195.222


You can list the IP address of target that is discovered by the server using the list command:

root@dat01:/etc# iscsiadm list discovery-address
Discovery Address: 10.129.195.222:3260
The following commands deletes a particular iSCSI target from discovery list :

root@dat01:/etc# iscsiadm remove discovery-address 10.129.195.222
root@dat01:/etc# iscsiadm list target -vS


This discovery method can be disabled as under:

iscsiadm modify discovery --sendtargets disable


Once the initiator is configured with a valid discovery method, the initiator should see one or more targets by issuing the list target subcommand

iscsiadm list target


root@dat02:~# iscsiadm list target
Target: iqn.1986-03.com.sun:02:b3c7500a-b7ca-c791-a949-d79bf0862c17
        Alias: iscsi-target
        TPGT: 2
        ISID: 4000002a0000
        Connections: 1
To get the list of all the LUNs exposed to the server, use -S option

root@dat02:~# iscsiadm list target  -help
iscsiadm: '-h': invalid option
        iscsiadm list target [OPTIONS] [<target-name ...>]
        OPTIONS:
                -v, --verbose
                -S, --scsi-target
For more information, please see iscsiadm(1M)
root@dat02:~# iscsiadm list target  -vS
Target: iqn.1986-03.com.sun:02:b3c7500a-b7ca-c791-a949-d79bf0862c17
        Alias: iscsi-target
        TPGT: 2
        ISID: 4000002a0000
        Connections: 1
                CID: 0
                  IP address (Local): 10.129.195.190:32957
                  IP address (Peer): 10.129.195.222:3260
                  Discovery Method: SendTargets
                  Login Parameters (Negotiated):
                        Data Sequence In Order: yes
                        Data PDU In Order: yes
                        Default Time To Retain: 20
                        Default Time To Wait: 2
                        Error Recovery Level: 0
                        First Burst Length: 65536
                        Immediate Data: yes
                        Initial Ready To Transfer (R2T): yes
                        Max Burst Length: 262144
                        Max Outstanding R2T: 1
                        Max Receive Data Segment Length: 32768
                        Max Connections: 32
                        Header Digest: NONE
                        Data Digest: NONE

        LUN: 3
             Vendor:  SUN
             Product: ZFS Storage 7420
             OS Device Name: /dev/rdsk/c0t600144F0EE8675B100004FD626260006d0s2
        LUN: 2
             Vendor:  SUN
             Product: ZFS Storage 7420
             OS Device Name: /dev/rdsk/c0t600144F0EE8675B100004FD61BBC0005d0s2
        LUN: 1
             Vendor:  SUN
             Product: ZFS Storage 7420
             OS Device Name: /dev/rdsk/c0t600144F0EE8675B100004FD39DE50004d0s2
        LUN: 0
             Vendor:  SUN
             Product: ZFS Storage 7420
             OS Device Name: /dev/rdsk/c0t600144F0EE8675B100004FD39CC80003d0s2



On the storage side:

Go to Configuration->SAN-> and setup target and initiator portal. There is a default target and initiator created  which can be used as well.
Go to Confiiguration->Storage pool and create a new storage pool if required
Go to Shares->Project and create a new project.
Add LUNS to these project and select the target and initiator portals for the LUNS.(the default ones or the ones you created for extra security). You can select the target and initiator options under the Protocol sections when you edit the LUN

You do not need to create target and initiator group. The default ones work one. If you can discovery address and cannot view the target, check /var/adm/messages for any error messages. Try restarting iSCSI service on target ZFSSA and reboot the Solaris server , if needed.
Additional info:

http://prefetch.net/articles/solarisiscsi.html
  http://sosc-dr.sun.com/bigadmin/features/articles/7000_oracle_iscsi_asm.pdf
http://www.youtube.com/watch?feature=endscreen&NR=1&v=iETgvITMQ1w
http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-104-iscsi-luns-linux-518400.html

Creating volume with Sun StorageTek Common Array Manager (CAMS)

Some terminology:

Storage Pool: We can now take all or part of the disks in the array and attach a profile to it. This would mean that all the disks in this pool would have the RAID level, the segment size, read ahead property, etc of the storage profile that was attached to it.

Virtual Disks: We can think of virtual disks as groups of physical disks on which the volumes are created.

Volumes: From the Storage Pool that we created we could create one or more volumes. The volume that we create will be the ones that the servers see as disks. We can then mount filesystems on them, or use them as raw devices from the server side.

Mappings: The volumes that we create are now mapped onto controllers. A volume can only be mapped/attached to one controller. It's through the ports of the controller that the volume talks to the server.

Controller Ports: Controller usually have multiple ports, where each port could have a speed of say 2Gbps or 4 Gbps or something. If our controller has 4 x 4 Gbps ports, and you have connected Links from the 4 ports to the server, then you'll have 4 different paths leading to the same LUN.We  could use MpxIO to fold the 4 paths into a single path on the server side.

Note that if we are using only 1 x 4 Gbps port of your controller, and converting Gbps to MB/sec,  that you can get a throughput of around 500 MB/sec.

Here are the steps to create a Volume using Sun StorageTek Common Array Manager(CAMS):

Start with creating a Profile → Storage Pool → Create a new Volume → Map the Volume to a controller.

Now when you do a format from the server(Solaris) side you'll see the LUNs that you have created. 

Documentation :  http://docs.sun.com/app/docs/prod/stor.arrmgr66?l=en&a=view 

f you want to create 7 volumes out of 14 disks, each having 2 disk,
first create a profile with 2 disks

name : raid0_RA
Raid : RAID)
Segment L 512KB
Noof disk: 2
Disk type : Any

Assign that profile to a pool

name : RA_POOL
storage profile: raid0_RA


Create volume out of that pool, select Currently unassigned disk ; Select specific disks to be used; choose two disk from the list; give a volume name, select number ot create as 1; assign it a 
controller ; Map to the default storage domain and select ok.;

Volume Name:	vol7
Volume Capacity:	Fill One Virtual Disk with volumes of size 930.523 GB
Pool Name:	raid0_pool
Storage Profile:	raid0
Storage Characteristics:	RAID:0, Segment Size: 512 KB, Read-Ahead: false, Number Of Disks: 2, Disk Type: ANY
Virtual Disk Name:	Create New Virtual Disk Using the Following Drives: [t85d16, t85d15]
Controller:	B
Mapping To:	Default Storage Domain - LUN:7

If you do not care about which disks are getting assigned and you want to create 7 volumes for LDOM OS disk, you can create a profile namely ldom_disk and assign it 7 disks. You can then create a pool out of it and create 7 volumes from that pool with each containing one virtual disk.

On the server side,run devfsadm -Cv. Also, check the OS device name on which the HBA is attached and issue cfgadm -al  | grep c<portnumber> to verify that you can view all the volumes.  You can also run the following command : cfgadm -al -o show_FCP_dev c2 to see all the FC devices on the C<portnumber> controller

Fiber Channel Storage: Figure out what is attached

• Find out how many HBA ports the server has
• fcinfo hba-port | grep HBA
• HBA Port WWN: 2100001b32016904
• HBA Port WWN: 2101001b32216904
• HBA Port WWN: 2100001b3214bf2b
• HBA Port WWN: 2101001b3234bf2b
• fcinfo hba-port | grep "OS Device Name"
• OS Device Name: /dev/cfg/c8
• OS Device Name: /dev/cfg/c9
• OS Device Name: /dev/cfg/c1
• OS Device Name: /dev/cfg/c10
•  fcinfo hba-port | grep State
• State: online
• State: online
• State: online
• State: online
• Issue cfgadm command to find out the ports attached to the OS devices
• cfgadm -al  | more
• Ap_Id                          Type         Receptacle   Occupant     Condition
• c1                             fc-private   connected    configured   unknown
• c1::201400a0b8563966           disk         connected    configured   unknown
• c8::200600a0b82aa8e3           disk         connected    configured   unknown
• c9                             fc-private   connected    configured   unknown
• c9::200700a0b82aa8e3           disk         connected    configured   unknown
• c10                            fc-private   connected    configured   unknown
• c10::201500a0b8563966          disk         connected    configured   unknown
• On the storage side, find the ports and match them with the ids next to OS device name
• To find out the disk name as listed in format command, refer to the WWN number listed in the volume sections of the storage 

Look at the mapping section on storage side
     Volume name     Controller   LUN
       V1                    A               0
 
This matches to <controller id for A>,0 on format command

• Verify if multipathing is enabled on a disk
• mpathadm show lu
  /dev/rdsk/c0t600A0B800056396600000FDB4EBC22C4d0s0

Mapping disk instance name with physical name

Say you need to find the instance name for the following disks:
'/dev/rdsk/c6t20030003BACCC8FAd0s6',
'/dev/rdsk/c6t20030003BACCC8FAd1s6',
'/dev/rdsk/c6t20030003BACCC902d0s6',
'/dev/rdsk/c6t20030003BACCC902d1s6



Look for 902 (0 and 1) in /etc/path_to_inst file:
"/pci@0/pci@0/pci@8/pci@0/pci@a/SUNW,qlc@0,1/fp@0,0/ssd@w20030003baccc902,1" 3 "ssd"
"/pci@0/pci@0/pci@8/pci@0/pci@a/SUNW,qlc@0,1/fp@0,0/ssd@w20030003baccc902,0" 4 "ssd"

Look for 8FA (0 and 1 ) in /etc/path_to_inst file:
"/pci@0/pci@0/pci@8/pci@0/pci@a/SUNW,qlc@0,1/fp@0,0/ssd@w20030003baccc8fa,1" 72 "ssd"
"/pci@0/pci@0/pci@8/pci@0/pci@a/SUNW,qlc@0,1/fp@0,0/ssd@w20030003baccc8fa,0" 7 "ssd"


The mapping is as under:
'/dev/rdsk/c6t20030003BACCC8FAd0s6  : ssd7
'/dev/rdsk/c6t20030003BACCC8FAd1s6:ssd72
/dev/rdsk/c6t20030003BACCC902d0s6:ssd4
/dev/rdsk/c6t20030003BACCC902d1s6:ssd3

Looking at another FC storage

Storage configuration is as under:

• luxadm  -e port

4 HBAs are connected:
/devices/pci@1,700000/SUNW,qlc@0/fp@0,0:devctl                     CONNECTED
/devices/pci@1,700000/SUNW,qlc@0,1/fp@0,0:devctl                   CONNECTED
/devices/pci@13,700000/SUNW,qlc@0/fp@0,0:devctl                    CONNECTED
/devices/pci@13,700000/SUNW,qlc@0,1/fp@0,0:devctl                  CONNECTED

You can find out the WWN number of each HBA as under :

•  luxadm -e dump_map /devices/pci@1,700000/SUNW,qlc@0/fp@0,0:devctl

Pos AL_PA ID Hard_Addr Port WWN         Node WWN         Type
0     e1  4     e1     200700a0b832f4bd 200600a0b832f4bc 0x0  (Disk device)
1     1   7d    0      2100001b320b6958 2000001b320b6958 0x1f (Unknown Type,Host Bus Adapter)

• # luxadm -e dump_map /devices/pci@1,700000/SUNW,qlc@0,1/fp@0,0:devctl

Pos AL_PA ID Hard_Addr Port WWN         Node WWN         Type

0     e8  1     e8     200600a0b832f4bd 200600a0b832f4bc 0x0  (Disk device)

1     1   7d    0      2101001b322b6958 2001001b322b6958 0x1f (Unknown Type,Host Bus Adapter)

• # luxadm -e dump_map /devices/pci@13,700000/SUNW,qlc@0/fp@0,0:devctl

Pos AL_PA ID Hard_Addr Port WWN         Node WWN         Type
0     ef  0     ef     202400a0b83892c4 200400a0b83892c4 0x0  (Disk device)
1     1   7d    0      2100001b329bdf46 2000001b329bdf46 0x1f (Unknown Type,Host Bus Adapter)

• # luxadm -e dump_map /devices/pci@13,700000/SUNW,qlc@0,1/fp@0,0:devctl

Pos AL_PA ID Hard_Addr Port WWN         Node WWN         Type
0     e8  1     e8     203400a0b83892c4 200400a0b83892c4 0x0  (Disk device)
1     1   7d    0      2101001b32bbdf46 2001001b32bbdf46 0x1f (Unknown Type,Host Bus Adapter)


Additional info about HBAs including their speed can be found by issuing the following comand

• fcinfo hba-ports

# fcinfo hba-port
HBA Port WWN: 2100001b320b6958
        OS Device Name: /dev/cfg/c1
        Manufacturer: QLogic Corp.
        Model: 375-3356-02
        Firmware Version: 05.03.02
        FCode/BIOS Version:  BIOS: 1.24; fcode: 1.24; EFI: 1.08;
        Serial Number: 0402G00-0814437006
        Driver Name: qlc
        Driver Version: 20100301-3.00
        Type: L-port
        State: online
        Supported Speeds: 1Gb 2Gb 4Gb
        Current Speed: 4Gb
        Node WWN: 2000001b320b6958
HBA Port WWN: 2101001b322b6958
        OS Device Name: /dev/cfg/c2
        Manufacturer: QLogic Corp.
        Model: 375-3356-02
        Firmware Version: 05.03.02
        FCode/BIOS Version:  BIOS: 1.24; fcode: 1.24; EFI: 1.08;
        Serial Number: 0402G00-0814437006
        Driver Name: qlc
        Driver Version: 20100301-3.00
        Type: L-port
        State: online
        Supported Speeds: 1Gb 2Gb 4Gb
        Current Speed: 4Gb
        Node WWN: 2001001b322b6958
HBA Port WWN: 2100001b329bdf46
        OS Device Name: /dev/cfg/c6
        Manufacturer: QLogic Corp.
        Model: 371-4325-01
        Firmware Version: 05.03.02
        FCode/BIOS Version:  BIOS: 2.02; fcode: 2.03; EFI: 2.01;
        Serial Number: 0402H00-1002804353
        Driver Name: qlc
        Driver Version: 20100301-3.00
        Type: L-port
        State: online
        Supported Speeds: 2Gb 4Gb 8Gb
        Current Speed: 4Gb
        Node WWN: 2000001b329bdf46
HBA Port WWN: 2101001b32bbdf46
        OS Device Name: /dev/cfg/c7
        Manufacturer: QLogic Corp.
        Model: 371-4325-01
        Firmware Version: 05.03.02
        FCode/BIOS Version:  BIOS: 2.02; fcode: 2.03; EFI: 2.01;
        Serial Number: 0402H00-1002804353
        Driver Name: qlc
        Driver Version: 20100301-3.00
        Type: L-port
        State: online
        Supported Speeds: 2Gb 4Gb 8Gb
        Current Speed: 4Gb
        Node WWN: 2001001b32bbdf46

You can find out which disk are attached to which HBA by the following command:

(where P is WWN of the HBA port found by luxadm dump_map command or fcinfo hba_port command)

• fcinfo remote-port -sl -p 2100001b320b6958

Remote Port WWN: 200700a0b832f4bd

        Active FC4 Types:

        SCSI Target: yes

        Node WWN: 200600a0b832f4bc

        Link Error Statistics:

                Link Failure Count: 3

                Loss of Sync Count: 9

                Loss of Signal Count: 133

                Primitive Seq Protocol Error Count: 0

                Invalid Tx Word Count: 0

                Invalid CRC Count: 0

        LUN: 0

          Vendor: SUN

          Product: CSM200_R

          OS Device Name: /dev/rdsk/c1t4d0s2

        LUN: 1

          Vendor: SUN

          Product: CSM200_R

          OS Device Name: /dev/rdsk/c1t4d1s2

        LUN: 2

          Vendor: SUN

          Product: CSM200_R

          OS Device Name: /dev/rdsk/c1t4d2s2

        LUN: 3

          Vendor: SUN

          Product: CSM200_R

          OS Device Name: /dev/rdsk/c1t4d3s2

• # fcinfo remote-port -sl  -p  2101001b32bbdf46

Remote Port WWN: 203400a0b83892c4

        Active FC4 Types:

        SCSI Target: yes

        Node WWN: 200400a0b83892c4

        Link Error Statistics:

                Link Failure Count: 13

                Loss of Sync Count: 2167

                Loss of Signal Count: 3158

                Primitive Seq Protocol Error Count: 0

                Invalid Tx Word Count: 0

                Invalid CRC Count: 0

        LUN: 0

          Vendor: SUN

          Product: LCSM100_F

          OS Device Name: /dev/rdsk/c7t1d0s2

        LUN: 1

          Vendor: SUN

          Product: LCSM100_F

          OS Device Name: /dev/rdsk/c7t1d1s2

        LUN: 2

          Vendor: SUN

          Product: LCSM100_F

          OS Device Name: /dev/rdsk/c7t1d2s2

        LUN: 3

          Vendor: SUN

          Product: LCSM100_F

          OS Device Name: /dev/rdsk/c7t1d3s2

• # fcinfo remote-port -sl -p 2101001b322b6958

Remote Port WWN: 200600a0b832f4bd

        Active FC4 Types:

        SCSI Target: yes

        Node WWN: 200600a0b832f4bc

        Link Error Statistics:

                Link Failure Count: 3

                Loss of Sync Count: 9

                Loss of Signal Count: 69

                Primitive Seq Protocol Error Count: 0

                Invalid Tx Word Count: 0

                Invalid CRC Count: 0

        LUN: 0

          Vendor: SUN

          Product: CSM200_R

          OS Device Name: /dev/rdsk/c2t1d0s2

        LUN: 1

          Vendor: SUN

          Product: CSM200_R

          OS Device Name: /dev/rdsk/c2t1d1s2

        LUN: 2

          Vendor: SUN

          Product: CSM200_R

          OS Device Name: /dev/rdsk/c2t1d2s2

        LUN: 3

          Vendor: SUN

          Product: CSM200_R

          OS Device Name: /dev/rdsk/c2t1d3s2

• # fcinfo remote-port -sl -p 2100001b329bdf46

Remote Port WWN: 202400a0b83892c4

        Active FC4 Types:

        SCSI Target: yes

        Node WWN: 200400a0b83892c4

        Link Error Statistics:

                Link Failure Count: 10

                Loss of Sync Count: 1776

                Loss of Signal Count: 3331

                Primitive Seq Protocol Error Count: 0

                Invalid Tx Word Count: 0

                Invalid CRC Count: 0

        LUN: 0

          Vendor: SUN

          Product: LCSM100_F

          OS Device Name: /dev/rdsk/c6t0d0s2

        LUN: 1

          Vendor: SUN

          Product: LCSM100_F

          OS Device Name: /dev/rdsk/c6t0d1s2

        LUN: 2

          Vendor: SUN

          Product: LCSM100_F

          OS Device Name: /dev/rdsk/c6t0d2s2

        LUN: 3

          Vendor: SUN

          Product: LCSM100_F

          OS Device Name: /dev/rdsk/c6t0d3s

The above output shows that each HBA has four disk attached
1 HBA : 200700a0b832f4bd : c1t4do-d3
2 HBA : 203400a0b83892c4:  c7t1d0-d3
3 HBA :200600a0b832f4bd: c2t1d0-d3
4 HBA: 202400a0b83892c4:  c6t0d0-d3

On the storage side:
There are two arrays:
One is 6140 with 1.7 terabytes of storage with a total of 13 disk each of 136 GB . It has two controllers
6140 is divided into four volumes of 442GB each. Two volumes are connected to controller A and two are connected to controller B.
Each controller has 2 ports. However, the link of only port per controller is up
The wwn of the port that is up on Controller A (A/1) is
20:06:00:A0:B8:32:F4:BD
The wwn of the port that is up on controller B (B/1)is
20:07:00:A0:B8:32:F4:BD

The second storage is 2540 with 1.6 TB of storage with a total of 12 SAS disk each of 136 GB. It consist of single controller.The link of both ports on the controller are active.
The WWN of the first port of the controller (A/1) is
20:24:00:A0:B8:38:92:C4
the WWN of the second port of the controller (A/2) is oth the
20:34:00:A0:B8:38:92:C4
2540 consist of 4 volumes each of 408 GB

Combining info about storage and HBA info on server side.
c1t4 disk and c2t1 disks are from 6140.
c7t1 and c6t0 are disks from 2150. These two disk are from single controller and are identical.

Looking at a FC Storage

• fcinfo hba-port

Only one HBA is online. The state of the first HBA is offline.

 

HBA Port WWN: 2100001b320f4f21

        OS Device Name: /dev/cfg/c5

        Manufacturer: QLogic Corp.

        Model: 375-3356-02

        Firmware Version: 05.01.02

        FCode/BIOS Version:  BIOS: 1.24; fcode: 1.24; EFI: 1.08;

        Serial Number: 0402G00-0810426346

        Driver Name: qlc

        Driver Version: 20090929-2.32

        Type: unknown

        State: offline

        Supported Speeds: 1Gb 2Gb 4Gb

        Current Speed: not established

        Node WWN: 2000001b320f4f21

HBA Port WWN: 2101001b322f4f21

        OS Device Name: /dev/cfg/c6

        Manufacturer: QLogic Corp.

        Model: 375-3356-02

        Firmware Version: 05.01.02

        FCode/BIOS Version:  BIOS: 1.24; fcode: 1.24; EFI: 1.08;

        Serial Number: 0402G00-0810426346

        Driver Name: qlc

        Driver Version: 20090929-2.32

        Type: N-port

        State: online

        Supported Speeds: 1Gb 2Gb 4Gb

        Current Speed: 2Gb

        Node WWN: 2001001b322f4f21

• fcinfo remote-port -s -p 2101001b322f4f21

Remote Port WWN: 20030003ba68ec2e
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 10000003ba68ec2e
        LUN: 0
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BA68EC2Ed0s2
        LUN: 1
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BA68EC2Ed1s2
        LUN: 2
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BA68EC2Ed2s2
Remote Port WWN: 216000c0ff886cfa
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 206000c0ff086cfa
        LUN: 0
          Vendor: SUN
          Product: StorEdge 3510
          OS Device Name: /dev/rdsk/c6t216000C0FF886CFAd0s2
Remote Port WWN: 20030003ba13ec0f
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 10000003ba13ec0f
        LUN: 0
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BA13EC0Fd0s2
Remote Port WWN: 20030003baccc8fa
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 10000003baccc8fa
        LUN: 0
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BACCC8FAd0s2
        LUN: 1
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BACCC8FAd1s2
Remote Port WWN: 20030003ba13ebd2
Remote Port WWN: 20030003ba13ebd2
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 10000003ba13ebd2
        LUN: 0
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BA13EBD2d0s2
        LUN: 1
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BA13EBD2d1s2
Remote Port WWN: 216000c0ff803d33
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 206000c0ff003d33
        LUN: 0
          Vendor: SUN
          Product: StorEdge 3510
          OS Device Name: /dev/rdsk/c6t216000C0FF803D33d0s2
        LUN: 1
          Vendor: SUN
          Product: StorEdge 3510
          OS Device Name: /dev/rdsk/c6t216000C0FF803D33d1s2
        LUN: 2
          Vendor: SUN
          Product: StorEdge 3510
          OS Device Name: /dev/rdsk/c6t216000C0FF803D33d2s2
Remote Port WWN: 226000c0ffa03d33
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 206000c0ff003d33
        LUN: 0
          Vendor: SUN
          Product: StorEdge 3510
          OS Device Name: /dev/es/ses5
Remote Port WWN: 256000c0ffc86cfa
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 206000c0ff086cfa
        LUN: 0
          Vendor: SUN
          Product: StorEdge 3510
          OS Device Name: /dev/es/ses1
Remote Port WWN: 216000c0ff803a7b
Remote Port WWN: 216000c0ff803a7b
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 206000c0ff003a7b
        LUN: 0
          Vendor: SUN
          Product: StorEdge 3510
          OS Device Name: /dev/rdsk/c6t216000C0FF803A7Bd0s2
        LUN: 1
          Vendor: SUN
          Product: StorEdge 3510
          OS Device Name: /dev/rdsk/c6t216000C0FF803A7Bd1s2
Remote Port WWN: 20030003baccc902
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 10000003baccc902
        LUN: 0
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BACCC902d0s2
        LUN: 1
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BACCC902d1s2
Remote Port WWN: 256000c0ffc03a7b
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 206000c0ff003a7b
        LUN: 0
          Vendor: SUN
          Product: StorEdge 3510
          OS Device Name: /dev/es/ses2
Remote Port WWN: 20030003ba13e6a1
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 10000003ba13e6a1
        LUN: 0
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BA13E6A1d0s2
        LUN: 1
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BA13E6A1d1s2
Remote Port WWN: 20030003ba4e8829
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 10000003ba4e8829
        LUN: 0
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BA4E8829d0s2
        LUN: 1
          Vendor: SUN
          Product: T4
          OS Device Name: /dev/rdsk/c6t20030003BA4E8829d1s2
~
The above output means
Remote Port WWN: 20030003ba68ec2e  : T4:  three LUN from d0-d2
Remote Port WWN: 216000c0ff886cfa : 3510: one LUN d0
Remote Port WWN: 20030003ba13ec0f  : T4: one LUN d0
Remote Port WWN: 20030003baccc8fa: :t4: Two luns d0 and d1
Remote Port WWN: 20030003ba13ebd2:t4: two luns d0-d1
Remote Port WWN: 216000c0ff803d33:3510: thre luns do-d2
Remote Port WWN: 20030003ba4e8829: T4: two luns d0-d2
Remote Port WWN: 20030003ba13e6a1:T4: two luns d0-d1
Remote Port WWN: 216000c0ff803a7b:3510: two luns d0-d1
Remote Port WWN: 20030003baccc902 : T4 two luns d0-d1

Find out HBA port WWN on Solaris

• Issue the following command on the host to find out how many HBAs are connected to the host:

• luxadm -e port 

• The output will be as under:
• /devices/pci@0/pci@0/pci@8/pci@0/pci@a/SUNW,qlc@0/fp@0,0:devctl    NOT CONNECTED
• /devices/pci@0/pci@0/pci@8/pci@0/pci@a/SUNW,qlc@0,1/fp@0,0:devctl  CONNECTED
• The output shows that one HBA card is connected to the host

• Issue the following command to get the port numbers and associated WWN for each HBA connected to the host
• luxadm -e dump_map /devices/pci@0/pci@0/pci@8/pci@0/pci@a/SUNW,qlc@0,1/fp@0,0:devctl 

•  The sample output is as under:

Pos  Port_ID Hard_Addr Port WWN         Node WWN         Type 0    170900  0         20030003baccc902 10000003baccc902 0x0  (Disk device) 1    1c0300  0         10000000c9722f33 20000000c9722f33 0x1f (Unknown Type) 2    1f0200  0         20030003ba13e6a1 10000003ba13e6a1 0x0  (Disk device) 3    140300  0         2101001b322f4f21 2001001b322f4f21 0x1f (Unknown Type,Host Bus Adapter)

• The above output shows that the QLC HBA has two ports  namely 0 and 2 with port  wwn 20030003baccc902 and 20030003ba13e6a1 which can be mapped to the luns on the storage