Tuesday, June 19, 2012

NFS setup for Oracle install

  • On the client side:

mkdir /test1
Edit the /etc/vfstab file and add the following line to mount the file system as NFS

10.129.195.224:/export/test1    -       /test1  nfs     -       yes     rw,bg,hard,nointr,rsize=32768,wsize=32768,proto=tcp,noac,forcedirectio,vers=3,suid

or
zfssaIp:/fs    -    /clientfs    nfs    -    yes    {mount options}

Refer to following metalink doc for mount option info
Mount Options for Oracle files when used with NFS on NAS devices [ID 359515.1]

You can unmount the filesystem with the following command
umount /test1

  • On the storage side:

Create a storage pool (Configuration ->Storage).
Create a new project (Shares->Project)
Create a FS under that project (Shares->Shares)


  • Issue resolution:

  • Direct NFS: Failed to set socket buffer size.wtmax=[1048576] rtmax=[1048576], errno=-1
You have to increase tcp_buf_size. Kindly set the value of the tcp buffer to a value higher than wtmax and rtmax specified above. Setting to the same value does not resolve the issue.

You can issue the following commands to set and display the values:

# /usr/sbin/ndd -set /dev/tcp tcp_max_buf 1056768
# /usr/sbin/ndd -set /dev/tcp tcp_xmit_hiwat 1056768
#  /usr/sbin/ndd -set /dev/tcp tcp_recv_hiwat 1056768
#
# /usr/sbin/ndd /dev/tcp tcp_max_buf
1056768
# /usr/sbin/ndd /dev/tcp tcp_xmit_hiwat
1056768
# /usr/sbin/ndd /dev/tcp tcp_recv_hiwat
1056768
Refer to  MOS ID 1352886.1 for details.



  • Additional info:

https://blogs.oracle.com/taylor22/entry/nfs_root_access_on_sun
http://robpetti.com/?p=85 : NFS mounts as nobody:nobody




Backup Solaris Zones

Find out zonepath of the zone you need to backup


root@isve-111-116:~# zonecfg -z rac4Zone info zonepath
zonepath: /zonepools/zones/rac4Zone

Find out the data-set of zonepath

root@isve-111-116:~# zfs list -H -o name /zonepools/zones/rac4Zone
zonepools/zones/rac4Zone

Create a snapshot of the zonepath

root@isve-111-116:~# zfs snapshot -r zonepools/zones/rac4Zone@backup1

Verify that snapshot was successfully created

 zfs snapshot -r zonepools/zones/rac4Zone@backup1

root@isve-111-116:~# zfs list -t snap -r zonepools/zones/rac4Zone
NAME                                                      USED  AVAIL  REFER  MOUNTPOINT
zonepools/zones/rac4Zone@backup1                             0      -    34K  -
zonepools/zones/rac4Zone/rpool@backup1                       0      -    31K  -
zonepools/zones/rac4Zone/rpool/ROOT@backup1                  0      -    31K  -
zonepools/zones/rac4Zone/rpool/ROOT/solaris@install      6.97M      -   291M  -
zonepools/zones/rac4Zone/rpool/ROOT/solaris@backup1       387K      -  5.36G  -
zonepools/zones/rac4Zone/rpool/ROOT/solaris/var@install  27.4M      -  66.2M  -
zonepools/zones/rac4Zone/rpool/ROOT/solaris/var@backup1      0      -  82.5M  -
zonepools/zones/rac4Zone/rpool/dump@backup1                  0      -    16K  -
zonepools/zones/rac4Zone/rpool/export@backup1                0      -    32K  -
zonepools/zones/rac4Zone/rpool/export/home@backup1       6.24M      -  1.83G  -
zonepools/zones/rac4Zone/rpool/swap@backup1                  0      -    16K  -


Archive the zonepath

zfs send -rc zonepools/zones/rac4Zone@backup1 > /cont/rac4

Verify that the archive of the zonepath is created
root@isve-111-116:/cont# file /cont/rac4
/cont/rac4:     ZFS snapshot stream

iSCSI setup

This blog describes the steps to setup S7420 storage as iSCSI device.

Verify that the iscsi initiator service is enabled on the Solaris initiator. If not, enable the service.


root@dat02:~# svcs | grep initiator
online         Jun_08   svc:/system/fcoe_initiator:default
online         Jun_08   svc:/network/iscsi/initiator:default


Log on to Solaris initiator and find out the IQN value of the server and the number of configured sessions

root@dat02:~# iscsiadm list initiator-node
Initiator node name: iqn.1986-03.com.sun:01:002128f52ca4.4fb2e431
Initiator node alias: dat02
        Login Parameters (Default/Configured):
                Header Digest: NONE/-
                Data Digest: NONE/-
                Max Connections: 65535/-
        Authentication Type: NONE
        RADIUS Server: NONE
        RADIUS Access: disabled
        Tunable Parameters (Default/Configured):
                Session Login Response Time: 60/-
                Maximum Connection Retry Time: 180/-
                Login Retry Time Interval: 60/-
        Configured Sessions: 1



Solaris initiator can use the target only after configuring a discovery method (a way to identify targets on the network). Solaris supports three discovery methods
  • static discovery
  • SendTargets
  • iSNS

The following commands list which discovery method is enabled

root@dat02:~# iscsiadm list discovery
Discovery:
        Static: disabled
        Send Targets: enabled
        iSNS: disabled

The following commands enables the sendTarget discovery method

SendTargets:

Enable the send target discovery method and add the IP address of the target (storage) to the discovery address.

root@dat02:~# iscsiadm modify discovery --sendtarget enable
root@dat02:~# iscsiadm add discovery-address 10.129.195.222


You can list the IP address of target that is discovered by the server using the list command:

root@dat01:/etc# iscsiadm list discovery-address
Discovery Address: 10.129.195.222:3260
The following commands deletes a particular iSCSI target from discovery list :

root@dat01:/etc# iscsiadm remove discovery-address 10.129.195.222
root@dat01:/etc# iscsiadm list target -vS


This discovery method can be disabled as under:

iscsiadm modify discovery --sendtargets disable


Once the initiator is configured with a valid discovery method, the initiator should see one or more targets by issuing the list target subcommand

iscsiadm list target


root@dat02:~# iscsiadm list target
Target: iqn.1986-03.com.sun:02:b3c7500a-b7ca-c791-a949-d79bf0862c17
        Alias: iscsi-target
        TPGT: 2
        ISID: 4000002a0000
        Connections: 1
To get the list of all the LUNs exposed to the server, use -S option

root@dat02:~# iscsiadm list target  -help
iscsiadm: '-h': invalid option
        iscsiadm list target [OPTIONS] [<target-name ...>]
        OPTIONS:
                -v, --verbose
                -S, --scsi-target
For more information, please see iscsiadm(1M)
root@dat02:~# iscsiadm list target  -vS
Target: iqn.1986-03.com.sun:02:b3c7500a-b7ca-c791-a949-d79bf0862c17
        Alias: iscsi-target
        TPGT: 2
        ISID: 4000002a0000
        Connections: 1
                CID: 0
                  IP address (Local): 10.129.195.190:32957
                  IP address (Peer): 10.129.195.222:3260
                  Discovery Method: SendTargets
                  Login Parameters (Negotiated):
                        Data Sequence In Order: yes
                        Data PDU In Order: yes
                        Default Time To Retain: 20
                        Default Time To Wait: 2
                        Error Recovery Level: 0
                        First Burst Length: 65536
                        Immediate Data: yes
                        Initial Ready To Transfer (R2T): yes
                        Max Burst Length: 262144
                        Max Outstanding R2T: 1
                        Max Receive Data Segment Length: 32768
                        Max Connections: 32
                        Header Digest: NONE
                        Data Digest: NONE

        LUN: 3
             Vendor:  SUN
             Product: ZFS Storage 7420
             OS Device Name: /dev/rdsk/c0t600144F0EE8675B100004FD626260006d0s2
        LUN: 2
             Vendor:  SUN
             Product: ZFS Storage 7420
             OS Device Name: /dev/rdsk/c0t600144F0EE8675B100004FD61BBC0005d0s2
        LUN: 1
             Vendor:  SUN
             Product: ZFS Storage 7420
             OS Device Name: /dev/rdsk/c0t600144F0EE8675B100004FD39DE50004d0s2
        LUN: 0
             Vendor:  SUN
             Product: ZFS Storage 7420
             OS Device Name: /dev/rdsk/c0t600144F0EE8675B100004FD39CC80003d0s2



On the storage side:

Go to Configuration->SAN-> and setup target and initiator portal. There is a default target and initiator created  which can be used as well.
Go to Confiiguration->Storage pool and create a new storage pool if required
Go to Shares->Project and create a new project.
Add LUNS to these project and select the target and initiator portals for the LUNS.(the default ones or the ones you created for extra security). You can select the target and initiator options under the Protocol sections when you edit the LUN

You do not need to create target and initiator group. The default ones work one. If you can discovery address and cannot view the target, check /var/adm/messages for any error messages. Try restarting iSCSI service on target ZFSSA and reboot the Solaris server , if needed.
Additional info:

http://prefetch.net/articles/solarisiscsi.html
  http://sosc-dr.sun.com/bigadmin/features/articles/7000_oracle_iscsi_asm.pdf
http://www.youtube.com/watch?feature=endscreen&NR=1&v=iETgvITMQ1w
http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-104-iscsi-luns-linux-518400.html