Wednesday, November 3, 2010

RAC Preinstallation Check II

  • Refer to previous post to setup ssh connectivity between nodes
  • Create the required user and groups and directories
    • # groupadd -g 1000 oinstall
    • # groupadd -g 1031 dba
    • # useradd -u 1101 -g oinstall -G dba oracle
    • # mkdir -p /u01/app/11.2.0/grid
    • # mkdir -p /u01/app/oracle
    • # chown -R oracle:oinstall /u01
    • # chmod -R 775 /u01/
  • Create the fixup scripts before running the installer to fix up the prerequisite requirements:
    • ./runcluvfy.sh stage -pre crsinst -n node1,node2 -fixup -verbose
  • Setup the following values as root
    • ndd -set /dev/tcp tcp_smallest_anon_port 9000
    •  ndd -set /dev/tcp tcp_largest_anon_port 65500
    •  ndd -set /dev/udp udp_smallest_anon_port 9000
    •  ndd -set /dev/udp udp_largest_anon_port 65500
  • Regarding NTP (network time protocol)service\
    • Solaris NTP enables time synchronization on the network. Solaris NTP uses a software called xntpd. This is a OS daemon which sets and maintains the system time-of-day in synchronism with Internet standard time servers  Detailed :http://www.sun.com/blueprints/0701/NTP.pdf 
    • Verify that the service is up
      • svcs ntp
    • Enable the service
      • svcadm enable ntp

Tuesday, October 19, 2010

RAC Management tools
  • olsnodes
    • olsnodes command provides the list , their VIP and numbers of nodes in the cluster. It is found at $ORA_CRS_HOME/bin directory
  • Oifcfg
    • Oracle Interface configuration tool (oifcfg) helps to define and administer network interfaces
      1. oifcig iflist
      2. oifcfg setif
      3. oifcfg getif
      4. oifcfg delif
  • crsctl
    • crsctl commands perform the following tasks on Oracle clusterware:
      • Starting and stopping Oracle clusterware resources
        •  crsctl start cluster
        • crsctl stop cluster
        • crsctl check cluster
      • Enabling and disabling Oracle clusterware daemons
      • Checking the health of the cluster
        • crsctl check cluster
        • crsctl check crs
        • crsctl check resource
        • crsctl check ctss
      •  Managing thrid party application resources
      • Integrating Intelligent Platform mangement interface (IPMI) with Oracle clusterware
      • Debugging Oracle clusterware components

Monday, October 11, 2010

RAC Preinstallation Check

  • Memory requirement
    • Every node should have minimum of 1 GB
    • prtconf | grep Memory
  • Swap requirement
    • Swap space should be set to twice the amount of RAM for systems with 2GB of RAM or less. For systems with 2GB to 8GB, use swap space equal to RAM. For systems over 8GB , use .75 times of the size of RAM
    • Verify the swap is set to .75 times the size of RAM
    • swap -s
  • tmp space
    • Atleast 400MB of disk space is required in /tmp
    • df -h /tmp
  • Maximum Open File descriptors
    • To check ulimit -n
    • To set ulimit -n <new value>
  • Network requirements
  • You should have minimum of 2 network interfaces per node
    • dladm show-link
  • You should have three network address for each node
    • Public IP address
      • ping <public-node-name>
    • Virtual IP address : Used by applications for failover in case of node failure
      • Do not plump Virtual ip address. Pinging virutal address should result in failure
      • The virtual IP address is on the same subnet as your public interface
    • Private IP address: Used by Oracle clusterware for internode communication
      • It should be on the same subnet reserved for private networks such as 10.0.0.0 or 192.168.0.0
      • It should use dedicated switches or physically separate private network, reachable only by the cluster member nodes, prefably using high-speed NICs
      • It cannot be registered on the same subnet that is registered to a public IP address
      • ping <private-node-name>
  • The /etc/hosts should have following entries for each node
    • Your public node name,public node name.domainname
    • Your private node name, private nodename.domainname
    • Your vip node name, vip nodename.domainname
  • About interfaces on all nodes
    • The public interface names associated with the network adapters for each network must be the same on all nodes and the private interface names assoicated with the network adaptors should be the same For example: With a two-node cluster, you cannot configure network adapters on node1 with eth0 as the public interface, but on node2 have eth1 as the public interface. Public interface names must be the same, so you must configure eth0 as public on both nodes. You should configure the private interfaces on the same network adapters as well. If eth1 is the private interface for node1, then eth1 should be the private interface for node2.
  • SSH connectivity
    • Passwordless SSH connectivity should be establed between all cluster nodes.  OUI can automatically configure password SSH connectivity. For that to happen, make sure there are no stty commands in oracle user probile. By default OUI searches for public keys in /usr/local/etc directory and it searches for ssh-keygen binaries in /usr/local/bin directory. However, in Solaris public keys are found under /etc/ssh and ssh-keygen binaries are under /usr/bin. So the following softlinks needs to be created prior to starting OUI
      • ln -s /etc/ssh /usr/local/etc
      • ln -s /usr/bin /usr/local/bin
    • Create the links as mentioned above and invoke sshsetup.sh script in staging area. Verify you can ssh without password
      • ssh <node1> date
      • ssh <node2> date
  • More info
  • Additional notes
  • Verifying the existance of public IPs and VIPs
    • Use ypwhich, ypcat hosts    
  • Network setup : Refer to IP services guide for details
    • Issue dladm show-link command to find out installed interfaces   
      • Issue ifconfig -a command to determine which interface is plumped 
        • To configure and plump an interface named el000g1
          • ifconfig e1000g1 plump up      
          •  ifconfig e1000g1 <address> netmask+  
          • Verify that interface is up : ifconfig -a    
        • To make interface e1000g1 plumping persistent across reboots
          • Create a file /etc/hostname.e1000g1      
          • Add the address of the interface to this file
          • vi /etc/hostname.e1000g1    
          • Add entries for the new interface into /etc/inet/ipnodes     
        • Perform a reconfiguration reboot
          • reboot -- -r       
        • Verfiy that interface is up : ifconfig -a      
      • Solaris supports two types of interfaces
        • Legacy interfaces
          • They are DLPI and GLDv2 interfaces. Some legacy types are eri, qge,and ce  
        • Non-VLAN interfaces
          • These interfaces are GLDv3 interfaces.    
          •  bge,xge and e1000g are non-VLAN interfaces      


Wednesday, May 12, 2010

ASM does not start after system reboot

Metalink note if ASM is not coming up
https://support.oracle.com/CSP/main/article?cmd=show&id=947520.1&type=NOT


inf01% crsctl check css
CRS-4529: Cluster Synchronization Services is online
inf01% crsctl check has
CRS-4638: Oracle High Availability Services is online
inf01%


if has is not online : crsctl start has


Also, start the listener from $ASM_HOME. Also verify that ORACLE_HOME and ORACLE_SID are set correctly.

About tablespace

execute the following query to find out datafile-tablespace association:


col file_name format a50
col tablespace_name format a10

 
SELECT file_name, tablespace_name, (bytes/1024 * 1024) MB
FROM dba_data_files
ORDER BY 1;
 
Set autoextend on for the datafile associated with required tbsp:

alter database datafile
   '/u01/app/oracle/ts_32k.dbf'
autoextend on;

More info about tbspc: http://psoug.org/snippet/TABLESPACE--List-tablespaces-files-allocated-and-free-space_852.htm

About Schema

To calculate the schema (user) size :



  • The schema is owned by the user. To find out which user has how big of the DB:


SELECT s.owner,SUM (s.BYTES) / (1024 * 1024 * 1024) SIZE_IN_GB FROM dba_segments s GROUP BY s.owner;


  • If you are logged in as the schema user, you can issue the following command to get the size of the database files:

select sum(bytes)/1024/1024/1024 data_size from dba_data_files ;


  • You can find the size of the individual tables in that schema using the following query:


select owner, table_name, round((num_rows*avg_row_len)/(1024*1024)) MB
from all_tables
where owner='<schema owner username>'  -- Exclude system tables.
and num_rows > 0  -- Ignore empty Tables.
order by MB desc -- Biggest first.
;



  • To list all the objects owner by the schema user :


SELECT tablespace_name, owner, segment_type "Object Type"
FROM   sys.dba_segments WHERE  owner='TKCSOWNER'  ;

select object_name, object_type from user_objects where object_type='TABLE' ;
select object_name, object_type from user_objects where object_type='INDEX' ;


Here is the script that does that :


spool  sch.out replace
column object_name format a30
select object_name, object_type from user_objects where object_type='TABLE' ;
column owner format a30
SELECT s.owner,SUM (s.BYTES) / (1024 * 1024 * 1024) SIZE_IN_GB FROM dba_segments s GROUP BY s.owner;
select sum(bytes)/1024/1024/1024 data_size from dba_data_files ;
column owner format a30
column table_name format a30
select owner, table_name, round((num_rows*avg_row_len)/(1024*1024)) MB
from all_tables
where owner='tpcc'  -- Exclude system tables.
and num_rows > 0  -- Ignore empty Tables.
order by MB desc -- Biggest first.
;
spool off ;


  • To find info about columns and indexes on all tables



spool /tmp/123.lst replace
column table_name format a20
select  table_name, count(*) from all_tab_columns where owner='TKCSOWNER' and table_name in (select object_name from user_objects where object_type='TABLE') group by table_name;

column uniqueness format a15
column index_name format a15
column table_name format a15
column column_name format a15
select
b.uniqueness, a.index_name, a.table_name, a.column_name
from all_ind_columns a, all_indexes b
where a.index_name=b.index_name
and a.index_owner='TKCSOWNER'
and a.table_name in (select object_name from user_objects where object_type='TABLE')
order by a.table_name, a.index_name, a.column_position;

spool off;








To find out if any non standard block size is used:
select unique block_size from V$buffer_pool ;

Tuesday, May 4, 2010

Undo Space Mangement

Undo space management:

You can look at V$UNDOSTAT view to estimate the amount of undo space required for current workload.
Oracle uses this view to tune undo usage in the system. This view returns NULL if the system is in manual undo managment mode.

Each row in the view keeps statistics collected in the instance for a 10 minute interval. The rows are in descending order; so the first row  has the info about the current time period. The view contains a total of 576 rows max, spanning a 4 day cycle.

It contains following important statistics:

undotsn: the tablespace id of the active undo tablespace
txncount : total number of transactions in 10 minute cycle
maxquerylen: how much time in seconds did the longest query take

Find out HBA port WWN on Solaris

  • Issue the following command on the host to find out how many HBAs are connected to the host:
    • luxadm -e port 
    • The output will be as under:
      • /devices/pci@0/pci@0/pci@8/pci@0/pci@a/SUNW,qlc@0/fp@0,0:devctl    NOT CONNECTED
      • /devices/pci@0/pci@0/pci@8/pci@0/pci@a/SUNW,qlc@0,1/fp@0,0:devctl  CONNECTED
    • The output shows that one HBA card is connected to the host
  • Issue the following command to get the port numbers and associated WWN for each HBA connected to the host
    • luxadm -e dump_map /devices/pci@0/pci@0/pci@8/pci@0/pci@a/SUNW,qlc@0,1/fp@0,0:devctl 
  •  The sample output is as under:
        Pos  Port_ID Hard_Addr Port WWN         Node WWN         Type 0    170900  0         20030003baccc902 10000003baccc902 0x0  (Disk device) 1    1c0300  0         10000000c9722f33 20000000c9722f33 0x1f (Unknown Type) 2    1f0200  0         20030003ba13e6a1 10000003ba13e6a1 0x0  (Disk device) 3    140300  0         2101001b322f4f21 2001001b322f4f21 0x1f (Unknown Type,Host Bus Adapter)
  • The above output shows that the QLC HBA has two ports  namely 0 and 2 with port  wwn 20030003baccc902 and 20030003ba13e6a1 which can be mapped to the luns on the storage

Wednesday, March 31, 2010

Setting directIO for UFS system

Preferred way to enable directio on Oracle is to set the following option in init.ora

FILESYSTEMIO_OPTIONS = setall


Check out the following to verify if DIRECT IO is enabled in Solaris:

http://developers.sun.com/solaris/articles/solUFSdiskIO.html

Oracle 11g background processes

Oracle 11g process:

  • smon
  • lgwr
  • dbwr (multiple process)
  • chkpt
  • pmon
  • mmon
    • process to collect statistics for AWR
  • mmnl
    • process to perform manageability related task
  • mman
    • process to manage automatic shared memory
  • pnnn (optional)
    • Process for parallel query operations
  • vktm (Virtual keeper of time
    • Process tracks wall-clock time and is used as a reference time counter
  • dia0
    • Diagnostic process to detects and resolve hangs and deadlocks
  • diag
    • Diagnosability capture process for performing diagnostic dumps
  • dbrm
    • Database resource management process
  • psp0
    • Process spawner to spawn oracle process as needed
  • reco
    • recoverer process
  • d000
    • dispatcher process to listen to and receive request from connected sessions
  • s0000
    • shared server process
  • smco
    • Space management co-ordinator ...spawns wnnn proces to carry out the task
  • fbda
    • flashback data archiver process
  • cjq0
    • Job queue process used for batch processing
  • gen0
    • General tak execution process
  • qmnc
    • Advanced queing coordination process. Spawns Qnnn process as needed

Friday, March 26, 2010

Enterprise manager : ERROR: NMO not setuid-root (Unix-only)

I was not able to login as OS user using Enterprise manager 11gR1. I kept getting the following error


ERROR: NMO not setuid-root (Unix-only)

All google results indicate that this error occurs if you have not run root.sh after installation. In my scenario , I verified that I had successfully ran and that nm? binaries in $ORACLE_HOME had the right permission:
-rwsr-s--- 1 root dba 22216 Mar 16 10:37 nmb
-rwsr-s--- 1 root dba 27056 Mar 16 10:37 nmo
-rwsr-s--- 1 root dba 95328 Mar 16 10:37 nmhs

However, I still continued to get the error.

Taking a closer look, I released that the directory which had $ORACLE_HOME installed was mounted with nosetuid options. Fixing that resolved the issue.

Out of memory error while create database

  • Issue prctl $$ command as oracle and root user and ensure that proj.max-shm-mem variable shows the value you need. Else update the /etc/system with the following and reboot the system:
    • set shmsys:shminfo_shmmax=0xb40000000
  • If you do not want to reboot the system, make sure you update the project for both root user and oracle user with prcl commands given below.
  • Create a project named oracle using following command:
    • projadd oracle
  • Verify that project was created
    • projects -l
  • Associate this project with oracle user:
    • Edit /etc/user_attr file and add the following line
      • oracle::::project=oracle (four colon)
  • Verify that project oracle is associated with oracle user:
    • login as oracle user and issue id -p. It should show oracle as project for oracle user.
  • Check the existing value of shared memory variable for this project:
    • prctl -n project.max-shm-memory -i project oracle
  • To set it to a different value:
    • To set the value dynamically for the given session...will be lost after reboot
      •  prctl -n project.max-shm-memory -v 40G -r -i project oracle
    • To set it in /etc/project file so it is persistent across reboot
      • # projmod -s -K "project.max-shm-memory=(priv,4gb,deny)" oracle

Setting up DBConsole for a database

  • Verify if dbsnmp user exist:
    • select * from all_users where username= 'DBSNMP'
  • Grant users the right privileges:
    • alter user DBSNMP identified by "<new password>" ;
    • alter user DBSNMP account unlock ;
  • Ensure remote login is enabled
    • Ensure REMOTE_LOGIN_PASSWORDFILE=EXCLUSIVE is set in the init.ora for the database
      •  show parameters remote% ;
    • Ensure password file exist in $ORACLE_HOME/dbs. Else create it as under: 
    • $ orapwd file=$ORACLE_HOME/dbs/orapw$ORACLE_SID password=< some password> entries= 5
  • Create DBConsole along with repos:
    • emca -config dbcontrol db -repos create
  • For additional info refer to
  • Verify that dbconsole is up and running
    • emctl status dbconsole
      If you recreated the db, just recreate the dbcontrol repos as under:
      emca -config dbcontrol db -repos recreate
    • If db console is running on your server, you can access EM through browser as under:

Wednesday, March 17, 2010

Location of log files in Oracle 11g

Log Location
  • Foreground process traces: $ADR_HOME/trace
  • Background process traces : $ADR_HOME/trace
  • Alert Log data : $ADR_HOME/alert and trace
  • Core dumps: $ADR_HOME/cdump
  • Incident dumps : $ADR_HOME/incident/incdir_n
  • ADR_ROOT or ADR_BASE is set to $DIAGNOSTIC_DEST initialization parameter. If this parameter is not set, the database sets $DIAGNOSTIC_DEST to $ORACLE_BASE if this variable is set. Else it sets it to $ORACLE_HOME/log
  • ADR_HOME for database is set to ADR_ROOT/diag/rdbms/DB_NAME/ORACLE_SID
  • You can find out the location of all ADR variables by following sql query
    • sql > select * from V$DIAG_INFO ;

Wednesday, March 3, 2010

Interesting AOL sites

I go to these sites quiet often and hence thought of sharing it with you all.

For yoga related info, check this out:
http://artoflivingyoga.org/index.php

For the daily dose of wisdom, check this out:
http://wisdomfromsrisriravishankar.blogspot.com/
http://www.facebook.com/#!/pages/Sri-Sri-Ravi-Shankar/330938752535?ref=ts

To check for an upcoming AOL course in your area , go to
http://us.artofliving.org/
Select the state and area and it will list all the upcoming AOL courses in that area.....


For Ramdev yoga
For controlling obesity : http://www.youtube.com/watch?v=lM6Yu1Q6u8k
English version :  http://www.youtube.com/watch#!v=2xqgW1AzBLc&feature=channel

Pranayam (Breathing excercise) http://www.youtube.com/watch#!v=1e77exc7ukI&feature=related

Upcoming AOL Course in Union City and Art Excel course in Castro Valley

There is an Art of Living course (part 1 course) happening in Union City from March 17th to March 21. You can find more information at :

http://secure.artofliving.org/course_details.aspx?course_id=6720

Also, for the first time in Castro valley (about 5 minutes from my residence) an ART Excel course is happening from March 25th to March 28th. You can find more information at :

http://secure.artofliving.org/course_details.aspx?course_id=6750

ART EXCEL course is for kids between 8 to 13 years of age. You can find more information for ART EXCEL course at http://www.artoflivingyouth.org/kidzone.html

Wednesday, February 10, 2010

Oracle Configuration File

Oracle configuration file:

Oracle reads configuration parameters from either server parameter file (SPFILE) or text initialization parameter file (PFILE). When you start database instance, it tries to read SPFILE in default location. Else it searches for pfile. The search order is as under:
  • spfileORAACLE_SID.ora
  • spfile.ora
  • initORACLE_SID.ora


The default location for spfile and pfile is OH/dbs. If ASM is present, SPFILE default location is the same disk group as the datafiles.Alter system has optional scope clause that specifies the scope of the change
  • scope=spfile : The change is applied only to spfile only which impliesNo change is made to the current instance. For both dynamic and static parameters, the change is effective at the next startup and is persistent.
  • SCOPE=memory The change is applied in memory only which impliesThe change is made to the current instance and is effective immediately. For dynamic parameters, the effect is immedidate , but is not persistent as the server parameter file is not updated. This specification is not allowed for static parameters
  • SCOPE=both  The change is applied to both server parameter file and memory which implies The change is made to the current instance and is effective immediately. For dynamic parameters, the effect is persistent. For static parameters, this specification is not allowed.
The default is both if SPFILE is used for startup and MEMORY if pfile is used for startup.
You can use deferred keyword for dynamic parameters, indicating to defer the change for the future sessions.

Managing the Oracle Instance

Stages of database startup and shutdown:

  • Startup nomount:
    •  Initialization parameter file is read
  • startup open
    •  datafiles and redo log files are read
  • startup mount
    •  control files are read
Oracle shutdown options:
  • Shutdown normal
    • no new connections are allowed

before database is shutdown, the database waits for
all connected users to disconnect recovery is not required on startup

  • shutdown immediate
    • no new connections are allowed, nor are new transactions allowed to be started after the statement is issuedAny uncommited transactions are rolled back Database rollbacks active transactions and disconnects all users Recovery is not required on startup
  • shutdown transactional
    • Prevents clients from losing work and at the same  time does not require all users to log off ( no new transactions are allowed, existing transactions are completed and users disconnected after their transactions have completed)no recovery is required
  • shutdown abort
Issue this command when
You have problems starting a database instance
When you have to shut down the database instantaneously
No new connections are allowed nor are any new transactions allowed

Current SQL statements are immediately terminated
Uncomitted transactions are not rolled back
Oracle disconnects all the connected users
Recover is required on startup
shutdown timeout
If recovery is required, Oracle starts SMON (system
monitor process) to do the recovery after opening the database.

Monday, January 25, 2010

Managing Schema Objects

Create and Modify tables:
TBD



Manage Constraints: An integrity constraint defined on the table can be in one of the following states
  • enable, validate (validate is default if constraint is enabled)
  • enable, novalidate
  • disable, validate
  • disable novalidate (novalidate is default if constrant is disabled)
Create Indexes:
TBD


Create and use temporary tables
The definition of a temporary table is visible to all sessions, but the data in a temporary table is visible only to the session that inserts data into the table. There are two types of temporary tables:
  • Session specific
    • The db truncates the table when you terminate the session
    • Specified by ON COMMIT PRESERVE ROWS clause
  • Transaction specific
    • The db truncates the table after each commit.
    • Specified by ON COMMIT DELETE ROWS clause.
By default temporary tables are stored in the default temporary tablespace of the user who created it. You can alternatively create a temporary tablespace and create the temporary table on that tablespace. Eg: create global temporary table test_temp (startdate date, ......) on commit DELETE ROWS [tablespace some_temp_tbsp]

Performing Database Recovery

Overview of Data recovery advisor:

Data recovery manager is a tool to help reduce MTTS (Mean time to recover). The GUI of data recovery manager is available in database control and grid control by selecting Perform Recovery in Availability tab. Data recovery manager has following commands:
  • LIST FAILURE
    • To list all the failures
  • ADVISE FAILURE
    • View repair options which typically include both automated and manual options.
  • REPAIR FAILURE
    • To repair failure automatically. Once the failure is repaired, the repair failure command closes the relevant repaired failure.
  • CHANGE FAILURE
    • Used to alter the status and priority of failure. You can change the status of the failure to closed when you manually repair it. You can also change the priority to high or to prioritu low. You cannot set the status to open nor can you set the priority to critical by issuing this command.
Some examples:
  • To list all failures:
    • rman > list failure
  • To list details of a particular failure
    • rman > list failure detail
  • List a subset of failure
    • rman > list failure low ( low is the failure priority; high and critical are the other two failure priority values)
    • rman > list failure close
    • rman > list failure exclude failure
  • To determine repair options for a subset of failure
    • rman > advice failure <failure id>
  • To see what will be repaired if repair failure command is invoked
    • rman > repair failure preview 
  • After you list failures with list failure command, you should execute advise failure command before invoking repair failure command. Calling repair failure command before advise failure command results in error. 
  • Repair failure command repairs High and Critical failures in ADR and then closes them if they are fixed
  • DRA can be used to detect and analyze data failures in single instance database but not in RAC.
  • All failures can be automatically repaired by DRA except network I/O failures, which usually needs human intervention.

    Monday, January 11, 2010

    Announcing free workshop and AOL course in San Leandro

    Experience the power of breath and lead a life full of energy and happiness. Art of Living is pleased to announce the following two courses in San Leandro,CA. Free Science of Breath workshop
    • When : Jan 17th at 12pm
    • Where : All Saints Episcopal Church 911 Dowling Blvd, San Leandro, CA
    • Contact : Meera at 510-825-7462
    • Course fee : FREE
    • More Information: http://us.artofliving.org/sanleandro
    Art of Living Basic Course:
    • When : Jan 28 to Jan 31
    • Where : All Saints Episcopal Church. 911 Dowling Blvd, San Leandro, CA
    • Contact : Meera at 510-825-7462
    • Course fee : $175 (Offered at discounted price only this month)
    • More Information: http://us.artofliving.org/sanleandro