Setting Up RAC DataGuard on Oracle Database Appliance

Assumption:

ASM is installed as grid owner
Database is installed as oracle owner  ( RAC configured )
Password file , spfile are stored in ASM


Key Steps are as follows :
Setting up ODATEST RAC to RAC Data Guard
Primary Environment Configuration
Standby Environment Configuration
RMAN Duplicate Clone Primary to Standby Database
On the Primary Database Configure Data Guard
Register Single Instance Standby DB with Clusterware
Convert the standby database to Oracle RAC
Reconfigure the RAC Database Services for Data Guard
Configure Data Guard Broker
Testing the Data Guard Switchover Using Broker.


Primary Environment Configuration

The first step is to create the Primary Database using the ODA Manager Web Console on the Node 0 of the cluster.
Wait while the database creation job completes. This will take approx. 10-20 Mins.
Add the new Oracle SID to /etc/oratab on both of the Primary ODA nodes and check the statuso ( customized script)  output to show when database is up and running.
Set the ORACLE SID and ensure the database is archivelog and flashback log mode.
Switch on ARCHIVELOG and FLASHBACK  for the database if required.
Set the database flashback retention period to at least 48 hours.
Setup the appropriate service(s) for the database. See the script /home/oracle/bin/create_service_example
Create a test schema that can be used when we test that Data Guard replication is working correctly.
Grant the necessary permissions and create a test table.
Create the Standby Redo Logs - one more than the current redo logs, same size and on both threads.
Check that force logging is switched on.

Enable Standby File Management - to ensure that datafiles are created and dropped automatically on the Standby database as well as the primrary.
alter system set standby_file_management=auto scope=both sid='*';

Enable Remote Privileged Login - This should be set to exclusive by default on ODA's.
Setup TNS Enteries on both Primary and Standby Servers (all 4 nodes)
Check the listener status on both nodes 0 and 1 for the service enteries.

Setup Redo Transport on the Primary database to the Standby database.
alter system set log_archive_dest_2='SERVICE=odatdg LGWR ASYNC REGISTER VALID_FOR=(online_logfile, primary_role) REOPEN=60 DB_UNIQUE_NAME=odatdg' scope=both sid='*';
alter system set log_archive_dest_state_2='defer' scope=both sid='*';

Setup Fetch Archive Log Server (FAL_SERVER)
Alter system set fal_server=epstormb scope=both sid='*';

Ensure FAL_CLIENT  is not set.

Standby Environment Configuration.

NB. You will be setting up a single instance Standby database initially and the Single Instance will be converted to a
RAC database at the end of the process.

Add the Oracle SID to the /etc/oratab file on NODE 0. (ephqoda10orp)
Create a Static Listener Configuration - As the grid user add the following entry to the listener.ora file.
Reload the listener and check the status.
As the Oracle user create an initial Standby Parameter File.
Create the Standby Password File ( Remember to keep the same password as primary ,if you are using ACTIVE DATABASE RMAN Command to do the standby creation )
Create the audit directory structure on BOTH NODES.
Create the ASM Storage Directories on the Standby Side on NODE 0. As root use the odacli create dbstorage command.
odacli create-dbstorage -n odatdg
odacli list-jobs
Startup the Standby Instance on NODE 0  ( in nomount mode )
Validate the Network Connectivity from Standby Server

RMAN Duplicate Clone Primary to  Standby Database
Use the following RMAN script to duplicate standby database from active database.

$ export NLS_DATE_FORMAT='HH24:MI:SS'
$ rman
connect target sys/test__2019@odatest
connect auxiliary sys/test__2019@odatdg

run {
allocate channel p1 type disk;
allocate channel p2 type disk;
allocate channel p3 type disk;
allocate channel p4 type disk;
allocate auxiliary channel s1 type disk;
allocate auxiliary channel s2 type disk;
allocate auxiliary channel s3 type disk;
allocate auxiliary channel s4 type disk;
duplicate target database for standby from active database using backupset
dorecover
spfile
parameter_value_convert='/odatest','/odatdg','ODATEST','ODATDG'
set db_unique_name = 'odatdg'
set cluster_database = 'false'
set audit_file_dest = '/u01/app/oracle/admin/odatdg/adump'
set db_create_file_dest = '+DATA'
set log_archive_config="DG_CONFIG=(odatest,odatdg)"
set diagnostic_dest="/u01/app/oracle"
set db_recovery_file_dest="+RECO"
set db_create_online_log_dest_1="+REDO"
set log_archive_dest_2 = 'service=odatest lgwr async register valid_for=(online_logfiles, primary_role) db_unique_name=odatest'
set remote_listener = 'ephqoda1-scan-lsnr:1521'
set fal_server='odatest'
set "_cluster_flash_cache_slave_file"=""
nofilenamecheck;
}

Enable Flashback Database on the Standby Database.
Set the log_archive_dest_1 on the Standby database.

alter system set log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST valid_for=(all_logfiles,all_roles) db_unique_name=odatdg'
Start managed recovery using the Standby Logs.

recover managed standby database using current logfile disconnect;
or
alter database recover managed standby database disconnect;

Check the status of the MRP process
select inst_id, process, status, thread#, sequence#, block#, blocks from gv$managed_standby where process in ('RFS','LNS','MRP0');

If it is showing WAIT_FOR_LOG, then switch logfiles on both Primary nodes.

This ORA-12520 error will need to be resolved on odatdg. Check TNS entry on both Nodes.

As you can see it is incorrect on kphqoda11orp. Set the entry correctly in the TNSNAMES.ora file - Use the hostname and the short service name. Correcting the entry in the TNSNAMES.ora file immediately resolved the problem and media recovery kicked in and caught up.
On the Primary Database Configure Data Guard
alter system set log_archive_config='DG_CONFIG=(odatest, odatdg)' scope=both sid='*';
alter system set log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST valid_for=(all_logfiles,all_roles) db_unique_name=odatest' scope=both sid='*';
alter system set log_archive_dest_state_2=enable scope=both sid='*';

Ensure both instances are running on the primary database and ensure that both threads are being applied to the Standby database

Register Single Instance Standby DB with Clusterware

srvctl add database -d odatdg -o $ORACLE_HOME -p "/u01/app/oracle/product/12.2.0.1/dbhome_1/dbs/spfileodatdg.ora" -r physical_standby -s mount -c SINGLE -x ephqoda10orp

Check the database is registered by looking at crsstat and srvctl configuration.


/home/oracle/bin=> ./crsstat.ksh
HA Resource                                   Target     State
-----------                                   ------     -----
ora.ASMNET1LSNR_ASM.lsnr                      ONLINE     ONLINE on ephqoda10orp
ora.ASMNET2LSNR_ASM.lsnr                      ONLINE     ONLINE on ephqoda10orp
ora.DATA.COMMONSTORE.advm                     ONLINE     ONLINE on ephqoda10orp
ora.DATA.SHARED_ACFS.advm                     ONLINE     ONLINE on ephqoda10orp
ora.DATA.dg                                   ONLINE     ONLINE on ephqoda10orp
ora.LISTENER.lsnr                             ONLINE     ONLINE on ephqoda10orp
ora.LISTENER_LEAF.lsnr                        OFFLINE    OFFLINE
ora.LISTENER_SCAN1.lsnr                       ONLINE     ONLINE on ephqoda11orp
ora.LISTENER_SCAN2.lsnr                       ONLINE     ONLINE on ephqoda11orp
ora.MGMTLSNR                                  ONLINE     ONLINE on ephqoda11orp
ora.RECO.dg                                   ONLINE     ONLINE on ephqoda10orp
ora.REDO.dg                                   ONLINE     ONLINE on ephqoda10orp
ora.asm                                       ONLINE     ONLINE on ephqoda10orp
ora.chad                                      ONLINE     ONLINE on ephqoda10orp
ora.cvu                                       ONLINE     ONLINE on ephqoda11orp
ora.data.commonstore.acfs                     ONLINE     ONLINE on ephqoda10orp
ora.data.shared_acfs.acfs                     ONLINE     ONLINE on ephqoda10orp
ora.epdga.db                                  ONLINE     ONLINE on ephqoda10orp
ora.epdga.epdga_service.svc                   ONLINE     ONLINE on ephqoda10orp
ora.ephqoda10orp.ASM1.asm                     ONLINE     ONLINE on ephqoda10orp
ora.ephqoda10orp.LISTENER_EPHQODA10ORP.lsnr   ONLINE     ONLINE on ephqoda10orp
ora.ephqoda10orp.ons                          ONLINE     ONLINE on ephqoda10orp
ora.ephqoda10orp.vip                          ONLINE     ONLINE on ephqoda10orp
ora.ephqoda11orp.ASM2.asm                     ONLINE     ONLINE on ephqoda11orp
ora.ephqoda11orp.LISTENER_EPHQODA11ORP.lsnr   ONLINE     ONLINE on ephqoda11orp
ora.ephqoda11orp.ons                          ONLINE     ONLINE on ephqoda11orp
ora.ephqoda11orp.vip                          ONLINE     ONLINE on ephqoda11orp
ora.epoda.db                                  ONLINE     ONLINE on ephqoda10orp
ora.epoda.epoda_service.svc                   ONLINE     ONLINE on ephqoda10orp
ora.epstorma.db                               ONLINE     ONLINE on ephqoda10orp
ora.epstorma.epstorm_service.svc              ONLINE     ONLINE on ephqoda10orp
ora.mgmtdb                                    ONLINE     ONLINE on ephqoda11orp
ora.net1.network                              ONLINE     ONLINE on ephqoda10orp
ora.odatdg.db                                 OFFLINE    OFFLINE
ora.ons                                       ONLINE     ONLINE on ephqoda10orp
ora.proxy_advm                                ONLINE     ONLINE on ephqoda10orp
ora.qosmserver                                ONLINE     ONLINE on ephqoda11orp
ora.scan1.vip                                 ONLINE     ONLINE on ephqoda11orp
ora.scan2.vip                                 ONLINE     ONLINE on ephqoda11orp



Convert the standby database to Oracle RAC

This step is optional, if the primary database was a RAC database then the standby database can also be converted to a RAC database.

ODA (Oracle Database Appliance): How to Convert Single-Instance database to RAC on Oracle Database Appliance Using Rconfig Utility (Doc ID 1362116.1)

Pasted from <https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=441979367384725&parent=DOCUMENT&sourceId=1472797.1&id=1362116.1&_afrWindowMode=0&_adf.ctrl-state=kwluxgibf_184>

Check the status of the database is set to SINGLE.

/home/oracle/bin=> srvctl config database -d odatdg
Database unique name: odatdg
Database name:
Oracle home: /u01/app/oracle/product/12.2.0.1/dbhome_1
Oracle user: oracle
Spfile: /u01/app/oracle/product/12.2.0.1/dbhome_1/dbs/spfileodatdg.ora
Password file:
Domain:
Start options: mount
Stop options: immediate
Database role: PHYSICAL_STANDBY
Management policy: AUTOMATIC
Server pools:
Disk Groups:
Mount point paths:
Services:
Type: SINGLE
OSDBA group: dba
OSOPER group: dbaoper
Database instance: odatdg
Configured nodes: ephqoda10orp
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services:
Database is administrator managed

Change to the sample XML directory and create a copy of the ConvertToRAC_AdminManaged.xml file
Cd /u01/app/oracle/product/12.2.0.1/dbhome_1/assistants/rconfig/sampleXMLs

Edit the XML file and change YES to ONLY  -- this is for verify only


/home/oracle/bin=> cd /u01/app/oracle/product/12.2.0.1/dbhome_1/assistants/rconfig/sampleXMLs/
/u01/app/oracle/product/12.2.0.1/dbhome_1/assistants/rconfig/sampleXMLs=> ls -lrt
total 8
-rw-r----- 1 oracle oinstall 3068 Oct 13  2013 ConvertToRAC_AdminManaged.xml
-rw-r----- 1 oracle oinstall 3153 Oct 13  2013 ConvertToRAC_PolicyManaged.xml
/u01/app/oracle/product/12.2.0.1/dbhome_1/assistants/rconfig/sampleXMLs=> cp ConvertToRAC_AdminManaged.xml ConvertToRAC_AdminManaged_odatdg.xml

Run the rconfig tool as user oracle to validate if the conversion will work.

/u01/app/oracle/product/12.2.0.1/dbhome_1/assistants/rconfig/sampleXMLs=> rconfig ConvertToRAC_AdminManaged_odatdg.xml
<?xml version="1.0" ?>
<RConfig version="1.1" >
<ConvertToRAC>
    <Convert>
      <Response>
        <Result code="1" >
          Operation Failed
        </Result>
       <ErrorDetails>
             Grid Infrastructure is either not configured or not running on nodes: ephqoda11orp
       </ErrorDetails>
      </Response>
    </Convert>
  </ConvertToRAC></RConfig>
 
  Please Note : the above faile das there was no ssh passwordless connectivity between the standby nodes , hence used ssh-keygen utility to setup the connectivity and worked fine below
 
 
/u01/app/oracle/product/12.2.0.1/dbhome_1/assistants/rconfig/sampleXMLs=> rconfig ConvertToRAC_AdminManaged_odatdg.xml
<?xml version="1.0" ?>
<RConfig version="1.1" >
<ConvertToRAC>
    <Convert>
      <Response>
        <Result code="0" >
          Operation Succeeded
        </Result>
      </Response>
      <ReturnValue type="object">
There is no return value for this step     </ReturnValue>
    </Convert>
  </ConvertToRAC></RConfig>
 
 
Now change the convert verify from ONLY  to YES and complete the conversion, by running the rconfig command again.


/u01/app/oracle/product/12.2.0.1/dbhome_1/assistants/rconfig/sampleXMLs=> vi ConvertToRAC_AdminManaged_odatdg.xml
/u01/app/oracle/product/12.2.0.1/dbhome_1/assistants/rconfig/sampleXMLs=> rconfig ConvertToRAC_AdminManaged_odatdg.xml
Converting Database "odatdg.netr.ecis.police.uk" to Cluster Database. Target Oracle Home: /u01/app/oracle/product/12.2.0.1/dbhome_1. Database Role: PHYSICAL STANDBY.
Setting Data Files and Control Files
Adding Trace files
Adding Database Instances
Adding Redo Logs
Setting TEMP tablespace
Assigning UNDO tablespaces to instances
Setting Fast Recovery Area
Updating Oratab
Creating Password file(s)
Configuring related CRS resources
Starting Cluster Database
<?xml version="1.0" ?>
<RConfig version="1.1" >
<ConvertToRAC>
    <Convert>
      <Response>
        <Result code="0" >
          Operation Succeeded
        </Result>
      </Response>
      <ReturnValue type="object">
<Oracle_Home>
         /u01/app/oracle/product/12.2.0.1/dbhome_1
       </Oracle_Home>
       <Database type="ADMIN_MANAGED"  >
         <InstanceList>
           <Instance SID="odatdg1" Node="ephqoda10orp"  >
           </Instance>
           <Instance SID="odatdg2" Node="ephqoda11orp"  >
           </Instance>
         </InstanceList>
       </Database>     </ReturnValue>
    </Convert>
  </ConvertToRAC></RConfig>
/u01/app/oracle/product/12.2.0.1/dbhome_1/assistants/rconfig/sampleXMLs=> srvctl status database -d odatdg
Instance odatdg1 is running on node ephqoda10orp
Instance odatdg2 is running on node ephqoda11orp
/u01/app/oracle/product/12.2.0.1/dbhome_1/assistants/rconfig/sampleXMLs=> srvctl config database -d odatdg
Database unique name: odatdg
Database name: odatest
Oracle home: /u01/app/oracle/product/12.2.0.1/dbhome_1
Oracle user: oracle
Spfile: +DATA/odatdg/spfileodatdg.ora
Password file: +DATA/odatdg/orapwodatdg
Domain: netr.ecis.police.uk
Start options: mount
Stop options: immediate
Database role: PHYSICAL_STANDBY
Management policy: AUTOMATIC
Server pools:
Disk Groups: DATA,REDO
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group: dbaoper
Database instances: odatdg1,odatdg2
Configured nodes: ephqoda10orp,ephqoda11orp
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services:
Database is administrator managed

Check the status of the database.

Validate the configuration of the new standby RAC database.

Add the enteries to both of the /etc/oratab files.

Delete the old odatdg directories on node 0 (ephqoda10orp)- at one place under audit

Reconfigure the RAC Database Services for Data Guard

Now that the odatdg database has been converted to a RAC database data guard will no longer know about the services, so the primary database will need to be re-directed to the new RAC standby database.

As we can now see the service ODATDG is now status UNKNOWN, so will we need to change the service in all our TNSNAMES.ora files to the full service name "odatdg.<domain-name>"

This change should clear the ORA-01034 error and set the LNS process from OPENING to WRITING.

Now on the Standby database nodes put the database into recover managed standby mode to clear the waiting log files.

recover managed standby database using current logfile disconnect;

Comments

Popular posts from this blog

SQL SERVER – Event ID 107- Report Server Windows Service (MSSQLSERVER) cannot connect to the report server database.

SQL Server Builds Information

Using DBCA silent install and disabling automatic memory management