#################################################################### ## Copyright(c) Oracle Corporation 2016. All rights reserved. ## ## ## ## Specify values for the variables listed below to customize ## ## your installation. ## ## ## ## Each variable is associated with a comment. The comment ## ## can help to populate the variables with the appropriate ## ## values. ## ## ## ## IMPORTANT NOTE: This file should be secured to have read ## ## permission only by the oracle user or an administrator who ## ## own this installation to protect any sensitive input values. ## ## ## #################################################################### #------------------------------------------------------------------------------- # Do not change the following system generated value. #------------------------------------------------------------------------------- oracle.install.responseFileVersion=/oracle/install/rspfmt_ogginstall_response_schema_v12_1_2 ################################################################################ ## ## ## Oracle GoldenGate installation option and details ## ## ## ################################################################################ #------------------------------------------------------------------------------- # Specify the installation option. # Specify ORA12c for installing Oracle GoldenGate for Oracle Database 12c and # ORA11g for installing Oracle GoldenGate for Oracle Database 11g #------------------------------------------------------------------------------- INSTALL_OPTION=ORA12c #------------------------------------------------------------------------------- # Specify a location to install Oracle GoldenGate #------------------------------------------------------------------------------- SOFTWARE_LOCATION=/u01/app/oracle/product/gg123 #------------------------------------------------------------------------------- # Specify true to start the manager after installation. #------------------------------------------------------------------------------- START_MANAGER=FALSE #------------------------------------------------------------------------------- # Specify a free port within the valid range for the manager process. # Required only if START_MANAGER is true. #------------------------------------------------------------------------------- # MANAGER_PORT=8900 # MCulp is using the following value for test puposes MANAGER_PORT=8908 #------------------------------------------------------------------------------- # Specify the location of the Oracle Database. # Required only if START_MANAGER is true. #------------------------------------------------------------------------------- DATABASE_LOCATION= ################################################################################ ## ## ## Specify details to Create inventory for Oracle installs ## ## Required only for the first Oracle product install on a system. ## ## ## ################################################################################ #------------------------------------------------------------------------------- # Specify the location which holds the install inventory files. # This is an optional parameter if installing on # Windows based Operating System. #------------------------------------------------------------------------------- INVENTORY_LOCATION=/u01/app/oraInventory #------------------------------------------------------------------------------- # Unix group to be set for the inventory directory. # This parameter is not applicable if installing on # Windows based Operating System. #------------------------------------------------------------------------------- UNIX_GROUP_NAME=oinstall
Big Data adapters with GoldenGate
Download for Big Data Adapters
Using a environment file can help setup GoldenGate properly Here I'm using an environment file to setup various variables I include this as a function to the overall library gg_common.ksh
gg_cr_env()
###############################################################################
# Create the ggoraenv.env file
###############################################################################
{
echo "...Creating ggoraenv.env...."
# Creating the file
# echo > ggoraenv.env
cat << EOS >> ggoraenv.env
# export ORACLE_HOME=/oracle/product/11.2.0.4/db_11
# export ORACLE_SID=MRCXXD011
# export LD_LIBRARY_PATH=/oracle/product/11.2.0.4/db_11u/lib
# export GG_HOME=/oracle/product/gg12.1
export GGS_HOME=/u01/app/oracle/product/12.2/gg_2
export GGS_HOME=$gg_dir
export GG_SITE=01
export GG_LOC=TX
# if there is only one DB related to this GG home,
# set NEW_ORACLE_SID to avoid constant switch between a DB env and its GG env
# export NEW_ORACLE_SID=PIPERD011; . ~/.std_profile
# otherwise, set NEW_ORACLE_SID to dummy if there are multiple replicated databases related to same GG home
#export NEW_ORACLE_SID=dummy; . ~/.std_profile
# uncomment next three lines if ORACLE_SID is dummy
#export ORACLE_HOME=/oracle/product/11.2.0/db_1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib/$JAVA_HOME/lib/amd64/server:$LD_LIBRARY_PATH
export PATH=$ORACLE_HOME/bin:$PATH
export PATH=$GGS_HOME:$PATH
export LD_LIBRARY_PATH=$GGS_HOME:$LD_LIBRARY_PATH
# for OEM12c GG monitoring
# export JAVA_HOME=/oracle/product/12.1.0/oem_1/agent/core/12.1.0.2.0/jdk/jre
# export PATH=$JAVA_HOME/bin:$PATH
# export LD_LIBRARY_PATH=$JAVA_HOME/lib/amd64/server:$LD_LIBRARY_PATH
# Set and export the timezone
TZ=UTC
export TZ
#############################################
## For GI Agent (XAG)
##############################################
# export XAG_HOME=/oracle/product/xag71
# export PATH=$XAG_HOME/bin:$PATH
# Change the prompt
PS1="\\
\${PWD} \\
\${SNAME} [\${ORACLE_SID}] [GG_site$GG_SITE"_"$GG_LOC]-> "
export PS1
# alias ggsora12='. $HOME/.ggsora12_env;cd $GGS_HOME'
alias ggsora12='. /ora_rman/ora013/mrc/gg/ggoraenv.env;cd $GGS_HOME'
# alias xag='. $HOME/.ggsora12_env;cd $XAG_HOME'
# alias ggstatus='$XAG_HOME/bin/agctl status goldengate piperd01_oggapp'
# Show the home afterward
echo "GGS_HOME is : " $GGS_HOME
EOS
}
#######################################################
# GG These are all GoldenGate functions
#######################################################
#######################################################
# GG functions
# All of the GG functions should go here
#
# gg_init - Set some variables for other functions
# gg_init_dbrd - GoldenGate init dashboard
# gg_unzip - Unzip the downloads for GG binaries only set
# variables in gg_init first
# gg_zipfile_chk - Check for directory for distribution set variables
# in gg_init first
# gg_inst -
# cr_gg_env - Create the GoldenGate environment files
# this should take into account the existing stuff
# already established by .profile_env
# gg_chk_files -
# gg_unzip -
# xag_unzip -
# xag_inst_dir -
# xag_install -
# chk_xag -
# file_chk_ggzip -
# gg_chk_envfile -
# dir_chk_gen -
# dir_chk_ggstr -
# dir_chk_ggexe -
# gg_chk_usr -
# qq_qa_chk -
# add_gg_vip -
# chk_vip -
# chk_gg_fs -
# ggcr_file01_cr -
# mkdir_dbfs -
# inst_fuse -
#######################################################
gg_init_dbrd()
{
echo
echo " Change variables in gg_common.ksh gg_init...."
echo " display with gg_init_dbrd...................."
echo
echo "....GoldenGate Version...: " $ggver
echo "....GoldenGate Filename..: " $ggflnm
echo "....GoldenGate Directory.: " $ggdir
echo "....GoldenGate Home......: " $GGS_HOME
echo "....XAG Version..........: " $xagver
echo "....XAG directory........: " $xagdir
}
gg_init()
######################################################
# These variables should be set prior to setup
######################################################
{
# GoldenGate version
ggver="12.3"
# OGG_HOME=$GGS_HOME
# export DBFS_MNT=/dbfs
# This is the name of the goldengate zip file
ggflnm="123010_fbo_ggs_Linux_x64_shiphome.zip"
# This is the name of the directory GoldenGate software is stored
ggdir="/ora01/software/gg123/"
# This is the XAG version
xagver="8.1b"
# This is the name of the directory where the XAG software is stored
xagdir="/ora01/software/xag81b/"
# This is the name of the xag zip file
xagflnm="xagpack81b.zip"
# Read the GG environment files
. ./ggoraenv.env
# Show the variable status
gg_init_dbrd
}
The Integrated Capture GoldenGate Mode also known as the Integrated Extract Goldengate process in 12c (also backported to 11gr2) is one of the more interesting and useful feature released with this version. This capture process is the component responsible for extracting the DML transactional data and DDL’s from the source database redo log files. This data is then written to local trail files which eventually is move to the destination database to be applied there.
Here are the list of topics to be covered in this article.
• What is the GoldenGate Integrated Capture Mode?
• Integrated Extract Goldengate vs Classic Capture
• On-Source Capture
• Downstream Capture
• Prerequisites
• Configuration
• Monitoring/Views
What is the GoldenGate Integrated Capture Mode?
Integrated Capture Mode (IC) is a new form of the Extract process, were in this process is moved closer, inside the source database. In the traditional Classic extract process, the extract works on the redo logs outside the domain of the actual database. In this new integrated capture mode, a server Log Miner process is started which extracts all the DML data and DDLS statements creating Logical Change Records (LCR’s). These are then handed to the Goldengate memory processes which writes these LCR’s to the local trail files. This Log Miner server process is not the Log Miner utility we are used to in the database but is a similar mechanism which has been tuned and enhanced for specific use by the Goldengate processes.
The purpose of moving this inside the database is to be able to make use of the already existing internal procedures in the database, making it easier to provide support for the newer features of Oracle faster than was previously possible. Due to this change, Oracle is now able to provide the following.
• Full Support of Basic, OLTP and EHCC compressed data.
• No need to fetch LOB’s from tables.
• Full Secure File support for Secure file lobs.
• Full XML support.
• Automatically handles addition of nodes and threads in RAC environment.
• Senses node up down in RAC and handles it in its processes transparently.
Integrated Capture vs Classic Capture
The Integrated Capture mode offers the following.
• Integrated with Database features
• Allows to mine earlier versions of integrated capture on secondary
• More efficient. It does not have to fetch data because of the datatype, etc..
• No longer necessary to set this: Threads, ASMUSER, ASMBUF, DBLOGREADER, DECRYPASSWORD
• For RAC no additional manual steps required. Transparent with RAC.
Integrated Capture Modes
Integration capture supports two types of deployment configurations. They are:
• On-Source Capture
• Downstream Capture
On-Source Capture
When the integrated capture process is configured using the on-source capture mode, the capture process is started on the actual source database server itself. Changes as they happen on source database will be captured locally, routed, transformed and applied on target database in near real-time.
This may seem convenient but consideration needs to be given to the additional workload that will be placed by this process on the database server. However if real-time replication is required this is the best option.
Note: All features are supported in both On-Source or Downstream Deployment
Downstream Capture
In the downstream mode, the capture process is configured to run on a remote database host. All the database redo logs from the source database are shipped to this remote server using Dataguard technology and then mined there by the capture process.
In this mode there is an inherent latency introduced due to the fact that the redo log on the source needs to switch first before the log can be shipped downstream. So there will be some delay in the replication of data to a target database as the extraction will be delayed due to the log switch. The main benefit of this setup however is the offset of the resource usage on the source server.
In this mode, to overcome the log switch latency, Oracle has provide a near Real time capture using Standby redo logs for extraction. In this configuration the redo log from the source continuously writes into the standby redo logs of the downstream database. The capture process directly capture the data from here.
It is important to keep in mind when deciding whether to use the Integrated capture or the classic capture mechanism that both configuration will remain available in future releases. However Oracle recommends to use the new Integrated capture mechanism as Oracle will not be adding new features to classic capture in the future and it will only be there for legacy support purposes.
Prerequisites
The database where integrated capture runs:
• Must be at least 11.2.0..3
• Database patch 1411356.1 must be installed.
• Works with Oracle 12c.
If the GG Hub server is a standalone environment then the first step is to install the most recent Oracle client on the GoldenGate hub server, choosing the administrator installation.
In a hub configuration, GoldenGate is installed only on the hub server, it is not installed on the database servers. The Oracle client is installed on the hub server. We will be using the thick client.
In a hub environment that is install in a fault tolerant configuration, (RAC, DBFS, and XAG w/DataGuard).
GGS_HOME needs to be set, make this the actual directory
export GGS_HOME=/oracle/product/12.2/gg_1
This will allow gh
alias gh='cd $GGS_HOME'
alias gd='cd /ggate'
export PATH=$GGS_HOME:$PATH
export LD_LIBRARY_PATH=$GGS_HOME:$LD_LIBRARY_PATH
GoldenGate Script Library gg_common.ksh
Adding Existing Tables to GG Replication