I am going to start this article with a confession. As a system integration and automation consultant I am continuously shifting from one product, language and platform to another. The frequent changes strengthen the short-term memory and change the long-term memory to manage concepts more than details. A complex set of steps, processes or systems commands that can be performed without having to refer to Google, Evernote, or vendor documentation today can often fade to nothing more than a clear memory of having done them without recall of the specifics within a few months. Sometimes in mere weeks (or even days) depending on how different the next project is.  So the main reason I write these detailed articles is so that I can repeat a process if ever needed again.  Being able to share these mental breadcrumbs is just a bonus.

Lately I have been working a great deal with Informatica Cloud, an excellent iPaaS solution. Informatica Cloud uses an on-premises application called a Secure Agent to facilitate secure interactions between internal systems and the Cloud. During training and development it is useful to have a Secure Agent running on a developer’s laptop. While the Secure Agent will install on the operating systems of most laptops, it runs better in an environment that is configured to support server operations rather than day-to-day work. There is also a simple way to switch the Informatica Cloud instance (known as an Org), though it can be a hassle for a developer or consultant that needs to interact with multiple Orgs. To make life easier in such circumstances, I have taken a baseline VirtualBox image and added some applications and scripts so that the image can be quickly configured for any Org I have access to. Following are the steps I took to build this image, plus instructions for running the scripts that will make it reusable.

Customize a Linux Image

If you aren’t familiar with VirtualBox, you can download it for free at https://www.virtualbox.org/wiki/Downloads.  Be sure to get the Guest Additions download, too, you will want it later. I don’t have a preference for a specific Linux distribution. This article uses a CentOS 7.0 image available from https://s3-eu-west-1.amazonaws.com/virtualboxes.org/CentOS7-Gnome.ova.torrent, a choice resulting from the first image found that was of recent vintage and provided sufficient support for the task (it also has a graphical desktop, which is useful when you want to share the image with people who may not be comfortable with command-line-only interfaces). That said, support for various installation processes vary widely between distributions and builds, so if you choose a different distro you may need to find alternative methods to some of the steps on your own.

The first step is to import the pre-built appliance you have downloaded.  The screenshots below will help if you have never done this before.

I change the Name of the appliance to be descriptive of the purpose of the image after I have completed configuring it. I also increase the CPU count and RAM total so that it will not be too slow during development test runs. You may need to adjust these values according to your own machine. I recommend no more than half of your available resources.

One reason I use this particular base image is that it has adequate hard drive space, which many lack.  If you have an appliance image you like but the drive space is too small, see my blog post about Resizing the Root Drive on a Linux VirtualBox Guest Image with a Windows Host for a solution to use it anyway.

One value of a VM is the ability to use Snapshots to go back to a working point.  During the process we will create one to make a reusable appliance.  Create others as you go according to your own sense of adventure or caution. I suggest you delete old snapshots when you know your current state is stable to save disk space. Doing so at the start saves re-importing the appliance if there is a mis-step earlier in the process.

Install the Informatica Cloud Secure Agent

The next thing to do is to install the Secure Agent. If you plan to re-use this image as a starting point, do not initialize the agent after installation, just start it and verify the install as follows (screen shots follow the steps for those unfamiliar with them):

Download the agent from Informatica Cloud

then:

[centos@localhost ics]$ cd ~/Downloads
[centos@localhost Downloads]$ chmod +x agent64_install.bin
[centos@localhost Downloads]$ ./agent64_install.bin

Choosing the default install location is the simplest option.

[centos@localhost Downloads]$ cd ~/infaagent
[centos@localhost infaagent]$ ./infaagent startup
[centos@localhost infaagent]$ cd main/agentcore
[centos@localhost agentcore]$ ./consoleAgentManager.sh getstatus

Finally, add the following to ~/.bash_profile:

cd ~/infaagent/
./infaagent startup

Planning for Reuse

For this to be reusable, we will want to change the hostname, especially for a shared dev environment as we cannot have everyone using “localhost” as their hostname. The following is a bit of an overkill script that handles setting the hostname for a variety of distributions:

changehostname.sh:

#!/usr/bin/env bash

OLD_HOSTNAME="$( hostname )"
NEW_HOSTNAME="$1"

if [ -z "$NEW_HOSTNAME" ]; then
 echo -n "Please enter new hostname: "
 read NEW_HOSTNAME < /dev/tty
fi

if [ -z "$NEW_HOSTNAME" ]; then
 echo "Error: no hostname entered. Exiting."
 exit 1
fi

hostnamectl set-hostname "$NEW_HOSTNAME"
hostnamectl set-hostname "$NEW_HOSTNAME" --pretty
hostnamectl set-hostname "$NEW_HOSTNAME" --static
hostnamectl set-hostname "$NEW_HOSTNAME" --transient

echo "Changing hostname from $OLD_HOSTNAME to $NEW_HOSTNAME..."

hostname "$NEW_HOSTNAME"

if [ -n "$( grep "$OLD_HOSTNAME" /etc/sysconfig/network )" ]; then
 sed -i "s/HOSTNAME=.*/HOSTNAME=$NEW_HOSTNAME/g" /etc/sysconfig/network
else
 echo -e "HOSTNAME=$NEW_HOSTNAME" >> /etc/sysconfig/network
fi

if [ -n "$( grep "$OLD_HOSTNAME" /etc/hosts )" ]; then
 sed -i "s/$OLD_HOSTNAME/$NEW_HOSTNAME/g" /etc/hosts
else
 echo -e "$( hostname -I | awk '{ print $1 }' )\t$NEW_HOSTNAME" >> /etc/hosts
fi

echo "Re-boot for change to reflect correctly."

This is also a good time to place a readme on the desktop. Here is the one I use:

###### One time set up #######
Open a terminal and execute the folloiwng commands (excluding [examples])

cd ~/infaagent
sudo su
./changehostname.sh PL_USER_ID

[example:./changehostname.sh scott.nelson]

exit

Restart the VM

cd ~/infaagent/main/agentcore
./consoleAgentManager.sh configure USERNAME PASSWORD

[example:./consoleAgentManager.sh configure linux.virtualbox@primitivelogic.com pa55w0rd]


[LEAVE TERMINAL WINDOW OPEN]

Log in to Informatica Cloud with the same credentials registered above, go to Configure > Runtime Environments

Confirm that the PL_USER_ID agent has been created
wait for upgrade to complete (at least 10 minutes)
return to terminal and run:

reboot

Log in and should be good to go.

#################################
If you have questions: theitsolutionist@gmail.com

Useful commands:

cd ~/infaagent
./infaagent shutdown?
./infaagent startup?

cd ~/infaagent/main/agentcore
./consoleAgentManager.sh getstatus

Save the script in ~/infaagent, shut down the virtual machine and create a snapshot. Then start the machine and follow the readme. If all works well, restore the last snapshot and then export the appliance as your base image.

Add Guest Additions

I set up a shared folder to easily exchange files between the guest (Virtual Machine) and host (laptop), as well as to have a place to uses as a file location accessible to both while working with Informatica Cloud flat file connections and logs.

In order to access the shared folder (as well as have a shared clipboard), run the Guest Additions set up after logging in to the new VM.

CentOS7 is not all that easy to do that with. There is a good set of instructions at http://lifeofageekadmin.com/how-to-install-virtualbox-5-additions-on-centos-7/. One thing that is not clear in the instructions is that you will want to shut down and re-start the VM after the last yum update before running the Guest Additions installation. You may also have to run it twice to get everything working.

You will probably need to use Right-CTRL+C to release your mouse from the VM until the Guest Additions are installed.

Once the shared folder shows up under /media, add the centos user to the group for access:

sudo usermod -a -G vboxsf centos

Restart for change to take effect.

Installing MySQL

I often find that MySQL is handy to have on the VM. The following steps will work to get it installed on the CentOS 7.0 image available from https://s3-eu-west-1.amazonaws.com/virtualboxes.org/CentOS7-Gnome.ova.torrent.

First, remove the old install with

sudo yum -y remove mariadb-libs

Then download the latest install from https://dev.mysql.com/downloads/repo/yum/.

The Red Hat Enterprise Linux 7 version seems to work on Centos 7. Save the download rather than opening it, then go to your downloads folder and run the following commands:

sudo yum -y localinstall platform-and-version-specific-package-name.rpm
sudo yum -y install mysql-community-server
sudo service mysqld start

When done, verify with

sudo service mysqld status

Finally, find and update the database admin password and (optionally) install Workbench using:

grep 'temporary password' /var/log/mysqld.log
...
mysql_secure_installation
...
sudo yum -y install mysql-workbench-community

Be sure to add the new password to your readme.

© Scott S. Nelson

I have always struggled with setting up a fresh Oracle 11g Database install. I have made copious notes over the years on the steps to take each time I tackle the process, and when I go back to the notes they only serves as clues to the mystery. This time I had to do the whole thing on a Linux both without the use of the GUI installers and I have captured and scripted the process once and for all.

Of course, most new installs these days will be 12c and should be performed by someone with deep database skills. This article is intended for those that simply need a database for a proof-of-concept or development environment where 11g will suffice.

Acknowledgements

Before I get into the details, I want to point out that there are three key blog posts that I found most useful in this process. They are:

• How I Simplified Oracle Database 12c and 11g Installations on Oracle Linux 6 (by Ginny Henningsen; updated by Michele Casey)
• Install Oracle in silent mode 11G Release 2 (11.2) on OEL6 (by Tomasz Lesinski)
• a simple script to automate the oracle 12c setup (by Daniel Westermann)
• How I Enable Autostarting of Oracle Database for Demonstrations and Development (by Christopher Jones)

Step-By-Step

These steps are based on using the scripts in this article. Feel free to adjust to fit your own needs, which is how this process came about.

As a side note, while putting the process together and again while documenting it, I used a VM and took a snapshot at the completion of each step to save time if some issue arose in a particular step. You will probably benefit from this as well, since small differences can prevent fully using these steps exactly as described.

Downloads

The Database installs can be found at http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html. If you have access to MOS, you can get a slightly newer version, and you will only need to first two files.

Environment Variables

The values need to be set in the environment. You can place them in a bash script under /etc/profile.d for all users, or in the oracle user home directory if necessary. In the /etc/profile.d folder, be sure to chmod 644 to ensure it will run properly.

export DB_USER=oracle
export DB_PASS=welcome1
export DB_USER_HOME=/home/${DB_USER}
export ORACLE_HOSTNAME=${HOSTNAME}
export ORACLE_UNQNAME=MY11GDB
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export ORACLE_SID=MY11GDB
export LD_LIBRARY_PATH=${ORACLE_HOME}/lib:/lib:/usr/lib;
export TMP=/tmp
export APP_DIR=/u01/app
export PATH=/usr/sbin:/sbin:/bin:/usr/sbin:/usr/bin:${ORACLE_HOME}/bin
export CLASSPATH=${ORACLE_HOME}/jlib:${ORACLE_HOME}/rdbms/jlib;
export SOFTWARE_HOME=/media/sf_software
export SCRIPT_HOME=/media/sf_scripts

Regardless of where you place these values, be sure they are set before continuing and placed where they will always be set when administering to the database in the future. This may require either sourcing the file, or rebooting.

Script Variables

db_installprep.sh will set all variables except for the name of the install zip files as those can change over time. Update as necessary in install_11gDB.sh.

Install Prep Script

The script below performs the following tasks:

Creates User and Groups

The _create_groups function takes an inelegant approach to create the oracle user and groups. For those of you better versed in Linux administration, feel free to create your own script. Please share if you do.

Creates ORACLE_HOME path and Update SELinux

For reasons I can’t fathom, the installer will not create the ORACLE_HOME path, so it must be created before installing. Once installed, SELinux enforcement can cause problems with getting the listeners running. Since I only use the database behind a firewall, turning it off works best for me. I would not take that approach with a production server, though.

Dependency Installs

As noted in the Acknowledgements, there is a useful script for making sure the dependencies are in place.

Set Response File Variables

If you use the response files provided in the Downloads section, this will update the values to match your Environment Variables. If you are using your own or from a different source, comment out the _create_rsp_files function.

db_installprep.sh

#!/bin/bash
# Script based on instructions at http://dbaora.com/install-oracle-in-silent-mode-11g-release-2-11-2/
export INSTALL_DATE=`/bin/date "+%Y%m%d-%H-%M"`

function_logcomment()
{
 echo "#########################################################################################"
 echo -e "$(date +"%I:%M:%S"): $1"
 echo "#########################################################################################"
}

_check_user() {
# "Borrowed from" https://danielwestermann.com/2013/08/11/a-simple-script-to-automate-the-oracle-12c-setup/
    if [ $(id -un) != "${1}" ]; then
        _log "you must run this as ${1}"
        exit 0
    fi
}

_create_groups(){
	groupNames=( oinstall dba oper asmdba asmoper asmadmin )
	groupIds=( 54321 54322 54323 54327 54328 54329 )
	
	for index in "${!groupNames[@]}"
	do
		function_logcomment "adding group ${groupNames[index]}"
		groupadd ${groupNames[index]} -f -g ${groupIds[index]}
	done
#NOT THE MOST ELEGANT APPROACH...
	echo "Check for $DB_USER user"
	if [ ! id -u $DB_USER > /dev/null 2>&1 ]; then
	echo "Adding $DB_USER user"
		useradd -u 500 $DB_USER
		echo $DB_PASS | passwd $DB_USER --stdin
	fi

	for index in "${!groupNames[@]}"
	do
		echo "Adding $DB_USER user ${groupNames[index]} group"
		usermod -aG ${groupNames[index]} $DB_USER 
	done
}

_pre_env_settings() {
	mkdir -p $ORACLE_HOME
	chown oracle:oinstall -R /u01
	sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config
}

_package_updates(){
	cd /etc/yum.repos.d
#	cp $SCRIPT_HOME/build/database/public-yum-ol6.repo .
	wget http://public-yum.oracle.com/public-yum-ol6.repo
	wait $!
	yum -y install oracle-rdbms-server-11gR2-preinstall
}

_create_rsp_files(){
	cd $SCRIPT_HOME/build/database
	filenames=( db_install.rsp dbca.rsp )

	for index in "${!filenames[@]}"
	do
		sed 's#@ORACLE_BASE@#'"$ORACLE_BASE"'#g ; s#@ORACLE_HOME@#'"$ORACLE_HOME"'#g ; s#@ORACLE_SID@#'"$ORACLE_SID"'#g ; s#@DB_PASS@#'"$DB_PASS"'#g' ${filenames[index]}.template > ${filenames[index]}
	done
}

_check_user "root"
wait $!
_create_groups 2>&1 | tee ${APP_DIR}/db11g_install.log
wait $!
_pre_env_settings 2>&1 | tee ${APP_DIR}/db11g_install.log
wait $!
_create_rsp_files
wait $!
_package_updates 2>&1 | tee ${APP_DIR}/db11g_install.log
echo "DB Install Prep Complete"
#wait $!
#https://danielwestermann.com/2013/08/11/a-simple-script-to-automate-the-oracle-12c-setup/
#suggests the following firewall changes but have been successful so far without
#systemctl stop firewalld
#systemctl disable firewalld

Install the Database

As noted earlier, be sure that the install zip file names are correct in install_11gDB.sh. install_11gDB.sh will perform the following:

1. Unzip the install files
2. Perform a silent install of the database application
3. Create a default listener
4. Create a default database

install_11gDB.sh

#!/bin/bash
# Script based on instructions at http://dbaora.com/install-oracle-in-silent-mode-11g-release-2-11-2/
export INSTALL_DATE=`/bin/date "+%Y%m%d-%H-%M"`

function_logcomment()
{
 echo "#########################################################################################"
 echo -e "$(date +"%I:%M:%S"): $1"
 echo "#########################################################################################"
}

# check for the current os user
# "Borrowed from" https://danielwestermann.com/2013/08/11/a-simple-script-to-automate-the-oracle-12c-setup/
_check_user() {
    if [ $(id -un) != "${1}" ]; then
        _log "you must run this as ${1}"
        exit 0
    fi
}

_unzip_installs(){
	cd $DB_USER_HOME
	unzip -q $SOFTWARE_HOME/p13390677_112040_Linux-x86-64_1of7.zip
	unzip -q $SOFTWARE_HOME/p13390677_112040_Linux-x86-64_2of7.zip
}

_install_binaries(){
	cd $DB_USER_HOME/database
	./runInstaller -silent -responseFile $SCRIPT_HOME/build/database/db_install.rsp
}

_create_db(){
	mkdir $ORACLE_BASE/oradata
	mkdir $ORACLE_BASE/flash_recovery_area
}
_check_user $DB_USER
source ${DB_USER_HOME}/.bash_profile
_unzip_installs 2>&1 | tee ${APP_DIR}/db11g_install.log
wait $!
_install_binaries 2>&1 | tee ${APP_DIR}/db11g_install.log
wait $!
netca -silent -responseFile $SCRIPT_HOME/build/database/netca.rsp 2>&1 | tee ${APP_DIR}/db11g_install.log
wait $!
dbca -silent -responseFile $SCRIPT_HOME/build/database/dbca.rsp 2>&1 | tee ${APP_DIR}/db11g_install.log

Post Install Steps

After the installation, there are two scripts that are provided by Oracle that need to be run as root. Then there are some steps to perform so that the database is started every time the server is, because I have not had a case where this wasn’t what I needed. As noted in the Acknowledgements, How I Enable Autostarting of Oracle Database for Demonstrations and Development (by Christopher Jones) was my reference for this process and taken almost completely from that blog entry.

Once the above is completed, the install files are no longer necessary, so they are removed.

db_post_install.sh

#!/bin/bash

#script Distilled from https://blogs.oracle.com/opal/entry/how_i_enable_autostarting_of1

_check_user() {
# "Borrowed from" https://danielwestermann.com/2013/08/11/a-simple-script-to-automate-the-oracle-12c-setup/
    if [ $(id -un) != "${1}" ]; then
        echo "You must run this as ${1}!"
        exit 0
    fi
}
_update_oratab(){
	sed -i 's#'"$ORACLE_HOME"':N#'"$ORACLE_HOME"':Y#g' /etc/oratab 2>&1 | tee ${APP_DIR}/db11g_install.log
	#if above fails...
	grep -q -F ${ORACLE_SID}:${ORACLE_HOME}:Y /etc/oratab || echo ${ORACLE_SID}:${ORACLE_HOME}:Y >> /etc/oratab
}
_create_dbora(){
	cp $SCRIPT_HOME/build/database/dbora /etc/init.d/dbora
	sed -i 's#@ORACLE_HOME@#'"${ORACLE_HOME}"'#g' /etc/init.d/dbora
}
_update_dbora(){
	chgrp dba /etc/init.d/dbora
	chmod 750 /etc/init.d/dbora
	chkconfig --add /etc/init.d/dbora
	chkconfig dbora on
}
_check_user "root"
${ORACLE_BASE}/oraInventory/orainstRoot.sh
wait $!
${ORACLE_BASE}/product/11.2.0/db_1/root.sh
wait $!
_update_oratab
wait $!
_create_dbora
wait $!
_update_dbora
wait $!
rm -rf $DB_USER_HOME/database
echo "db_post_install.sh complete"

Verify the Installation

After running db_post_install.sh, you will either need to log out and log back in again as oracle, or simply reboot the server. First test with lsnrctl status, which should yield the following:

On occasion, this may not work properly. Before sending me an email reminding me of my imperfections, you may first want to try:

sqlplus / as sysdba

startup

exit

And then either log out and back in or (preferably) reboot the server and try again. This generally fixed it for me while coming up with these steps.

Downloads

The scripts described here are available at https://drive.google.com/folderview?id=0B1L4vdibbjSXeUJ5Q0pURExfbXc&usp=sharing

One More Thing

I found installing sqldeveloper to be a great way to do more through testing. At the time of this writing it can be found at http://www.oracle.com/technetwork/developer-tools/sql-developer/downloads/index.html.

© Scott S. Nelson