Modifying the VIP or VIP Hostname of a 10g or 11g Oracle Clusterware Node [ID 27

Applies to:

Oracle Server - Enterprise Edition - Version: 10.1.0.2 to 11.1.0.8
Information in this document applies to any platform.

Purpose

The purpose of this note is to illustrate the changing of a Virtual IP Address (VIP) or VIP hostname or other VIP parameters in an Oracle10g RAC/Oracle Clusterware environment.   Caution: This note can only be used to change the VIP IP address or Hostname or other parameters associated with the VIP. It is not meant to cover changing the public hostname or the public host IP address.

Scope and Application

Oracle 10g use a VIP (Virtual IP) for clients to connect to the database. This VIP is a static IP with a hostname defined and resolved thru DNS. During the installation of Oracle Clusterware users are prompted to enter a Virtual IP and Virtual hostname for each of the node in the cluster. These are stored within the OCR (Oracle Cluster Registry) and different components within 10g HA framework depend on these VIPs.   If for some reason the need arises  to change either the VIP or the VIP/hostname - perhpas due to a data center move, or changing IP subnets in a Data Center, this procedure should be followed.

Modifying the VIP or VIP Hostname of a 10g or 11g Oracle Clusterware Node

Planning for VIP changes

Changing the VIP involves modification of the nodeapps, which includes the Virtual IP address, the GSD, the Listener, and Oracle Notification Services (ONS).   The VIP can be modified while the nodeapps are running, however changes will not take effect until the VIP, and hence the nodeapps, are restarted.
Depending on the version of Oracle Clusterware that you are running, other resources on a node, such as database instances and ASM instances, are dependent on the VIP,  so stopping the nodeapps may cause other resources to be stopped - therefore, this change should be made during a scheduled outage.

In most cases, changing Data Centers or IP addresses within a Data Center will already incur an outage for other reasons, because changes need to be made at the operating system level, and servers may even need to be moved - so there is most likely a scheduled outage for this type of maintenence already.

However, in some cases - for example, if a VIP IP address was perhaps mistyped in DNS and needs to be corrected, you may be able to minimize the downtime by only modifying a single node.  Also, as of 10.2.0.3 and higher versions, the ASM instance and DB instances no longer depend on the VIP, so it is possible to stop the nodeapps without bringing down the ASM instance or DB instance - but client connectivity is limited while the VIP is offline.

Verifying Current VIP configuration

Below, we will provide examples on how to modify the nodeapps to change the VIP or VIP hostname.  


1. The first step that should be done is to confirm the current configuration of the VIP.  This is most easily accomplished by running the command:

 srvctl config nodeapps -n <nodename> -a

Using the '-a'  switch will give you the current VIP Hostname, VIP IP address and interface.   The following example is from a Windows System.   Note that the nodename is 'node1' as passed via the '-n'  argument.    

C:\>srvctl config nodeapps -n node1 -a
VIP exists.: /node1-v/10.148.44.94/255.255.255.0/Public

These outputs show that:

The VIP Hostname is 'node1-v'
The VIP IP address is '10.148.44.94'
The VIP subnet mask is '255.255.255.0'
The Interface Name used by the VIP is called 'Public'

Any of the above configuration parameters associated with the VIP can be changed if they were 4input incorrectly, or perhaps they need to be changed due to external reasons (such as a data center move, or IP change as mentioned above).

 

Note:
In addition, you can also verify the IP address and interface of the VIP by running the 'ifconfig' command or 'ifconfig -a', and looking for the IP address associated with an interface ending in ':1'. The example below, on Linux, shows the interface as 'eth0:1'

=============================
eth0:1 Link encap:Ethernet HWaddr 00:01:03:2C:69:BB
inet addr:192.168.1.125 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
...
...
=============================

On the Windows platform, run the command 'ipconfig /all'. This will list all IP addresses bound to a given adapter, including the VIP, but it will not be obvious just from this output which one is the VIP address and which is the actual host IP address.   Therefore, using 'srvctl config nodeapps -n <nodename> -a'  is the surest bet.

2. Once you have determined what the current configuration of the nodeapps is, be sure to make note of what values will remain unchanged, and what the correct/new values that you need will be.   For example, if you are just changing the "Interface Name"   you will need to know what the correct interface name is, but you also need to note the VIP hostname, IP address and subnet mask to ensure that this information is re-entered correctly.

Stopping Resources

3.  Once you are ready to make the change, stop all resources that are dependent on the VIP on a given node. This includes all RAC database instances on that node, as well as the ASM instance on that node, if it exists and is being used   (Note that starting with 10.2.0.3, the dependency between the VIP and the database and ASM instances has been removed):

3a). Stop database instances:
$ srvctl stop instance -d grid -i grid1

Where the database name is specified via the '-d' option, and the instance on the appropriate node is specified with the '-i' option.

Alternatively, to stop the entire database, on all nodes, you can issue the stop database command:

$ srvctl stop database -d grid

3b). To stop the ASM instance on the node, issue the following command:

$ srvctl stop asm -n node1

This command should be issued for each ASM instance using the appropriate node name.  Alternatively, it is also possible to stop these resources via SQL*Plus and/or, on the Windows platform by stopping the associated services.

4. Next, stop the nodeapps using srvctl - i.e:

$ srvctl stop nodeapps -n node1

This will stop the VIP, GSD, Listener, and ONS daemon currently running on the nodes specified

If this is being done as part of a data center move, then you will most likely be stopping these resources prior to moving the equipment.   In that case, you may want to disable the resources after stopping them, to prevent them from re-starting once the machines are brought back up and the Oracle Clusterware stack is started.   You can do this via commands such as:

$ srvctl -h | grep disable
Usage: srvctl disable database -d <name>
Usage: srvctl disable instance -d <name> -i "<inst_name_list>"
Usage: srvctl disable service -d <name> -s "<service_name_list>" [-i <inst_name>]
Usage: srvctl disable asm -n <node_name> [-i <inst_name>]

On 11gR1 and earlier, it is not possible to disable the nodeapps.

5. Verify that the VIP is no longer running by executing the 'ifconfig -a' or 'ipconfig /all' command again, and confirm that the IP address is no longer listed as running in the output.

If the interface still shows as online, this may be an indication that a resource which is dependent on the VIP is still running. The crs_stat command can help to show resources that are still online.

 

6. If the VIP IP Address or VIP Hostname are being changed, update the HOSTS file entries to reflect the updated information on all nodes - i.e. 

/etc/hosts files (on Unix/Linux)

or the

\WINDOWS\System32\drivers\etc\hosts file (on Windows)

and/or make the necessary DNS additions, to associate the new IP address with the old VIP hostname.

Note:  If changing one node at a time, be sure to only update the hosts/DNS entries for the VIP that is actually being changed.

 

Syntax for modifying nodeapps

7. To make the actual modification to the nodeapps, the Oracle Clusterware stack must be up on the node where you are running srvctl from.   To modify the nodeapps  use the 'srvctl modify nodeapps' command with the following syntax:

srvctl modify nodeapps -n <node_name> [-o <oracle_home>] [-A <new_vip_address>]

Options Description:
-n <node_name> Node name.
-o <oracle_home> Oracle home for the cluster software (CRS-Home).
-A <new_vip_address> The node level VIP address (<name|ip>/netmask[/if1[|if2|...]]).

As noted previously, any of the above parameters can be changed from their original values  (though it is unlikely that the ORACLE_HOME would change), provided that the match the expected characteristics. 

So - for example, be sure that the interface name specified is the correct name as seen from the OS (refer to Note:283684.1 ), be sure that the subnet mask used for the VIP matches the subnet mask used for the actual public IP addresses, and that the VIP hostname is correctly registered in DNS and/or the hosts file.   An example of the 'modify nodeapps' command is as follows:

$ srvctl modify nodeapps -n node1 -A 192.168.2.125/255.255.255.0/eth0

It should be noted that for the first parameter, you can specify either the hostname associated with the VIP, or the IP address associated with the VIP.    Either way, the srvctl command will actually attempt to resolve the IP to a hostname, or the hostname to an IP, and it will store both entries in the OCR.  So, assuming that the virtual hostname of 'node1-v'  resolves to an IP address 192.168.2.125, the below command would have the same effect as the command using the IP address:

$ srvctl modify nodeapps -n node1 -A  node1-v/255.255.255.0/eth0

Note that the interface names are case sensitive on all platforms.  On some platforms, such as Windows, the Interface Name may have spaces in it - i.e. "Local Area Connection 1".   If that is the case, you must enclose the interface name in double quotes - i.e.

srvctl modify nodeapps -n node1 -A 192.168.2.125/255.255.255.0/"Local Area Connection 1"

On Unix and Linux systems, this command should be run as root. Attempting to run this command
as the 'oracle' user or software owner will result in the following error:

PRKO-2117 : This command should be executed as the system privilege user.

On Windows systems, this command should be run as the user who did the original installation.   This account should be an account with Local Administrator privileges on each node.

8.  After making the change, you should verify that it is correct by re-running

'srvctl config nodeapps -n <nodename> -a'

Double-check the output to confirm that all parameters are correct. 

Starting Resources Back Up

9. Once the modify nodeapps has been executed, you can re-start node-level applications via srvctl with the following syntax:

srvctl start nodeapps -n <node_name>
i.e.:

$ srvctl start nodeapps -n rnode1

 

If any resources (such as database or ASM) were previously disabled, then they should now be re-enabled and re-started as well.

Repeat the same steps for all the nodes in the cluster.  Since SRVCTL is a cluster wide management tool, you can accomplish these tasks for any specific nodes from one node, without the need to login individually to each of the cluster nodes.

Note: If only the IP address is changed, it should not be necessary to make changes to the LISTENER.ORA and TNSNAMES.ORA, provided they are using the vip hostnames for
the 'HOST=' entries.

If changing both the hostname and the VIP for a node, it will be necessary to modify the LISTENER.ORA and change the 'HOST=' entries to the new VIP hostname. This can be done manually, or by using the NETCA to reconfigure the listener. In addition, changes may need to be made to the TNSNAMES.ORA of any clients connecting to the old HOSTNAME.

In addition to modifying the nodeapps after an IP address or network change of some type, it may also be necessary to modify the networks that are stored in the OCR.   If that is the case, please refer to the following note: