JBoss - Configuring Multiple JBoss Instances On One Machine

https://community.jboss.org/docs/DOC-9384

 

 

 

Configuring Multiple JBoss Instances On One Machine

版本 50  

创建于: 2004-10-8 下午9:02 作者 Elias Ross - 最后修改:  2010-7-30 下午12:55 作者 Vadim Kopichenko

Overview

 

Multiple instances of JBoss AS can run on a single machine assuming you have the necessary system resources (RAM, CPU, etc).  These instances can be clustered or can run completely independently of one another depending on your needs.

 

Why would you want to run multiple instances?

• 32-bit VM limitations : maybe you have a large box with LOTS of RAM that you cannot take advantage of because you are standardized on 32-bit (hardware, OS, VM) heap limitations

• isolation : maybe you need complete isolation of your apps because one app is unstable and could negatively impact the other applications

  • Keep in mind, that JBoss AS is perfectly capable of hosting MULTIPLE applications in a SINGLE instance!

• QA : maybe you would like a separate QA environment which is isolated from the development environment on the same box

• JBoss AS version dependencies : maybe you have multiple applications and some need version X of JBoss AS and some need version Y of JBoss AS

• JVM version dependencies : maybe one app requires JDK 1.4 and another app requires JDK 1.5.  Since JBoss AS can use either you can launch one instance using 1.4 and another instance using 1.5 (unclustered of course)

 

---

Separate filesystem trees for each JBoss AS instance

 

IMPORTANT: You should create a different SERVER CONFIGURATION DIRECTORY for each instance of JBoss AS you want to run.  For example, your directory tree should be :

 

• $JBOSS_HOME

  • server

    • minimal

    • default

    • all

    • node1

    • node2

    • node3

    • nodeX

 

And you then launch each node with the -c option of run.sh/run.bat  (e.g. run.bat/run.sh -c node1)

 

---

When I try to launch multiple instances I get port conflicts

You cannot have two daemons listening on the same IP address and same port for incoming requests.  This is not a limitation of JBoss or even of Java, this is true of any server including databases, web servers, application servers, etc.  To address this issue you have two choices:

 

1. Bind each JBoss AS instance to a different IP address

2. Ensure each instance is using a distinct port for each service.  This can be done manually (not recommended) or by using the Service Binding Manager service.

 

Using Multiple IP Addresses

 

If your machine has multiple IP addresses, the preferred solution is to use the -b command line option to bind each instance of JBoss AS to a particular IP address:

 

./run.sh -c node1 -b 192.168.0.10

 

A host name can be used in place of the IP address.

 

Multiple IP addresses can be added to your machine either by installing multiple NICs in the machine or by configuring virtual interfaces on a single NIC.

 

Beginning with EAP 4.3 or community AS 5.0, JBoss AS includes JBoss Messaging (JBM) as its JMS implementation. JBM requires that each server in the cluster have a unique integer id (whether the servers are running on the same physical machine or not).  This id can be set via the command line, so, to start two instances bound to different IP addresses you would:

 

./run.sh -c node1 -b 192.168.0.10

and

./run.sh -c node2 -b 192.168.0.11 -Djboss.messaging.ServerPeerID=1

 

On the node1 server, since we aren't setting jboss.messaging.ServerPeerID JBM uses the default value of 0.

 

Where possible, it is advised to use a different ip address for each instance of JBoss rather than changing the ports or using the Service Binding Manager for the following reasons:

 

• When you have a port conflict, it makes it very difficult to troubleshoot, given a large amount of ports and app servers.

• Too many ports makes firewall rules too difficult to maintain.

• Isolating the IP addresses gives you a guarantee that no other app server will be using the ports.

• Each upgrade requires that you go in and re set the binding manager again.  Most upgrades will upgrade the conf/jboss-service.xml file, which has the Service binding manager configuration in it.

• The configuration is much simpler.  When defining new ports(either through the Service Binding manager or by going in and changing all the ports in the configuration), it's always a headache trying to figure out which ports aren't taken already.  If you use a NIC per JBoss Instance, all you have to change is the Ip address binding argument when executing the run.sh or run.bat.  (-b <ip_address>)

• Once you get 3 or 4 applications using different ports, the chances really increase that you will step on another one of your applications ports.  It just gets more difficult to keep ports from conflicting.

• JGroups will pick random ports within a cluster to communicate.  Sometimes when clustering, if you are using the same ip address, two random ports may get picked in two different app servers(using the binding manager) that conflict.  You can configure around this, but it's better not to run into this situation at all.

• On a whole, having an individual IP addresses for each instance of an app server causes fewer problems (some of those problems are mentioned here, some aren't).

 

Providing alternate ports for each instance of JBoss AS

 

This is easier to do and does not require a sysadmin.  However, it is not the preferred approach for production systems for the reasons listed above.  This approach is usually used in development to try out clustering behavior.

 

For the 4.x series and earlier:

 

• If you installed JBoss from the ZIP file, modify conf/jboss-service.xml and uncomment the "Service Binding" section and select a "ServerName" value from sample-bindings.xml (e.g. ports-01 or ports-02 or make your own named port configuration)

  • NB.: If you are configuring an instance based on the "minimal" configuration, you have to copy the file $JBOSS_HOME/server/default/lib/bindingservice-plugin.jar into the "lib" directory of your configuration (e.g. $JBOSS_HOME/server/minimal/lib).

• If you installed JBoss using the installer, first, make sure you selected the "binding-service" package from the JMX group and then, open /conf/deploy/binding-service.xml file in the selected configuration and modify the "Service Binding" section as explained in the previous paragraph.

• See $JBOSS_HOME/docs/examples/binding-manager/sample-bindings.xml "ports-default" entries for all jboss port properties. default-bindings.xmlprovides a quick view of jboss-4.0.2 default port settings, but it may not reflect your installation's port settings.

• NOTE: JBoss AS 4.0.2 has a known bug : one of the ports (4446) is not included in the sample-bindings.xml file and therefore will continue to conflict even after selecting an alternate port configuration in your jboss-service.xml.  To fix this simply select an alternate port (something other than 4446 that is not being used) in cluster-service.xml configuration (obviously not necessary if you are not using the ALL configuration with clustering support).  This bug is being fixed in our next release.

 

Beginning with AS 5.0, it's simpler. The ServiceBindingManager is always enabled, so it's just a matter of telling it which set of bindings to use, which can be done via the command line.

 

For the first server, simply:

 

./run.sh -c node1

 

and for the second:

 

./run.sh -c node2 -Djboss.service.binding.set=ports-01 -Djboss.messaging.ServerPeerID=1

 

The jboss.service.binding.set system property tells the ServiceBindingManager what set of ports to use. If unset, the SBM uses "ports-default". See the discussion in "Using Multiple IP Addresses" above for the meaning of the jboss.messaging.ServerPeerID property.

 

---

When I simultaneously launch multiple instances configured for clustering I get startup errors.

 

Another problem people have results from using scripts to launch multiple JBoss instances.  If you intend to launch multiple JBoss instances on the same machine and have them form a cluster, it's common to write some kind of script to launch the two instances. It is a good practice to add some kind of pause in your script between the launch of the first instance and the second. 10 to 20 seconds is good.  Even if you launch manually, adding a delay is good practice.

 

This is because if both instances are launched simultaneously, they both may decide they are the JGroups coordinator. At this point, you will have two independent clusters of one node each. If this happens, both nodes may begin to start HASingleton services, such as HA-JMS. A few seconds later, the two nodes will discover each other and the two clusters will merge. One of the nodes will no longer be coordinator, and the HASingleton services will be stopped. Stopping a service that's in the middle of starting does not always go cleanly.

 

By putting a pause in your startup script, you give the first node a chance to become coordinator before the second node starts. The second node will then cleanly join the cluster, and no HASingleton services will be started on it.

附件：

• default-sample-bindings.xml (9.8 K)
• default-bindings.xml (9.8 K)
• sample-bindings.xml (33.8 K)
• advanced-port-bindings.xml.zip (2.3 K)