JBOSS Performance Tuning

JBOSS Performance Tuning

2. JVM TUNING
The first thing, find the certified version of JVM
http://www.jboss.com/products/platforms/application/supportedconfigurations/#JEAP5-0

2.1 Heap Sizing
Once the jboss server is started with the all configuration, the application server by itself will need about 512Mb of memory during startup to initialize properly. Once it is started, it will run within about 300Mb.

The configuration file run.conf, and the items are -Xms -Xmx. Max and Min values are always the same.

2.2 32-bit vs 64-bit JVM
If the memory is less than than 2Gb, the performance differences between 32-bit and 64-bit JVM is negligible.

In summary, if your operating system and hardware supports it along with enough memory committable, use the 64-bit JVM.

2.3 Large Page Memory
Most modern operating systems have a default memory page size of 4k. With a server with 4Gb of memory, you would get almost 1 million addressable pages.

Resulting in larger contiguous blocks of memory (better for applications) and fewer memory pages to manage (better for JVM).
-XX:+UseLargePages

2.4 Garbage Collection
Using the parallel collector on the old genereration -XX:+UseParallelOldGC, we have seen throughput be almost 20% higher using this collector as opposed to the CMS collector which does parallel collection on the Eden space.
>java -server -Xms12288m -Xmx12288m -XX:+UseLargePages -XX:UseParallelOldGC

2.5 Other JVM Options

3. Servlet Container Tuning
3.1 CachedConnectionManager
By default, the CachedConnectionManager is configured to be in the servlet container and it is set to be in debug mode in the default and all configurations.
There is a production configuration which still contains the CachedConnectionManager but turns the debug mode off.

There are two options here. You can use the production configuration as is or we go the extra step of removing it getting rid of the overhead completely. If you use bean managed transactions then do not remove this setting.

directory and file: /opt/jboss-as/server/default/deploy/jbossweb.sar/server.xml
comments the follow lines:
<!--
            <Valve className="org.jboss.web.tomcat.service.jca.CachedConnectionValve"
                cachedConnectionManagerObjectName="jboss.jca:service=CachedConnectionManager"
                transactionManagerObjectName="jboss:service=TransactionManager" />
             -->

directory and file: /opt/jboss-as/server/default/deploy/jbossweb.sar/META-INF/jboss-beans.xml
<!-- <depends>jboss.jca:service=CachedConnectionManager</depends> -->
<!-- <depends>jboss:service=TransactionManager</depends> -->

3.2 HTTP Session Replication

4. WEB Connectors
4.1 Java I/O Endpoint
This endpoint (connector) is used when clients send HTTP requests directly to the platform and not to a front end HTTPD server such as Apache HTTPD.
Some options: maxKeepAliveRequests, maxThreads, minSpareThreads
All of the above parameters are in the server.xml file for our embedded servlet contrainer, JBOSS Web (derived from Apache Tomcat).
configeration file: /opt/jboss-as/server/default/deploy/jbossweb.sar/server.xml
<Connector protocol="HTTP/1.1" port="80" address="${jboss.bind.address}"
               connectionTimeout="20000" redirectPort="443" />
maxKeepAliveRequests is not specified, the default value of this parameter is 100.

This means is that after an initial request from a client creates a persistent connection, it can send 100 requests over that connection before the server will close the connection.
1000
1       Setting the value to 1 will actually disable persistent connections for any requests regardless of protocol being used by the client or user agent (e.g. HTTP 1.0 vs. HTTP 1.1)
-1      Make the connector support an unlimited amount of persistent connections.

maxThreads creates the thread pool that sits directly behind the connector and processes whatever the request is, default value is 200 threads.

minSpareThreads ---- if maxThreads represent a peak, but that peak doesn't last very long, then setting minSpareThreads to a lower value is the right thing to do.

<Connector protocol="HTTP/1.1" port="80" address="${jboss.bind.address}"
               connectionTimeout="20000" redirectPort="443" maxKeepAliveRequests="-1" maxThreads="600" minSpareThreads="400" />

4.2 AJP Connector
The AJP connector is typically used when you place an HTTP server, such as Apache HTTPD, in front of the platform.

Options: maxThreads  minSpareThreads

5. EJB3 Containner Tuning
5.1 Stateless Session Bean
TheadlocalPool ---- unlimited, the pool size of the ThreadlocalPool is actually the number of unique stateless session beans you have in your application.

StrictMaxPool ----- the edge of the throughput.

configuration file: /opt/jboss-as/server/default/deploy/ejb3-interceptors-aop.xml
<domain name="Stateless Bean" extends="Intercepted Bean" inheritBindings="true">
...
<annotation expr="class(*) AND !class(@org.jboss.ejb3.annotation.Pool)">
         @org.jboss.ejb3.annotation.Pool (value="ThreadlocalPool", maxSize=30, timeout=10000)
</annotation>
...
<domain name="JACC Stateless Bean" extends="Intercepted Bean" inheritBindings="true">
...
<annotation expr="class(*) AND !class(@org.jboss.ejb3.annotation.Pool)">
         @org.jboss.ejb3.annotation.Pool (value="ThreadlocalPool", maxSize=30, timeout=10000)
</annotation>
...

5.2 Stateful Session Bean

5.3 Entity Beans
Entity beans have a bad reputation as the old EJB1 and 2 entities beans. But EJB 3 is good.
1. Use of a Second Level Cache
2. Prepared Statements
3. Batch Inserts
4. Batching Database Operations

Caching is great for data that doesn't change often.
Configuration file: persistence.xml

5.4 Message Driven Beans

6. JCA Container
6.1 Data Sources
As you may recall from our section on Entity Beans, we do this through a *-ds.xml file that we place in the deploy directory of our specific configuration.
<?xml version="1.0" encoding="UTF-8"?>
<datasources>
<xa-datasource>
                <jndi-name>jdniName</jndi-name>
                <xa-datasource-class>com.mysql.jdbc.jdbc2.optional.MysqlXADataSource</xa-datasource-class>
                <xa-datasource-property name="URL">jdbc:mysql://127.0.0.1:3306/database</xa-datasource-property>
                <xa-datasource-property name="User">username</xa-datasource-property>
                <xa-datasource-property name="Password">password</xa-datasource-property>
                <min-pool-size>20</min-pool-size>
                <max-pool-size>50</max-pool-size>
                <idle-timeout-minutes>10</idle-timeout-minutes>
                <valid-connection-checker-class-name>org.jboss.resource.adapter.jdbc.vendor.MySQLValidConnectionChecker</valid-connection-checker-class-name>
                <background-validation>true</background-validation>
                <background-validation-minutes>5</background-validation-minutes>
</xa-datasource>
</datasources>

min-pool-size defines the minmum size that the connection pool to the database should be.
Database connections tend to be fairly expensive to create and tear down, so we pool that resource and reuse them.

max-pool-size The pool shrinks during low activity down to min-pool-size.

6.2 JMS Integration / Provider

7. JMS Provider
Jboss provides two JMS providers within the platform. The first (and default) is Jboss Messaging and the second is our new technology called HornetQ.

From a performance perspective, HornetQ is far superior to Jboss Messaging.

8. Logging
1. Console logging
2. Appender type
3. Log file location
4. Log Guards

8.1 Console Logging
Console logging is simply not good in a production environment as it is slow and expensive.

The simplest way is to use the production configuration or turn it off when it is enabled by changing the configurations.

The configuration file for logging is called jboss-log4j.xml and it resides in the following directory:
/opt/jboss-as/server/default/conf/jboss-log4j.xml

There are 2 sections that you will need to comment out - one near the top and one near the bottom.
<!--
   <appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
      <errorHandler class="org.jboss.logging.util.OnlyOnceErrorHandler"/>
      <param name="Target" value="System.out"/>
      <param name="Threshold" value="DEBUG"/>
      <layout class="org.apache.log4j.PatternLayout">
         <param name="ConversionPattern" value="%d{ABSOLUTE} %-5p [%c{1}] %m%n"/>
      </layout>
   </appender> -->

The section at the bottom is the root category which contains the appenders used by logging for the system. Here you will need to comment out just the console appender.

<priority value="${jboss.server.log.threshold}"/>
<!--
      <appender-ref ref="CONSOLE"/>
-->
<appender-ref ref="FILE"/>

8.2 Appenders
Console logging is a form of an appender. There are two main types of file based appenders -
one that writes synchronously to the file and
another that writes asynchronously to the file.

We recommend using the asynchronous writer and this section will show you how to configure asynchronous log writing.

The configuration file: /opt/jboss-as/server/default/conf/jboss-log4j.xml

Uncomment the ASYNC appender

   <!-- Buffer events and log them asynchronously -->
   <appender name="ASYNC" class="org.apache.log4j.AsyncAppender">
     <errorHandler class="org.jboss.logging.util.OnlyOnceErrorHandler"/>
     <appender-ref ref="FILE"/>
     <!--
     <appender-ref ref="CONSOLE"/>
     <appender-ref ref="SMTP"/>
     -->
   </appender>

Make changes to the root element:
<root>
      <priority value="${jboss.server.log.threshold}"/>
      <!--
      <appender-ref ref="CONSOLE"/>
      <appender-ref ref="FILE"/>
      -->
      <appender-ref ref="ASYNC" />
</root>

8.3 Log File Location
Pass this parameter to command line to change the log file directory path.
-Djboss.server.log.dir=<path>

8.4 Log Guards
Java source codes should be like this:
if(debugEnabled()){
log.debug("some text..." + Object);
}

References:
jbossperformancetuning.pdf