log4j Usage

In our projects, there are always the following questions for log4j integration:

1. log4j could be exist in several jars, is there any conflict? For example, each components has their own logging implementation, is there any problem/conflict?
2. Why log4j.xml is provided but log4j.properties is always invoked by some sub-components?

We can find "How to use log4j" and basic log4j concepts from google easiliy with many demos. Here I would like to list some topics that I had not discovered frequentely from web site.

 

log4j.configuration and configuration file

log4j.properties VS. log4j.xml

We can find following description from org.apache.log4j.LogManager.java:

 

// if the user has not specified the log4j.configuration

// property, we search first for the file "log4j.xml" and then

// "log4j.properties"

public class LogManager {
	static {
		// By default we use a DefaultRepositorySelector which always returns
		// 'h'.
		Hierarchy h = new Hierarchy(new RootLogger((Level) Level.DEBUG));
		repositorySelector = new DefaultRepositorySelector(h);

		/** Search for the properties file log4j.properties in the CLASSPATH. */
		String override = OptionConverter.getSystemProperty(
				DEFAULT_INIT_OVERRIDE_KEY, null);

		// if there is no default init override, then get the resource
		// specified by the user or the default config file.
		if (override == null || "false".equalsIgnoreCase(override)) {

			String configurationOptionStr = OptionConverter.getSystemProperty(
					DEFAULT_CONFIGURATION_KEY, null);

			String configuratorClassName = OptionConverter.getSystemProperty(
					CONFIGURATOR_CLASS_KEY, null);

			URL url = null;

			// if the user has not specified the log4j.configuration
			// property, we search first for the file "log4j.xml" and then
			// "log4j.properties"
			if (configurationOptionStr == null) {
				url = Loader.getResource(DEFAULT_XML_CONFIGURATION_FILE);
				if (url == null) {
					url = Loader.getResource(DEFAULT_CONFIGURATION_FILE);
				}
			} else {
				try {
					url = new URL(configurationOptionStr);
				} catch (MalformedURLException ex) {
					// so, resource is not a URL:
					// attempt to get the resource from the class path
					url = Loader.getResource(configurationOptionStr);
				}
			}

			// If we have a non-null url, then delegate the rest of the
			// configuration to the OptionConverter.selectAndConfigure
			// method.
			if (url != null) {
				LogLog.debug("Using URL [" + url
						+ "] for automatic log4j configuration.");
				try {
					OptionConverter.selectAndConfigure(url,
							configuratorClassName, LogManager
									.getLoggerRepository());
				} catch (NoClassDefFoundError e) {
					LogLog.warn("Error during default initialization", e);
				}
			} else {
				LogLog.debug("Could not find resource: ["
						+ configurationOptionStr + "].");
			}
		} else {
			LogLog.debug("Default initialization of overridden by "
					+ DEFAULT_INIT_OVERRIDE_KEY + "property.");
		}
	}

}

 

log4j Appender (log4j.appender.{appenderName} or log4j:configuration/appender/@class)

 

Log4j allows logging requests to print to multiple destinations. In log4j speak, an output destination is called an appender. Currently, appenders exist for the console, files, GUI components, remote socket servers, JMS, NT Event Loggers, and remote UNIX Syslog daemons. It is also possible to log asynchronously.

 

1. org.apache.log4j.ConsoleAppender
2. org.apache.log4j.FileAppender
3. org.apache.log4j.DailyRollingFileAppender
4. org.apache.log4j.RollingFileAppender
5. org.apache.log4j.WriterAppender
6. org.apache.log4j.net.SMTPAppender
7. org.apache.log4j.jdbc.JDBCAppender

 

More than one appender can be attached to a logger.

 

org.apache.log4j.ConsoleAppender

Output log to console.

org.apache.log4j.FileAppender

Output log to specified log file.

/**
 *  FileAppender appends log events to a file.
 *
 *  <p>Support for <code>java.io.Writer</code> and console appending
 *  has been deprecated and then removed. See the replacement
 *  solutions: {@link WriterAppender} and {@link ConsoleAppender}.
 *
 * @author Ceki G&uuml;lc&uuml; 
 * */

 

org.apache.log4j.DailyRollingFileAppender

Output log to specified log file and store history log files by scheduled fresequence: monthly, weekly, half-daily, daily, hourly, or minutely. 

/**
   DailyRollingFileAppender extends {@link FileAppender} so that the
   underlying file is rolled over at a user chosen frequency.

   <p>The rolling schedule is specified by the <b>DatePattern</b>
   option. This pattern should follow the {@link SimpleDateFormat}
   conventions. In particular, you <em>must</em> escape literal text
   within a pair of single quotes. A formatted version of the date
   pattern is used as the suffix for the rolled file name.

   <p>For example, if the <b>File</b> option is set to
   <code>/foo/bar.log</code> and the <b>DatePattern</b> set to
   <code>'.'yyyy-MM-dd</code>, on 2001-02-16 at midnight, the logging
   file <code>/foo/bar.log</code> will be copied to
   <code>/foo/bar.log.2001-02-16</code> and logging for 2001-02-17
   will continue in <code>/foo/bar.log</code> until it rolls over
   the next day.

   <p>Is is possible to specify monthly, weekly, half-daily, daily,
   hourly, or minutely rollover schedules.

   ...
   
   <p>Do not use the colon ":" character in anywhere in the
   <b>DatePattern</b> option. The text before the colon is interpeted
   as the protocol specificaion of a URL which is probably not what
   you want.

*/

 

org.apache.log4j.RollingFileAppender

Output log to specified log file and limit the file size to specified value. In case file size reached, a new log file will be used.

/**
   RollingFileAppender extends FileAppender to backup the log files when
   they reach a certain size.
   
   The log4j extras companion includes alternatives which should be considered
   for new deployments and which are discussed in the documentation
   for org.apache.log4j.rolling.RollingFileAppender.
   

   @author Heinz Richter
   @author Ceki Gülcü

*/

 

org.apache.log4j.WriterAppender

Output log to specified destination in stream.

org.apache.log4j.net.SMTPAppender

Send an e-mail when a specific logging event occurs, typically on errors or fatal errors.

org.apache.log4j.jdbc.JDBCAppender

The JDBCAppender provides for sending log events to a database.

/**
  The JDBCAppender provides for sending log events to a database.
  
 <p><b><font color="#FF2222">WARNING: This version of JDBCAppender
 is very likely to be completely replaced in the future. Moreoever,
 it does not log exceptions</font></b>.

  <p>Each append call adds to an <code>ArrayList</code> buffer.  When
  the buffer is filled each log event is placed in a sql statement
  (configurable) and executed.

  <b>BufferSize</b>, <b>db URL</b>, <b>User</b>, & <b>Password</b> are
  configurable options in the standard log4j ways.

  <p>The <code>setSql(String sql)</code> sets the SQL statement to be
  used for logging -- this statement is sent to a
  <code>PatternLayout</code> (either created automaticly by the
  appender or added by the user).  Therefore by default all the
  conversion patterns in <code>PatternLayout</code> can be used
  inside of the statement.  (see the test cases for examples)
  
  ...
*/

Appender Additivity

The addAppender method adds an appender to a given logger. Each enabled logging request for a given logger will be forwarded to all the appenders in that logger as well as the appenders higher in the hierarchy. In other words, appenders are inherited additively from the logger hierarchy. For example, if a console appender is added to the root logger, then all enabled logging requests will at least print on the console. If in addition a file appender is added to a logger, say C, then enabled logging requests for C and C's children will print on a file and on the console. It is possible to override this default behavior so that appender accumulation is no longer additive by setting the additivity flag to false.

 

Appender Additivity

The output of a log statement of logger C will go to all the appenders in C and its ancestors. This is the meaning of the term "appender additivity".

However, if an ancestor of logger C, say P, has the additivity flag set to false, then C's output will be directed to all the appenders in C and its ancestors upto and including P but not the appenders in any of the ancestors of P.

Loggers have their additivity flag set to true by default.

log4j log layout

There are several layout classes:

1. org.apache.log4j.HTMLLayout 
2. org.apache.log4j.PatternLayout
3. org.apache.log4j.SimpleLayout
4. org.apache.log4j.TTCCLayout

Normally, PatternLayout is used, and we can define the ConversionPattern layout with following options:

• %c   qualification class name
• %d   date time
• %F   display from which file the log comes from. Normally it is the class java file name, like: XXX.java
• %l   location
• %m   log message
• %n   carriage return
• %p   log level
• %r   log output cost time: from log request and output into log file.
• %t   thread name

Extention configuration for layout options

Output align

We can define "left" and "right" align by specify a number value befor options, for example:

• %5p: the minimum width for log level display is 5, in case the level length less than 5, it will be right align.
• %-5p: the minimum width for log level display is 5, in case the level length less than 5, it will be left align.

Change default layout info

We can use "{new_layout}" after options charactor to change default output. For example:

• %c will display qualification full class name. We can use %c{1} to output only ClassName.
• %d will display the date in ISO8601 format by default. We can use %-d{yyyy-MM-dd HH:mm:ss.SSS} to specify the date output int format: yyyy-MM-dd HH:mm:ss.SSS.

log4j MDC

 

Mapped Diagnostic Context: it is instrument for distinguishing interleaved log output from different sources. org.apache.log4j.MDC.java:

 

/**
   The MDC class is similar to the {@link NDC} class except that it is
   based on a map instead of a stack. It provides <em>mapped
   diagnostic contexts</em>. A <em>Mapped Diagnostic Context</em>, or
   MDC in short, is an instrument for distinguishing interleaved log
   output from different sources. Log output is typically interleaved
   when a server handles multiple clients near-simultaneously.

   <p><b><em>The MDC is managed on a per thread basis</em></b>. A
   child thread automatically inherits a <em>copy</em> of the mapped
   diagnostic context of its parent.
  
   <p>The MDC class requires JDK 1.2 or above. Under JDK 1.1 the MDC
   will always return empty values but otherwise will not affect or
   harm your application.
   
   @since 1.2

   @author Ceki G&uuml;lc&uuml; 
*/

 

 

We can use %X{key} to output value information in MDC with specified key. For example:

 

    <layout class="org.apache.log4j.PatternLayout">
		<param name="ConversionPattern" value="%d %-5p [GTO] [%c] %X{MSISDN} %m%n"/>
    </layout>

During our business implementation, we can set the MSISDN value to MDC at any time by using following java code:

org.apache.log4j.MDC.put("MSISDN", new StringBuilder("[MSISDN=876543210000001Test]")
				.toString());

Then log4j will automatically use [MSISDN=876543210000001Test] to replace %X{MSISDN}.

 

If the value specified by %X{key} not existing, nothing will be ouptput in log.

Configure log4j Hierarchy

Log4j maintain logger by names in a Logger Hierarchy. This hierarchy is initialized as following.

org.apache.log4j.helpers.OptionConverter.java:

 

	/**
	 * Configure log4j given a URL.
	 * 
	 * <p>
	 * The url must point to a file or resource which will be interpreted by a
	 * new instance of a log4j configurator.
	 * 
	 * <p>
	 * All configurations steps are taken on the <code>hierarchy</code> passed
	 * as a parameter.
	 * 
	 * <p>
	 * 
	 * @param url
	 *            The location of the configuration file or resource.
	 * @param clazz
	 *            The classname, of the log4j configurator which will parse the
	 *            file or resource at <code>url</code>. This must be a subclass
	 *            of {@link Configurator}, or null. If this value is null then a
	 *            default configurator of {@link PropertyConfigurator} is used,
	 *            unless the filename pointed to by <code>url</code> ends in
	 *            '.xml', in which case
	 *            {@link org.apache.log4j.xml.DOMConfigurator} is used.
	 * @param hierarchy
	 *            The {@link org.apache.log4j.Hierarchy} to act on.
	 * @since 1.1.4
	 */

	static public void selectAndConfigure(URL url, String clazz,
			LoggerRepository hierarchy) {
		Configurator configurator = null;
		String filename = url.getFile();

		if (clazz == null && filename != null && filename.endsWith(".xml")) {
			clazz = "org.apache.log4j.xml.DOMConfigurator";
		}

		if (clazz != null) {
			LogLog.debug("Preferred configurator class: " + clazz);
			configurator = (Configurator) instantiateByClassName(clazz,
					Configurator.class, null);
			if (configurator == null) {
				LogLog.error("Could not instantiate configurator [" + clazz
						+ "].");
				return;
			}
		} else {
			configurator = new PropertyConfigurator();
		}

		configurator.doConfigure(url, hierarchy);
	}

 We can see that OptionConverter will invoke dedicate Configurator to configure log4j. 

 

org.apache.log4j.PropertyConfigurator

PropertyConfigurator will load log4j.properties and initialize logger hierarchy.

 

It provide a method  to watch log4j.properties file to periodically load latest configuration.

org.apache.log4j.xml.DOMConfigurator

DOMConfigurator is used to load log4j.xml and initialize logger hierarchy. If there is already Logger Hierarchy existing, it will update configuration.

 

It provide a method  to watch log4j.properties file to periodically load latest configuration.

 

log4j Design Parttern

LoggerFactory design

To be updated later.

 

log4j in My Projects

Customized log4j: Both PropertyConfigurator and DOMConfigurator are invoked in same Project

Different components in same project could customized log4j by themselves. For example, one third party library use "PropertyConfigurator" to force load "log4j.properties" and Another components use "DOMConfigurator" to force load "log4j.xml".  In order to integrate them successfully,  we should be aware of that:

• LogManager is singleton in ClassLoader. It manages a Hierarchy which manage the tree nodes of all Logger.
• PropertyConfigurator and DOMConfigurator could create different tree nodes hierarchies in same LoggerRepository.

We should provide both log4j.properties and log4j.xml. Then we can configure the log4j.properties to output logs into same log file (Appender.append=true) as that used in log4j.xml.