log4j的异步哲学AsyncAppender

 一份好的代码，从良好的注释习惯开始。接触的东西多了，愈有感触。

  最近调试一些c++的接口，20多个字段的接口，居然没有一个字的注释，字段间的层级依赖关系也一字不提。后果当然是可想而知了，抓3，5个人的群一个个字段的问过去，答过来。调用一次，沟通一次，耗时2天有余，至今不通，挫折感非常强。

 

/**
 * The AsyncAppender lets users log events asynchronously.
 * <p/>
 * <p/>
 * The AsyncAppender will collect the events sent to it and then dispatch them
 * to all the appenders that are attached to it. You can attach multiple
 * appenders to an AsyncAppender.
 * </p>
 * <p/>
 * <p/>
 * The AsyncAppender uses a separate thread to serve the events in its buffer.
 * </p>
 * <p/>
 * <b>Important note:</b> The <code>AsyncAppender</code> can only be script
 * configured using the {@link org.apache.log4j.xml.DOMConfigurator}.
 * </p>
 *
 * @author Ceki G&uuml;lc&uuml;
 * @author Curt Arnold
 * @since 0.9.1
 */

 

 先看构造函数：

  /**
   * Create new instance.
   */
  public AsyncAppender() {
    appenders = new AppenderAttachableImpl();

    //
    //   only set for compatibility
    aai = appenders;

    dispatcher =
      new Thread(new Dispatcher(this, buffer, discardMap, appenders));

    // It is the user's responsibility to close appenders before
    // exiting.
    dispatcher.setDaemon(true);

    // set the dispatcher priority to lowest possible value
    //        dispatcher.setPriority(Thread.MIN_PRIORITY);
    dispatcher.setName("Dispatcher-" + dispatcher.getName());
    dispatcher.start();
  }

 构造了一个叫despatcher的最低级别守护进程。 

 

 

其实唯一需要关注的方法就是append，我们看它怎么做异步append

  /**
   * {@inheritDoc}
   */
  public void append(final LoggingEvent event) {
    //
    //   if dispatcher thread has died then
    //      append subsequent events synchronously
    //   See bug 23021
	// 如果dispatcher线程不在了，直接调用各个appender的同步写方法，确保日志的写入
    if ((dispatcher == null) || !dispatcher.isAlive() || (bufferSize <= 0)) {
      synchronized (appenders) {
        appenders.appendLoopOnAppenders(event);
      }

      return;
    }

    // Set the NDC and thread name for the calling thread as these
    // LoggingEvent fields were not set at event creation time.
    event.getNDC();
    event.getThreadName();
    // Get a copy of this thread's MDC.
    event.getMDCCopy();
    if (locationInfo) {
      event.getLocationInformation();
    }

    synchronized (buffer) { //这里将buffer资源对象当锁，可以想象到所有操作buffer的方法都必须做同步。
      while (true) {
        int previousSize = buffer.size();

        if (previousSize < bufferSize) {
          buffer.add(event);

          //
          //   if buffer had been empty
          //       signal all threads waiting on buffer
          //       to check their conditions.
          //
          if (previousSize == 0) {
            buffer.notifyAll();  //notifyAll()方法是Object对象的方法，而不是Thread对象的方法。
          }

          break;
        }

        //
        //   Following code is only reachable if buffer is full
        //   如果buffer写满了，才会走到下面的代码流程
        //
        //   if blocking and thread is not already interrupted
        //      and not the dispatcher then
        //      wait for a buffer notification
        boolean discard = true;
        if (blocking  //buffer满就阻塞的标志位
                && !Thread.interrupted()
                && Thread.currentThread() != dispatcher) {
          try {
            buffer.wait(); //满了就等待把
            discard = false;
          } catch (InterruptedException e) {
            //
            //  reset interrupt status so
            //    calling code can see interrupt on
            //    their next wait or sleep.
            Thread.currentThread().interrupt(); //这里处理interrupt的方法值得体会：）当然可以学会套用这个模板
          }
        }

        //
        //   if blocking is false or thread has been interrupted
        //   add event to discard map.
        //
        if (discard) { //如果没能成功进入wait(),放到丢弃消息map中保存
          String loggerName = event.getLoggerName();
          DiscardSummary summary = (DiscardSummary) discardMap.get(loggerName);

          if (summary == null) {
            summary = new DiscardSummary(event);
            discardMap.put(loggerName, summary);
          } else {
            summary.add(event);
          }

          break;
        }
      }
    }
  }

 

 

这里出现了2个新的东西，dispatcher 和DiscardSummary，下面分析dispatcher线程的实现：比较简单看懂，也就不写注释了。、

 

  /**
   * Event dispatcher.
   */
  private static class Dispatcher implements Runnable {
    /**
     * Parent AsyncAppender.
     */
    private final AsyncAppender parent;

    /**
     * Event buffer.
     */
    private final List buffer;

    /**
     * Map of DiscardSummary keyed by logger name.
     */
    private final Map discardMap;

    /**
     * Wrapped appenders.
     */
    private final AppenderAttachableImpl appenders;

    /**
     * Create new instance of dispatcher.
     *
     * @param parent     parent AsyncAppender, may not be null.
     * @param buffer     event buffer, may not be null.
     * @param discardMap discard map, may not be null.
     * @param appenders  appenders, may not be null.
     */
    public Dispatcher(
      final AsyncAppender parent, final List buffer, final Map discardMap,
      final AppenderAttachableImpl appenders) {

      this.parent = parent;
      this.buffer = buffer;
      this.appenders = appenders;
      this.discardMap = discardMap;
    }

    /**
     * {@inheritDoc}
     */
    public void run() {
      boolean isActive = true;

      //
      //   if interrupted (unlikely), end thread
      //
      try {
        //
        //   loop until the AsyncAppender is closed.
        //
        while (isActive) {
          LoggingEvent[] events = null;

          //
          //   extract pending events while synchronized
          //       on buffer
          //
          synchronized (buffer) {
            int bufferSize = buffer.size();
            isActive = !parent.closed;

            while ((bufferSize == 0) && isActive) {
              buffer.wait();
              bufferSize = buffer.size();
              isActive = !parent.closed;
            }

            if (bufferSize > 0) {
              events = new LoggingEvent[bufferSize + discardMap.size()];
              buffer.toArray(events);

              //
              //   add events due to buffer overflow
              //
              int index = bufferSize;

              for (
                Iterator iter = discardMap.values().iterator();
                  iter.hasNext();) {
                events[index++] = ((DiscardSummary) iter.next()).createEvent();
              }

              //
              //    clear buffer and discard map
              //
              buffer.clear();
              discardMap.clear();

              //
              //    allow blocked appends to continue
              buffer.notifyAll();
            }
          }

          //
          //   process events after lock on buffer is released.
          //
          if (events != null) {
            for (int i = 0; i < events.length; i++) {
              synchronized (appenders) {
                appenders.appendLoopOnAppenders(events[i]);
              }
            }
          }
        }
      } catch (InterruptedException ex) {
        Thread.currentThread().interrupt();
      }
    }
  }

 

 

看完整个类的分析，发现其实是一个非常典型的“生产消费模型”，需要3个角色：

生成者（通常是前台实时线程），

中转服务(可以是缓冲区buffer，也可以是离线存储演化成JMS，加上逻辑处理)，

消费者()

 

AsyncAppender则是对这种模型的一种实践：

1.生产者通常基于资源限制和时间性能的考量，往往需要在生产环节很快的得到调用返回值。

2.生产者、消费者会将业务逻辑全部剥离，由中转服务去做。

3..中转服务会成为整个系统设计的精髓。 需要根据业务的性能指标、可靠性期望指标、以及异常流程处理（比如是否去重、掉线补发）上下功夫。 

4.消费者通常需要和生成者独立开。由于系统的控制逻辑全部在中转服务层，所以消费者完全可以使用低优先级别的守护线程。

5. 这种模型一定要做定量分析，因为消费速度一旦跟不上生成速度，中转buffer的溢出处理是非常麻烦的一件事情。在AsyncAppender中的DiscardSummary就是为此而设计。