ajp+mod_jk响应串号故障

淘宝网站某大压力项目的服务器出现了严重的用户响应串号问题，
追踪到根源还是ajp1.3协议本身容错性不强。

就拿apr配置来说,AjpAprProcessor,AjpAprProtocol,AprEndpoint这三个关键类
实行的是AjpAprProcessor复用即socket复用的机制，在代码
处理request,response导致io异常抛出的时候
apr socket没法正常销毁会引起用户串号,多发于post请求处理情况，
以下代码来自 jbossweb.jar jboss 4.2.2.

AjpAprProtocol.java 

        public SocketState process(long socket) {
            AjpAprProcessor processor = recycledProcessors.poll();
            try {

                if (processor == null) {
                    processor = createProcessor();
                }

                if (processor instanceof ActionHook) {
                    ((ActionHook) processor).action(ActionCode.ACTION_START, null);
                }

                if (processor.process(socket)) {
                    return SocketState.OPEN;
                } else {
                    return SocketState.CLOSED;
                }

            } catch(java.net.SocketException e) {
                // SocketExceptions are normal
                AjpAprProtocol.log.debug
                    (sm.getString
                     ("ajpprotocol.proto.socketexception.debug"), e);
            } catch (java.io.IOException e) {
                // IOExceptions are normal
                AjpAprProtocol.log.debug
                    (sm.getString
                     ("ajpprotocol.proto.ioexception.debug"), e);
            }
            // Future developers: if you discover any other
            // rare-but-nonfatal exceptions, catch them here, and log as
            // above.
            catch (Throwable e) {
                // any other exception or error is odd. Here we log it
                // with "ERROR" level, so it will show up even on
                // less-than-verbose logs.
                AjpAprProtocol.log.error
                    (sm.getString("ajpprotocol.proto.error"), e);
            } finally {
                if (processor instanceof ActionHook) {
                    ((ActionHook) processor).action(ActionCode.ACTION_STOP, null);
                }
                recycledProcessors.offer(processor);
            }
            return SocketState.CLOSED;
        }

实际上上面捕获ioexception这段代码已经很难生效，因为可能在之前调用时就被捕获
比如 org.apache.catalina.connector.Request

    /**
     * Parse request parameters.
     */
    protected void parseParameters() {

        parametersParsed = true;

        Parameters parameters = coyoteRequest.getParameters();

        // getCharacterEncoding() may have been overridden to search for
        // hidden form field containing request encoding
        String enc = getCharacterEncoding();

        boolean useBodyEncodingForURI = connector.getUseBodyEncodingForURI();
        if (enc != null) {
            parameters.setEncoding(enc);
            if (useBodyEncodingForURI) {
                parameters.setQueryStringEncoding(enc);
            }
        } else {
            parameters.setEncoding
                (org.apache.coyote.Constants.DEFAULT_CHARACTER_ENCODING);
            if (useBodyEncodingForURI) {
                parameters.setQueryStringEncoding
                    (org.apache.coyote.Constants.DEFAULT_CHARACTER_ENCODING);
            }
        }

        parameters.handleQueryParameters();

        if (usingInputStream || usingReader)
            return;

        if (!getMethod().equalsIgnoreCase("POST"))
            return;

        String contentType = getContentType();
        if (contentType == null)
            contentType = "";
        int semicolon = contentType.indexOf(';');
        if (semicolon >= 0) {
            contentType = contentType.substring(0, semicolon).trim();
        } else {
            contentType = contentType.trim();
        }
        if (!("application/x-www-form-urlencoded".equals(contentType)))
            return;

        int len = getContentLength();

        if (len > 0) {
            int maxPostSize = connector.getMaxPostSize();
            if ((maxPostSize > 0) && (len > maxPostSize)) {
                if (context.getLogger().isDebugEnabled()) {
                    context.getLogger().debug("Post too large");
                }
                return;
            }
            byte[] formData = null;
            if (len < CACHED_POST_LEN) {
                if (postData == null)
                    postData = new byte[CACHED_POST_LEN];
                formData = postData;
            } else {
                formData = new byte[len];
            }
            try {
                if (readPostBody(formData, len) != len) {
                    return;
                }
            } catch (IOException e) {
                // Client disconnect
                if (context.getLogger().isDebugEnabled()) {
                    context.getLogger().debug(
                            sm.getString("coyoteRequest.parseParameters"), e);
                }
            }
            parameters.processParameters(formData, 0, len);
        }

    }

而ajp1.3本身的协议在出现io异常而又没法销毁的情况下是非常危险的。

淘宝网目前某大牛直接修改ajp协议，加上任务id，丢弃串号响应

这种做法比较纠结，要同时fix升级mod_jk和jboss.后续可能考虑废弃ajp这个多年无人维护的协议。

apache的邮件列表上也有人反映该现象
https://issues.apache.org/bugzilla/show_bug.cgi?id=47714