关于hive处理日志中的异常栈遇到的问题

0 0

关于hive处理日志中的异常栈遇到的问题0

2012-02-16 16:27:33,393 [Prototyper] ERROR org.logicalcobwebs.proxool.proxool - Prototype
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:407)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:343)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2334)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2371)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2163)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:794)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        at sun.reflect.GeneratedConstructorAccessor112.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:407)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:378)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
        at java.sql.DriverManager.getConnection(DriverManager.java:582)
        at java.sql.DriverManager.getConnection(DriverManager.java:154)
        at org.logicalcobwebs.proxool.ProxyFactory.buildProxyConnection(ProxyFactory.java:41)
        at org.logicalcobwebs.proxool.Prototyper.buildConnection(Prototyper.java:150)
        at org.logicalcobwebs.proxool.Prototyper.sweep(Prototyper.java:95)
        at org.logicalcobwebs.proxool.PrototyperThread.run(PrototyperThread.java:42)
Caused by: java.net.ConnectException: Connection timed out
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
        at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
        at java.net.Socket.connect(Socket.java:529)
        at java.net.Socket.connect(Socket.java:478)
        at java.net.Socket.<init>(Socket.java:375)
        at java.net.Socket.<init>(Socket.java:218)
        at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:254)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:292)
        ... 17 more
2012-02-16 22:49:30,126 [[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO org.logicalcobwebs.proxool.ProxoolFacade - Proxool 0.8.3 (14-Dec-2003 16:06)
以上是我要处理的日志文件，我想通过hive过滤日志中的异常栈，然后把所有异常栈收集整理到一个文件中。我想通过SerDe的正则式匹配来做这个工作，我先尝试了写了一个正则，测试成功，然后在hive中建表：
create table regextest (line_date STRING, line_time STRING, classname STRING, message_type STRING, message_protocol STRING, message STRING) row format serde 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe' WITH SERDEPROPERTIES ("input.regex" = "^([^ ]*) ([^ ]*) (\\[[^ ]*\\]) (ERROR)+ ([^ ]*) (?:-) ([^\\|]*)(?=\\d{4}-\\d{2}-\\d{2})+", "output.format.string" = "%1$s %2$s %3$s %4$s %5$s %$6s") stored as textfile;然后倒入数据如上，但是select时候发现为null值，所以在这里请教各位我哪里出错了。之前测试过正则，应该没写错。