Lucene如何索引巨大的PDF文件？

0 0

Lucene如何索引巨大的PDF文件？5

我想用Lucene对一个529G大小的PDF文件进行索引，开始使用Tika1.5，异常为：

TIKA-198: Illegal IOException from org.apache.tika.parser.pdf.PDFParser@1455935
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:248)
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242)
org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:120)
com.cc.file.Office2007FileConverter.getContents(Office2007FileConverter.java:38)
test.PDFTest.testEnPDF5(PDFTest.java:58)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
junit.framework.TestCase.runTest(TestCase.java:168)

然后又试用了PDFbox1.8.4，异常如下：

Exception in thread "main" org.apache.pdfbox.exceptions.WrappedIOException
at org.apache.pdfbox.pdfparser.PDFParser.parse(PDFParser.java:263)
at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:1219)
at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:1186)
at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:1111)
at com.cc.test.BigPDFTest.main(BigPDFTest.java:16)
Caused by: java.lang.IndexOutOfBoundsException: Index: 13, Size: 13
at java.util.ArrayList.rangeCheck(Unknown Source)
at java.util.ArrayList.get(Unknown Source)
at org.apache.pdfbox.io.RandomAccessBuffer.seek(RandomAccessBuffer.java:109)
at org.apache.pdfbox.io.RandomAccessFileOutputStream.write(RandomAccessFileOutputStream.java:106)
at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
at java.io.BufferedOutputStream.flush(Unknown Source)
at java.io.FilterOutputStream.close(Unknown Source)
at org.apache.pdfbox.pdfparser.BaseParser.parseCOSStream(BaseParser.java:618)
at org.apache.pdfbox.pdfparser.PDFParser.parseObject(PDFParser.java:605)
at org.apache.pdfbox.pdfparser.PDFParser.parse(PDFParser.java:194)
... 4 more
请教各位高人，如何parse这么大的PDF文件？多谢！！