lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "808" <xiaoao...@qq.com>
Subject How to use TokenStream build two fields
Date Tue, 23 Apr 2013 11:51:53 GMT
I am a lucene user from China,so my English is bad.I will try my best to explain my problem.
The version I use is 4.2.I have a problem during I use lucene .
Here is my code:
public void testIndex() throws IOException, SQLException {
		NewsDao ndao = new NewsDao();
		List<News> newsList = ndao.getNewsListAll();
		Analyzer analyzer = new IKAnalyzer(true);
		Directory directory = FSDirectory.open(new File(INDEX_DRICTORY));


		IndexWriterConfig config = new IndexWriterConfig(MatchVersion, analyzer);
		config.setOpenMode(IndexWriterConfig.OpenMode.CREATE);


		IndexWriter writer = new IndexWriter(directory, config);
		StringField idField = new StringField("nid", String.valueOf(0),
				Field.Store.YES);
		TokenStream title_ts = null;
		TokenStream content_ts = null;
		for (News n : newsList) {
			Document doc = new Document();
			idField.setStringValue(String.valueOf(n.getId()));
			content_ts = analyzer.tokenStream("content", new StringReader(HTMLFilter
					.delHTMLTag(n.getNewsContext())));
			title_ts = analyzer.tokenStream("title",new StringReader(n.getNewsTitle()));
			getTokens(content_ts);
			doc.add(idField);
			doc.add(new TextField("content", content_ts));
			doc.add(new TextField("title", title_ts));
			writer.addDocument(doc);
		}
		if (content_ts != null) {
			try {
				content_ts.close();
			} catch (IOException e) {
				e.printStackTrace();
			}
		}
		writer.close(true);
		directory.close();
	}



I just want to use TokenStream to get the tokenized result,but I met NullpointException as
following:
Exception in thread "main" java.lang.NullPointerException
	at org.wltea.analyzer.core.AnalyzeContext.fillBuffer(AnalyzeContext.java:124)
	at org.wltea.analyzer.core.IKSegmenter.next(IKSegmenter.java:122)
	at org.wltea.analyzer.lucene.IKTokenizer.incrementToken(IKTokenizer.java:78)
	at org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:102)
	at org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:254)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:256)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:376)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1148)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1129)
	at manage.lucene.LuceneTools.testIndex(LuceneTools.java:130)
	at manage.lucene.LuceneTools.main(LuceneTools.java:95)

How can I solve this problem.Thanks~
Read
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message