nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eyeris Rodriguez Rueda (JIRA)" <j...@apache.org>
Subject [jira] [Created] (NUTCH-2357) Index metadata throw Exception because writable object cannot be cast to Text
Date Tue, 07 Feb 2017 13:53:41 GMT
Eyeris Rodriguez Rueda created NUTCH-2357:
---------------------------------------------

             Summary: Index metadata throw Exception because writable object cannot be cast
to Text
                 Key: NUTCH-2357
                 URL: https://issues.apache.org/jira/browse/NUTCH-2357
             Project: Nutch
          Issue Type: Bug
          Components: indexer
    Affects Versions: 1.12
         Environment: It was detected using Linux mint 18.
            Reporter: Eyeris Rodriguez Rueda
            Priority: Minor
             Fix For: 1.13


Index Metadata plugin use this property(see below), to take keys from Datum and index it.

<property>
  <name>index.db.md</name>
  <value></value>
  <description>
...
  </description>
</property>
Using any value from this property one Exception is thrown.
The problem occurs because Writable object can not be cast to Text see this line.
https://github.com/apache/nutch/blob/master/src/plugin/index-metadata/src/java/org/apache/nutch/indexer/metadata/MetadataIndexer.java#L58

A little change will fix it.

This is the Exception:
**********************************
2017-02-06 18:18:29,969 INFO  solr.SolrMappingReader - source: digest dest: digest
2017-02-06 18:18:29,969 INFO  solr.SolrMappingReader - source: tstamp dest: tstamp
2017-02-06 18:18:29,969 INFO  solr.SolrMappingReader - source: metatag.description dest: description
2017-02-06 18:18:29,969 INFO  solr.SolrMappingReader - source: metatag.keywords dest: keywords
2017-02-06 18:18:30,134 WARN  mapred.LocalJobRunner - job_local15168888_0001
java.lang.Exception: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot
be cast to org.apache.hadoop.io.Text
	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast to
org.apache.hadoop.io.Text
	at org.apache.nutch.indexer.metadata.MetadataIndexer.filter(MetadataIndexer.java:58)
	at org.apache.nutch.indexer.IndexingFilters.filter(IndexingFilters.java:51)
	at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:330)
	at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:56)
	at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
	at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
2017-02-06 18:18:30,777 ERROR indexer.IndexingJob - Indexer: java.io.IOException: Job failed!
	at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)
	at org.apache.nutch.indexer.IndexingJob.index(IndexingJob.java:145)
	at org.apache.nutch.indexer.IndexingJob.run(IndexingJob.java:228)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.nutch.indexer.IndexingJob.main(IndexingJob.java:237)
******************************************



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message