hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject enforcing value class type
Date Wed, 14 Sep 2016 22:36:44 GMT
Hi,
I have a question on the following check in
MapTask$MapOutputBuffer.collect():

      if (value.getClass() != valClass) {
        throw new IOException("Type mismatch in value from map: expected "
                              + valClass.getName() + ", received "
                              + value.getClass().getName());
      }

See an example of the above in the following:

2016-09-14 22:20:09,643 WARN  [Thread-1538] mapred.LocalJobRunner$Job(560):
job_local579834479_0004
java.lang.Exception: java.io.IOException: Type mismatch in value from map:
expected org.apache.hadoop.hbase.KeyValue, received
org.apache.hadoop.hbase.SizeCachedNoTagsKeyValue
        at
org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
        at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.io.IOException: Type mismatch in value from map: expected
org.apache.hadoop.hbase.KeyValue, received
org.apache.hadoop.hbase.SizeCachedNoTagsKeyValue
        at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1079)
        at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:715)
        at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
        at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
        at
org.apache.hadoop.hbase.backup.mapreduce.HFileSplitter$HFileCellMapper.map(HFileSplitter.java:93)
        at
org.apache.hadoop.hbase.backup.mapreduce.HFileSplitter$HFileCellMapper.map(HFileSplitter.java:77)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
        at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)

where:

public class SizeCachedNoTagsKeyValue extends SizeCachedKeyValue {

public class SizeCachedKeyValue extends KeyValue {

Can someone highlight why subclass of valClass is rejected by the check ?

Thanks

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message