gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Zolotov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GORA-346) Create shim layer to support multiple hadoop versions
Date Tue, 07 Oct 2014 09:15:35 GMT

    [ https://issues.apache.org/jira/browse/GORA-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161669#comment-14161669

Ivan Zolotov commented on GORA-346:

As I understood, gora shim layer allows to instantiate appropriate Job and JobContext classes
for the certain hadoop version. It works very well. But what about TaskAttemptContext?
I've just tried to run nutch2 on hadoop2 and the following stack trace has been shown:
java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext,
but class was expected at org.apache.gora.mapreduce.GoraOutputFormat.getRecordWriter(GoraOutputFormat.java:83)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:624)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:744) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Gora-core module has explicit dependency from hadoop-core where TaskAttemptContext is class,
but in hadoop2 TaskAttemptContext is an interface. So, to I have to replace gora-core dependency
to make gora-core compatible with hadoop2. Because of it, it seems like gora-0.5 isn't compatible
with hadoop2 out of the box.
Please, correct me if I'm wrong.

> Create shim layer to support multiple hadoop versions
> -----------------------------------------------------
>                 Key: GORA-346
>                 URL: https://issues.apache.org/jira/browse/GORA-346
>             Project: Apache Gora
>          Issue Type: Improvement
>    Affects Versions: 0.5
>            Reporter: Renato Javier MarroquĂ­n Mogrovejo
>            Assignee: Moritz Hoffmann
>              Labels: patch
>             Fix For: 0.5
>         Attachments: GORA-345_MUNGE_v1.patch, GORA-346_v1.patch, GORA-346_v2.patch, GORA-346_v3.patch,
GORA-346_v4_profiles.patch, GORA-346_v5.patch

This message was sent by Atlassian JIRA

View raw message