kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Milkowski <amgm2...@gmail.com>
Subject kafka hadoop consumer and producer use older 0.19.2 hadoop api's
Date Wed, 05 Jun 2013 16:56:56 GMT
New hadoop api present in 0.20.1 especially package  org.apache.hadoop.
mapredude.lib is not used

code affected is both consumer and producer in kafka in the contrib package

[amilkowski@localhost contrib]$ pwd
/opt/local/git/kafka/contrib
[amilkowski@localhost contrib]$ ls -lt
total 12
drwxrwxr-x 8 amilkowski amilkowski 4096 May 30 11:14 hadoop-consumer
drwxrwxr-x 6 amilkowski amilkowski 4096 May 29 19:31 hadoop-producer
drwxrwxr-x 6 amilkowski amilkowski 4096 May 29 16:43 target
[amilkowski@localhost contrib]$

in example

import org.apache.hadoop.mapred.JobClient;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.RunningJob;
import org.apache.hadoop.mapred.TextOutputFormat;

use 0.19.2 hadoop api format, this prevents merging of hadoop feature into
more modern hadoop implementation

instead of drawing from 0.20.1 api set in import org.apache.hadoop.mapreduce

submitting jira feature  request

-A

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message