sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sharad Agarwal <sharad.apa...@gmail.com>
Subject Map/Reduce being idempotent for export
Date Wed, 30 Nov 2011 04:29:22 GMT
How do we ensure that map/reduce tasks are idempotent. To be more clear: if
there are more than 1 map/reduce attempts for an export job, how do we make
sure that data from only single task attempt is committed ?

Normally this is handled by OutputCommitter, however I see
NullOutputCommitter which doesn't do anything.

Thanks
Sharad

Mime
View raw message