sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Attila Szabo <mau...@apache.org>
Subject Re: Review Request 56984: SQOOP-3139: change default of mapreduce.map.maxattempts to 1 for import (so it's consistent with export)
Date Fri, 24 Feb 2017 07:06:24 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56984/#review166655
-----------------------------------------------------------



Hey Anna,

I've reviewed your proposed changes and also checked the related JIRA task, but TBH I'm a
bit concerned about:
- the nature of the problem (the problem description provided is not clear for me enough,
and I'm unsure if it is related to issue what was solved in case of export)
- eliminating retry would be a good idea (IMHO it's a good feature in case of Sqoop that we're
trying to rerun failed tasks).

Could you please provide me some more insights around this problem?

Many thanks,
Attila


src/java/org/apache/sqoop/mapreduce/ImportJobBase.java (lines 53 - 54)
<https://reviews.apache.org/r/56984/#comment238679>

    Hey Anna,
    
    Could you please extrace/move this constant to somewhere else (e.g. JobBase)?
    
    It's a bit confusing that ImportJobBase imports statically something from ExportJobBase.



src/java/org/apache/sqoop/mapreduce/ImportJobBase.java (lines 350 - 355)
<https://reviews.apache.org/r/56984/#comment238680>

    Are we sure about the assumption that re-trying an import does not make sense?
    
    Are we sure that users do not depend on this mechanism in production right now?


- Attila Szabo


On Feb. 23, 2017, 1:56 p.m., Anna Szonyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56984/
> -----------------------------------------------------------
> 
> (Updated Feb. 23, 2017, 1:56 p.m.)
> 
> 
> Review request for Sqoop, Boglarka Egyed, Attila Szabo, Szabolcs Vasas, and Liz Szilagyi.
> 
> 
> Bugs: SQOOP-3139
>     https://issues.apache.org/jira/browse/SQOOP-3139
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> SQOOP-3139: sqoop tries to re-execute select query during import in case of a connection
reset error and this is causing lots of duplicate records from source
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/mapreduce/ImportJobBase.java 105917c 
> 
> Diff: https://reviews.apache.org/r/56984/diff/
> 
> 
> Testing
> -------
> 
> ant test
> 
> 
> Thanks,
> 
> Anna Szonyi
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message