sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Cecho" <jar...@apache.org>
Subject Re: Review Request 27543: DOC: Connector SDK docs + validation to be updated
Date Tue, 04 Nov 2014 03:08:36 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27543/#review59718
-----------------------------------------------------------


Good material Veena! I have couple of notes below:


docs/src/site/sphinx/ConnectorDevelopment.rst
<https://reviews.apache.org/r/27543/#comment101023>

    I think that we should mention here that "from" and "to" job can have different inputs.



docs/src/site/sphinx/ConnectorDevelopment.rst
<https://reviews.apache.org/r/27543/#comment101025>

    We should not be talking about a Mapreduce job here - that is specific to the execution
engine. I would say that initializer is executed first before any other action.



docs/src/site/sphinx/ConnectorDevelopment.rst
<https://reviews.apache.org/r/27543/#comment101024>

    I believe that we do required empty Schema right? Null can actually throw a NullPointerException
at some places (not 100% sure after recent code changes).



docs/src/site/sphinx/ConnectorDevelopment.rst
<https://reviews.apache.org/r/27543/#comment101026>

    Let's not confuse the connector developer with Driver here. Let's just state that they
are suppose to create 0..N partitions where N is driven by user.
    
    Similarly I don't think that we should talk about mappers and reducers here as that is
a specific to one single execution engine (albeit the only one right now). Perhaps we can
add some explanation block to say "hey in MR exec engine it's implemented this way" as an
example?



docs/src/site/sphinx/ConnectorDevelopment.rst
<https://reviews.apache.org/r/27543/#comment101027>

    ... must implement... 
    
    Might be better wording (nit).



docs/src/site/sphinx/ConnectorDevelopment.rst
<https://reviews.apache.org/r/27543/#comment101028>

    The extractor needs to iterate through the entire partition, not data set. All partitions
together will give entire dataset.



docs/src/site/sphinx/ConnectorDevelopment.rst
<https://reviews.apache.org/r/27543/#comment101029>

    Nit: Typo in the words "developers"
    
    Also the loader technically runs in the output committer and not on the reducer side.
That is important because we can run map-only job and still run the loader.



docs/src/site/sphinx/ConnectorDevelopment.rst
<https://reviews.apache.org/r/27543/#comment101030>

    Two @ signs in teh ConfigClass



docs/src/site/sphinx/ConnectorDevelopment.rst
<https://reviews.apache.org/r/27543/#comment101031>

    Nit: Typo in "Validatora"



docs/src/site/sphinx/ConnectorDevelopment.rst
<https://reviews.apache.org/r/27543/#comment101033>

    Typo "wiork"


- Jarek Cecho


On Nov. 4, 2014, 3:07 a.m., Veena Basavaraj wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27543/
> -----------------------------------------------------------
> 
> (Updated Nov. 4, 2014, 3:07 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Repository: sqoop-sqoop2
> 
> 
> Description
> -------
> 
> I have taken liberty to write some stuff, happy to amend it!
> 
> 
> I will remove the WS once the cotnents are approved. Typing some of this at the qcon:)
> 
> 
> Diffs
> -----
> 
>   docs/src/site/sphinx/ConnectorDevelopment.rst e4b5402 
> 
> Diff: https://reviews.apache.org/r/27543/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Veena Basavaraj
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message