sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SQOOP-1811) IDF API changes
Date Mon, 01 Dec 2014 17:52:12 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230096#comment-14230096
] 

Veena Basavaraj edited comment on SQOOP-1811 at 12/1/14 5:52 PM:
-----------------------------------------------------------------

I am very confused at this point. 

Earlier you suggest that CSV is not mandatory.
>>>
When using methods setData() and setObjectData() the IDF implementation is responsible to
convert given data into CSV-ish text and call method setSqoopCSVString(). This will mean that
we will always convert the data into CSV-ish text, regardless whether we need that or not.


The above was the source of confusion: when I inferred that it may not be always necessary.



In the last comment it isa said it is required. So kindly provide an example of a AvroIDF
might look like?


was (Author: vybs):

I am very confused at this point. 

Earlier you suggest that CSV is not mandatory.

When using methods setData() and setObjectData() the IDF implementation is responsible to
convert given data into CSV-ish text and call method setSqoopCSVString(). This will mean that
we will always convert the data into CSV-ish text, regardless whether we need that or not.


In the last comment it isa said it is required. So kindly provide an example of a AvroIDF
might look like?

> IDF API changes
> ---------------
>
>                 Key: SQOOP-1811
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1811
>             Project: Sqoop
>          Issue Type: Sub-task
>          Components: sqoop2-framework
>            Reporter: Veena Basavaraj
>             Fix For: 1.99.5
>
>
> 1. update the java docs for IDF apis.
> 2.  Make the getTextData final and call it getCSV and setCSV, so it is obvious that we
want to enforce CSV format
>  the following code can move to the base class IntermediateDataFormat and made final,
so there is no way to override this and we can enforce all to return String instead of generic
T
> {code}
> // hold the string in IDF base class
>  private final String text.
>  
>   public final String getCSVTextData() {
>     return text;
>   }
>  
>   public final void setCSVTextData(String text) {
>     this.text = text;
>   }
> {code}
> There is code in CSVIDF implementation that has the rules for CSV parsing that can be
pulled out into CSV Utils so that the connectors can use
> The T in CSV happens to String, which is just a coincidence, If I write a new IDF implementation
T can be a custom object that could encapsulate the whole row.
> Third, getData and setData can have custom implementation so they can be overriden to
return the generic type T
> Correction :
> {code}
> // hold the string in IDF base class, is !final
>  private String text.
>  
>   public final String getCSVTextData() {
>     return text;
>   }
>  
>   public final void setCSVTextData(String text) {
>     this.text = text;
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message