sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-1870) IDF API and impementation package depednency
Date Tue, 09 Dec 2014 18:04:13 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Veena Basavaraj updated SQOOP-1870:
-----------------------------------
    Description: 
Currently the IDF API is in connector SDK package, 

The right place for IDF API is sqoop-spi package since custom implementations of the IDF can
be done in future.

Instead today sqoop SPI that holds connectors depends on SDK which in fact should be the other
way round, the connector API is the root and should not depend on its implemenation package.



connector SDK was intended to hold concrete implementations and some extent utility classes
that  the connectors, there is obvioulsy some leaks and execution engine even depends on connecto-sdk



  was:
1. update the java docs for IDF apis.

2.  Make the getTextData final and call it getCSV and setCSV, so it is obvious that we want
to enforce CSV format

 the following code can move to the base class IntermediateDataFormat and made final, so there
is no way to override this and we can enforce all to return String instead of generic T
{code}
// hold the string in IDF base class

 private final String text.
 
  public final String getCSVTextData() {
    return text;
  }

 
  public final void setCSVTextData(String text) {
    this.text = text;
  }
{code}

There is code in CSVIDF implementation that has the rules for CSV parsing that can be pulled
out into CSV Utils so that the connectors can use
The T in CSV happens to String, which is just a coincidence, If I write a new IDF implementation
T can be a custom object that could encapsulate the whole row.
Third, getData and setData can have custom implementation so they can be overriden to return
the generic type T

Correction :

{code}
// hold the string in IDF base class, is !final

 private String text.
 
  public final String getCSVTextData() {
    return text;
  }

 
  public final void setCSVTextData(String text) {
    this.text = text;
  }
{code}


> IDF API and impementation package depednency
> --------------------------------------------
>
>                 Key: SQOOP-1870
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1870
>             Project: Sqoop
>          Issue Type: Sub-task
>          Components: sqoop2-framework
>            Reporter: Veena Basavaraj
>            Assignee: Veena Basavaraj
>             Fix For: 1.99.5
>
>
> Currently the IDF API is in connector SDK package, 
> The right place for IDF API is sqoop-spi package since custom implementations of the
IDF can be done in future.
> Instead today sqoop SPI that holds connectors depends on SDK which in fact should be
the other way round, the connector API is the root and should not depend on its implemenation
package.
> connector SDK was intended to hold concrete implementations and some extent utility classes
that  the connectors, there is obvioulsy some leaks and execution engine even depends on connecto-sdk



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message