sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SQOOP-1870) IDF API and impementation package depednency
Date Mon, 22 Dec 2014 20:39:14 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14255983#comment-14255983
] 

Veena Basavaraj edited comment on SQOOP-1870 at 12/22/14 8:38 PM:
------------------------------------------------------------------

[~abec] and I discussed offline.

We agree on what I had suggested before, that SPI should not depend on SDK, we also agreed
to move the IDF and reference implementation CSVIDF ( which is used as the default implementation
in the IDF base class to the sqoop-spi) as one alternative.

Abe suggested that we try to merge sqoop-api and connector-sdk together and call connector-sdk,
assuming we do not create any other cyclic dependency. So the next action item is to try to
merge the two into one package and get rid of sqoop-api all together. 

The connector-sdk will have 
1. sqoop connector and the connector lifecycle interfaces/ base classes such as Extractor,
Loader, etc
2. Matcher API and its implementations
3. IDF API and its implementations

If this creates any cyclic dependency then we will revisit the discussion again.

[~abec] will follow up another ticket  SQOOP-1926  to add annotations for connector apis for
stability etc. There is a ticket already created for that.


was (Author: vybs):
[~abec] and I discussed offline.

We agree on what I had suggested before, that SPI should not depend on SDK, we also agreed
to move the IDF and reference implementation CSVIDF ( which is used as the default implementation
in the IDF base class to the sqoop-spi) as one alternative.

Abe suggested that we try to merge sqoop-api and connector-sdk together and call connector-sdk,
assuming we do not create any other cyclic dependency. So the next action item is to try to
merge the two into one package and get rid of sqoop-api all together. 

The connector-sdk will have 
1. sqoop connector and the connector lifecycle interfaces/ base classes such as Extractor,
Loader, etc
2. Matcher API and its implementations
3. IDF API and its implementations

If this creates any cyclic dependency then we will revisit the discussion again.

[~abec] will follow up another ticket to add annotations for connector apis for stability
etc. There is a ticket already created for that.

> IDF API and impementation package depednency
> --------------------------------------------
>
>                 Key: SQOOP-1870
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1870
>             Project: Sqoop
>          Issue Type: Bug
>          Components: sqoop2-framework
>            Reporter: Veena Basavaraj
>            Assignee: Veena Basavaraj
>             Fix For: 1.99.5
>
>
> Currently the IDF API is in connector SDK package, 
> The right place for IDF API is sqoop-spi package since custom implementations of the
IDF can be done in future.
> Instead today sqoop SPI that holds connectors depends on SDK which in fact should be
the other way round, the connector API is the root and should not depend on its implemenation
package.
> connector SDK was intended to hold concrete implementations and some extent utility classes
that  the connectors use, there is obvioulsy some leaks and execution engine even depends
on connecto-rsdk



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message