sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bilung Lee" <bleeapa...@gmail.com>
Subject Review Request: SQOOP-588 Provide MapReduce classes for executing ETL classes
Date Tue, 28 Aug 2012 20:25:38 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6811/
-----------------------------------------------------------

Review request for Sqoop.


Description
-------

This patch provides necessary MapReduce classes (split, input format, mapper, and output format)
for executing ETL classes (partition, partitioner, extractor, and loader).

Basically
- partition is used by split
- partitioner is invoked by input format
- extractor is invoked by mapper
- loader is invoked by output format


This addresses bug SQOOP-588.
    https://issues.apache.org/jira/browse/SQOOP-588


Diffs
-----

  branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportDestroyer.java
1373167 
  branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportInitializer.java
1373167 
  branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java
1373167 
  branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportDestroyer.java
1373167 
  branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java
1373167 
  branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportInitializer.java
1373167 
  branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartition.java
PRE-CREATION 
  branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java
1373167 
  branches/sqoop2/core/pom.xml 1373167 
  branches/sqoop2/core/src/main/java/org/apache/sqoop/core/CoreError.java 1373167 
  branches/sqoop2/core/src/main/java/org/apache/sqoop/job/JobConstants.java PRE-CREATION 
  branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlContext.java PRE-CREATION

  branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlMutableContext.java PRE-CREATION

  branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/Data.java PRE-CREATION 
  branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/FieldTypes.java PRE-CREATION

  branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopFileOutputFormat.java PRE-CREATION

  branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java PRE-CREATION

  branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java PRE-CREATION

  branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopNullOutputFormat.java PRE-CREATION

  branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java
PRE-CREATION 
  branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopSplit.java PRE-CREATION

  branches/sqoop2/core/src/test/java/org/apache/sqoop/job/TestMapReduce.java PRE-CREATION

  branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Context.java 1373167 
  branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Destroyer.java 1373167 
  branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java 1373167 
  branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Initializer.java 1373167 
  branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Loader.java 1373167 
  branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Partition.java 1373167 
  branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Partitioner.java 1373167 
  branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataReader.java 1373167 
  branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataWriter.java 1373167 

Diff: https://reviews.apache.org/r/6811/diff/


Testing
-------

Create a new TestMapReduce test case


Thanks,

Bilung Lee


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message