sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-1717) Sqoop2: Remove Data class from docs
Date Mon, 17 Nov 2014 01:56:34 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Veena Basavaraj updated SQOOP-1717:
-----------------------------------
    Attachment:     (was: SQOOP-1717.patch)

> Sqoop2: Remove Data class from docs
> -----------------------------------
>
>                 Key: SQOOP-1717
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1717
>             Project: Sqoop
>          Issue Type: Bug
>          Components: docs
>    Affects Versions: 1.99.5
>            Reporter: Abraham Elmahrek
>            Assignee: Veena Basavaraj
>             Fix For: 1.99.5
>
>         Attachments: SQOOP-1717.patch
>
>
> From ConnectorDevelopment.java:
> {noformat}
> The diagram below describes the initialization phase of a job.
> ``SqoopInputFormat`` create splits using ``Partitioner``.
> ::
>       ,----------------.          ,-----------.
>       |SqoopInputFormat|          |Partitioner|
>       `-------+--------'          `-----+-----'
>    getSplits  |                         |
>   ----------->|                         |
>               |      getPartitions      |
>               |------------------------>|
>               |                         |         ,---------.
>               |                         |-------> |Partition|
>               |                         |         `----+----'
>               |<- - - - - - - - - - - - |              |
>               |                         |              |          ,----------.
>               |-------------------------------------------------->|SqoopSplit|
>               |                         |              |          `----+-----'
> The diagram below describes the map phase of a job.
> ``SqoopMapper`` invokes ``From`` connector's extractor's ``extract`` method.
> ::
>       ,-----------.
>       |SqoopMapper|
>       `-----+-----'
>      run    |
>   --------->|                                   ,------------------.
>             |---------------------------------->|SqoopMapDataWriter|
>             |                                   `------+-----------'
>             |                ,---------.               |
>             |--------------> |Extractor|               |
>             |                `----+----'               |
>             |      extract        |                    |
>             |-------------------->|                    |
>             |                     |                    |
>            read from DB           |                    |
>   <-------------------------------|      write*        |
>             |                     |------------------->|
>             |                     |                    |           ,----.
>             |                     |                    |---------->|Data|
>             |                     |                    |           `-+--'
>             |                     |                    |
>             |                     |                    |      context.write
>             |                     |                    |-------------------------->
> The diagram below decribes the reduce phase of a job.
> ``OutputFormat`` invokes ``To`` connector's loader's ``load`` method (via ``SqoopOutputFormatLoadExecutor``
).
> ::
>     ,------------.  ,---------------------.
>     |SqoopReducer|  |SqoopNullOutputFormat|
>     `---+--------'  `----------+----------'
>         |                 |   ,-----------------------------.
>         |                 |-> |SqoopOutputFormatLoadExecutor|
>         |                 |   `--------------+--------------'        ,----.
>         |                 |                  |---------------------> |Data|
>         |                 |                  |                       `-+--'
>         |                 |                  |   ,-----------------.   |
>         |                 |                  |-> |SqoopRecordWriter|   |
>       getRecordWriter     |                  |   `--------+--------'   |
>   ----------------------->| getRecordWriter  |            |            |
>         |                 |----------------->|            |            |     ,--------------.
>         |                 |                  |-----------------------------> |ConsumerThread|
>         |                 |                  |            |            |     `------+-------'
>         |                 |<- - - - - - - - -|            |            |         
  |    ,------.
>   <- - - - - - - - - - - -|                  |            |            |         
  |--->|Loader|
>         |                 |                  |            |            |            |
   `--+---'
>         |                 |                  |            |            |            |
      |
>         |                 |                  |            |            |            |
load  |
>    run  |                 |                  |            |            |            |------>|
>   ----->|                 |     write        |            |            |         
  |       |
>         |------------------------------------------------>| setContent |         
  | read* |
>         |                 |                  |            |----------->| getContent
|<------|
>         |                 |                  |            |            |<-----------|
      |
>         |                 |                  |            |            |            |
- - ->|
>         |                 |                  |            |            |            |
      | write into DB
>         |                 |                  |            |            |            |
      |-------------->
> .. _`Intermediate representation`: https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+Intermediate+representation
> {noformat}
> Notice the use of the "Data" entity.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message