sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Masatake Iwasaki (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1411) The number of tasks is not set properly in PGBulkloadExportManager
Date Mon, 04 Aug 2014 19:32:12 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085159#comment-14085159
] 

Masatake Iwasaki commented on SQOOP-1411:
-----------------------------------------

bq. the code no longer calls setJobNumMappers and removes an important validation and warning.
The number of map tasks is set in ExportJobBase#configureNumTasks which is called from PGBulkloadExportJob.
{code}
    int numMaps = super.configureNumTasks(job);
{code}
I would like not copying and pasting the code in configureNumTasks but calling super.configureNumTasks
in order to follow future changes.

bq. Why rename a parameter? What was wrong with the existing name?
Because the value of mapred.reduce.tasks (which is deprecated by mapreduce.job.reduces) are
set to 0 in JobBase#configureNumTasks. 
{code}
    job.setNumReduceTasks(0);
{code}

> The number of tasks is not set properly in PGBulkloadExportManager
> ------------------------------------------------------------------
>
>                 Key: SQOOP-1411
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1411
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors
>    Affects Versions: 1.4.5
>            Reporter: Masatake Iwasaki
>            Assignee: Masatake Iwasaki
>         Attachments: SQOOP-1411-0.patch
>
>
> The '-m' option does not work and the number of reduce tasks is always set to 0. 
> The cause seems to be the changes of configureNumTasks in ExportJobBase and JobBase.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message