sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ron Gonzalez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-2453) Running sqoop with --as-parquet option always fails the container with a memory usage error
Date Thu, 30 Jul 2015 18:01:04 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-2453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14648035#comment-14648035
] 

Ron Gonzalez commented on SQOOP-2453:
-------------------------------------

The entire data set with compression (I think default is snappy) was around 1 GB. By running
it as a LocalJobRunner, I was able to get it to complete.

> Running sqoop with --as-parquet option always fails the container with a memory usage
error
> -------------------------------------------------------------------------------------------
>
>                 Key: SQOOP-2453
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2453
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.4.6
>         Environment: Ubuntu Linux 64 bit
>            Reporter: Ron Gonzalez
>
> I tried invoking sqoop with the --as-parquet option. The default container size with
the --as-avro option could handle any sized data load, but with the parquet option, it always
fails with Yarn killing the mapper containers. I've increased the container size to 2 GB to
no avail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message