sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From burakkk <burak.isi...@gmail.com>
Subject Sqoop Memory Exception
Date Wed, 23 Jul 2014 15:56:42 GMT
Hi,
I'm using sqoop 1.4.3. I'm trying to export to Oracle. I got an connection
reset error.  Using the sqoop user guide I found a solution about it (
http://sqoop.apache.org/docs/1.4.3/SqoopUserGuide.html#_oracle_connection_reset_errors)
but in order to solve it, I need to use urandom. However if I use urandom,
sqoop uses up to 32gb memory and then the job fails.

sqoop export -D oraoop.disabled=true -D
mapred.child.java.opts="-Djava.security.egd=file:/dev/../dev/urandom"
 --connect $XDDS_CONNECT --export-dir
$HDFS_DATA_PATH/nordwh/output_join_call_drop_churn_20140723 --table
DDS_USAGE.NWK_CALL_DROP_CHRN_DELTA --columns
contract_sk,msisdn_number,txn_date,traffic_tp_sk,network_tp_sk,destination_operator_sk,call_group_tp_sk,geography_city_sk,geography_sk,b_number,b_contract_sk,txn_tp_sk,initial_etl_date,modification_systime
--input-fields-terminated-by '\t' --num-mappers 80

TaskTree [pid=42014,tipID=attempt_201405211054_2089_m_000000_0] is
running beyond memory-limits. Current usage : 32544432128bytes. Limit
: 2147483648bytes. Killing task. TaskTree
[pid=42014,tipID=attempt_201405211054_2089_m_000000_0] is running
beyond memory-limits. Current usage : 32544432128bytes. Limit :
2147483648bytes. Killing task.

Do you have any idea what the cause is and how I'm gonna solve it?

Thanks
Best regards...


-- 

*BURAK ISIKLI* | *http://burakisikli.wordpress.com
<http://burakisikli.wordpress.com>*

Mime
View raw message