spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Piotr Kołaczkowski (JIRA) <j...@apache.org>
Subject [jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages
Date Mon, 05 May 2014 19:37:14 GMT

    [ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989882#comment-13989882
] 

Piotr Kołaczkowski commented on SPARK-1712:
-------------------------------------------

There are some logs from SparkWorker and SparkMaster:
{noformat}
INFO 21:32:45,654 SparkMaster: 14/05/05 21:32:45 INFO Slf4jLogger: Slf4jLogger started
 INFO 21:32:45,698 SparkMaster: 14/05/05 21:32:45 INFO Remoting: Starting remoting
 INFO 21:32:45,849 SparkMaster: 14/05/05 21:32:45 INFO Remoting: Remoting started; listening
on addresses :[akka.tcp://sparkMaster@127.0.0.1:7077]
 INFO 21:32:46,112 SparkMaster: 14/05/05 21:32:46 INFO Master: Starting Spark master at spark://127.0.0.1:7077
 INFO 21:32:46,136 SparkMaster: 14/05/05 21:32:46 INFO Server: jetty-7.6.8.v20121106
 INFO 21:32:46,142 SparkMaster: 14/05/05 21:32:46 INFO ContextHandler: started o.e.j.s.h.ContextHandler{/metrics/master/json,null}
 INFO 21:32:46,142 SparkMaster: 14/05/05 21:32:46 INFO ContextHandler: started o.e.j.s.h.ContextHandler{/metrics/applications/json,null}
 INFO 21:32:46,143 SparkMaster: 14/05/05 21:32:46 INFO ContextHandler: started o.e.j.s.h.ContextHandler{/static,null}
 INFO 21:32:46,143 SparkMaster: 14/05/05 21:32:46 INFO ContextHandler: started o.e.j.s.h.ContextHandler{/app/json,null}
 INFO 21:32:46,143 SparkMaster: 14/05/05 21:32:46 INFO ContextHandler: started o.e.j.s.h.ContextHandler{/app,null}
 INFO 21:32:46,143 SparkMaster: 14/05/05 21:32:46 INFO ContextHandler: started o.e.j.s.h.ContextHandler{/json,null}
 INFO 21:32:46,143 SparkMaster: 14/05/05 21:32:46 INFO ContextHandler: started o.e.j.s.h.ContextHandler{*,null}
 INFO 21:32:46,152 SparkMaster: 14/05/05 21:32:46 INFO AbstractConnector: Started SelectChannelConnector@0.0.0.0:7080
 INFO 21:32:46,153 SparkMaster: 14/05/05 21:32:46 INFO MasterWebUI: Started Master web UI
at http://m4600.local:7080
 INFO 21:32:46,169 SparkMaster: 14/05/05 21:32:46 INFO Master: I have been elected leader!
New state: ALIVE
 INFO 21:32:46,474 Started SparkWork connected to 127.0.0.1:7077
 INFO 21:32:46,994 SparkWorker: 14/05/05 21:32:46 WARN Utils: Your hostname, m4600 resolves
to a loopback address: 127.0.0.2; using 192.168.122.1 instead (on interface virbr0)
 INFO 21:32:46,994 SparkWorker: 14/05/05 21:32:46 WARN Utils: Set SPARK_LOCAL_IP if you need
to bind to another address
 INFO 21:32:47,473 SparkWorker: 14/05/05 21:32:47 INFO Slf4jLogger: Slf4jLogger started
 INFO 21:32:47,519 SparkWorker: 14/05/05 21:32:47 INFO Remoting: Starting remoting
 INFO 21:32:47,661 SparkWorker: 14/05/05 21:32:47 INFO Remoting: Remoting started; listening
on addresses :[akka.tcp://sparkWorker@127.0.0.1:45858]
 INFO 21:32:47,827 SparkWorker: 14/05/05 21:32:47 INFO Worker: Starting Spark worker 127.0.0.1:45858
with 8 cores, 4.0 GB RAM
 INFO 21:32:47,828 SparkWorker: 14/05/05 21:32:47 INFO Worker: Spark home: /home/pkolaczk/Projekty/datastax/bdp/resources/spark
 INFO 21:32:47,945 SparkWorker: 14/05/05 21:32:47 INFO Server: jetty-7.6.8.v20121106
 INFO 21:32:47,951 SparkWorker: 14/05/05 21:32:47 INFO ContextHandler: started o.e.j.s.h.ContextHandler{/metrics/json,null}
 INFO 21:32:47,951 SparkWorker: 14/05/05 21:32:47 INFO ContextHandler: started o.e.j.s.h.ContextHandler{/static,null}
 INFO 21:32:47,952 SparkWorker: 14/05/05 21:32:47 INFO ContextHandler: started o.e.j.s.h.ContextHandler{/log,null}
 INFO 21:32:47,952 SparkWorker: 14/05/05 21:32:47 INFO ContextHandler: started o.e.j.s.h.ContextHandler{/logPage,null}
 INFO 21:32:47,952 SparkWorker: 14/05/05 21:32:47 INFO ContextHandler: started o.e.j.s.h.ContextHandler{/json,null}
 INFO 21:32:47,952 SparkWorker: 14/05/05 21:32:47 INFO ContextHandler: started o.e.j.s.h.ContextHandler{*,null}
 INFO 21:32:47,961 SparkWorker: 14/05/05 21:32:47 INFO AbstractConnector: Started SelectChannelConnector@0.0.0.0:7081
 INFO 21:32:47,962 SparkWorker: 14/05/05 21:32:47 INFO WorkerWebUI: Started Worker web UI
at http://m4600.local:7081
 INFO 21:32:47,963 SparkWorker: 14/05/05 21:32:47 INFO Worker: Connecting to master spark://127.0.0.1:7077...
 INFO 21:32:48,199 SparkMaster: 14/05/05 21:32:48 INFO Master: Registering worker 127.0.0.1:45858
with 8 cores, 4.0 GB RAM
 INFO 21:32:48,225 SparkWorker: 14/05/05 21:32:48 INFO Worker: Successfully registered with
master spark://127.0.0.1:7077
 INFO 21:33:00,119 SparkMaster: 14/05/05 21:33:00 INFO Master: Registering app Spark shell
 INFO 21:33:00,124 SparkMaster: 14/05/05 21:33:00 INFO Master: Registered app Spark shell
with ID app-20140505213300-0000
 INFO 21:33:00,145 SparkMaster: 14/05/05 21:33:00 INFO Master: Launching executor app-20140505213300-0000/0
on worker worker-20140505213247-127.0.0.1-45858
 INFO 21:33:00,185 SparkWorker: 14/05/05 21:33:00 INFO Worker: Asked to launch executor app-20140505213300-0000/0
for Spark shell
 INFO 21:33:00,812 SparkWorker: 14/05/05 21:33:00 INFO ExecutorRunner: Launch command: "/opt/jdk/bin/java"
"-cp" ":/home/pkolaczk/Projekty/datastax/bdp/build/dse-4.5.0-SNAPSHOT.jar:/home/pkolaczk/Projekty/datastax/bdp/build/maven-ant-tasks-2.1.3.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/cassandra-driver-core-2.0.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/commons-codec-1.6.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/commons-io-2.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/guava-15.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/HdrHistogram-1.0.9.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/java-uuid-generator-3.1.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/jbcrypt-0.3m.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/jline-1.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/jna-3.4.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/journalio-1.4.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/log4j-1.2.17.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/metrics-core-3.0.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/netty-3.9.0.Final.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/netty-all-4.0.13.Final.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/slf4j-api-1.7.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/slf4j-log4j12-1.7.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/conf:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/conf:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/conf:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/tools/lib/stress.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/antlr-2.7.7.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/antlr-3.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/antlr-runtime-3.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/cassandra-all-2.0.7.31.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/cassandra-clientutil-2.0.7.31.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/cassandra-thrift-2.0.7.31.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/commons-cli-1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/commons-codec-1.6.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/commons-lang-2.6.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/commons-lang3-3.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/commons-logging-1.1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/compress-lzf-0.8.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/concurrentlinkedhashmap-lru-1.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/disruptor-3.0.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/elephant-bird-hadoop-compat-4.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/guava-15.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/hibernate-validator-4.3.0.Final.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/high-scale-lib-1.1.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/httpclient-4.2.5.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/httpcore-4.2.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/jackson-core-asl-1.9.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/jamm-0.2.5.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/jbcrypt-0.3m.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/joda-time-1.6.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/json-simple-1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/libthrift-0.9.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/log4j-1.2.16.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/lz4-1.2.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/metrics-core-2.2.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/netty-3.6.6.Final.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/reporter-config-2.1.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/slf4j-api-1.7.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/snakeyaml-1.11.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/snappy-java-1.0.5.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/snaptree-0.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/stringtemplate-3.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/super-csv-2.1.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/thrift-server-0.3.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/validation-api-1.0.0.GA.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/../driver/lib/cassandra-driver-core-2.0.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/../driver/lib/cassandra-driver-dse-2.0.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/../driver/lib/metrics-core-3.0.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/../driver/lib/netty-3.9.0.Final.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/../driver/lib/slf4j-api-1.7.5.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/slf4j-api-1.7.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/dse/lib/slf4j-log4j12-1.7.2.jar:::/home/pkolaczk/Projekty/datastax/bdp/resources/spark/conf:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/activation-1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/akka-actor_2.10-2.2.3-shaded-protobuf.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/akka-remote_2.10-2.2.3-shaded-protobuf.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/akka-slf4j_2.10-2.2.3-shaded-protobuf.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/akka-zeromq_2.10-2.2.3-shaded-protobuf.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/algebird-core_2.10-0.1.11.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/asm-4.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/asm-commons-4.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/asm-tree-4.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/avro-1.7.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/avro-ipc-1.7.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/calliope_2.10-0.9.0-EA.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/chill_2.10-0.3.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/chill-java-0.3.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/colt-1.2.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-beanutils-1.7.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-beanutils-core-1.8.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-cli-1.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-codec-1.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-collections-3.2.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-compress-1.4.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-configuration-1.6.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-digester-1.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-el-1.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-httpclient-3.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-io-2.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-lang-2.5.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/commons-logging-1.1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/compress-lzf-1.0.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/concurrent-1.3.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/config-1.0.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/core-3.1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/fastutil-6.4.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/flume-ng-sdk-1.2.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/gmetric4j-1.0.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/guava-14.0.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/hadoop-client-1.0.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/hbase-0.94.6.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/high-scale-lib-1.1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/httpclient-4.1.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/httpcore-4.1.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jackson-annotations-2.2.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jackson-core-2.2.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jackson-core-asl-1.8.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jackson-databind-2.2.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jackson-jaxrs-1.8.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jackson-mapper-asl-1.8.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jackson-xc-1.8.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jamon-runtime-2.3.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jansi-1.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jasper-compiler-5.5.23.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jasper-runtime-5.5.23.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/JavaEWAH-0.6.6.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/java-xmlbuilder-0.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/javax.servlet-2.5.0.v201103041518.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jaxb-api-2.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jaxb-impl-2.2.3-1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jblas-1.2.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jersey-core-1.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jersey-json-1.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jersey-server-1.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jets3t-0.9.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jettison-1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jetty-6.1.26.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jetty-continuation-7.6.8.v20121106.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jetty-http-7.6.8.v20121106.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jetty-io-7.6.8.v20121106.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jetty-server-7.6.8.v20121106.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jetty-util-6.1.26.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jetty-util-7.6.8.v20121106.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jline-2.10.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jnr-constants-0.8.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jruby-complete-1.6.5.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jsp-2.1-6.1.14.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jsp-api-2.1-6.1.14.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jsr305-1.3.9.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/jul-to-slf4j-1.7.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/kafka_2.10-0.8.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/kryo-2.21.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/libthrift-0.7.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/lift-json_2.10-2.5.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/mesos-0.13.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/metrics-annotation-2.2.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/metrics-core-2.1.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/metrics-core-3.0.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/metrics-ganglia-3.0.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/metrics-graphite-3.0.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/metrics-json-3.0.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/metrics-jvm-3.0.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/minlog-1.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/mqtt-client-0.4.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/netty-3.5.9.Final.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/netty-all-4.0.13.Final.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/objenesis-1.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/oncrpc-1.0.7.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/paranamer-2.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/protobuf-java-2.4.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/protobuf-java-2.4.1-shaded.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/reflectasm-1.07-shaded.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/scala-compiler-2.10.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/scala-library-2.10.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/scala-reflect-2.10.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/servlet-api-2.5-20081211.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/servlet-api-2.5-6.1.14.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/spark-bagel_2.10-0.9.0-incubating.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/spark-core_2.10-0.9.0-incubating.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/spark-examples_2.10-0.9.0-incubating.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/spark-mllib_2.10-0.9.0-incubating.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/spark-repl_2.10-0.9.0-incubating.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/spark-streaming_2.10-0.9.0-incubating.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/spark-streaming-flume_2.10-0.9.0-incubating.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/spark-streaming-kafka_2.10-0.9.0-incubating.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/spark-streaming-mqtt_2.10-0.9.0-incubating.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/spark-streaming-twitter_2.10-0.9.0-incubating.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/spark-streaming-zeromq_2.10-0.9.0-incubating.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/stax-api-1.0.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/stream-2.4.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/twitter4j-core-3.0.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/twitter4j-stream-3.0.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/uncommons-maths-1.2.2a.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/velocity-1.7.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/xz-1.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/zeromq-scala-binding_2.10-0.0.7.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/zkclient-0.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/lib/zookeeper-3.4.5.jar:/home/pkolaczk/.spark/cassandra-context/spark-cassandra-context.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/spark/conf::/home/pkolaczk/Projekty/datastax/bdp/resources/shark/lib/commons-codec-1.6.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/shark/lib/commons-httpclient-3.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/shark/lib/commons-io-2.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/shark/lib/commons-logging-1.1.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/shark/lib/guava-14.0.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/shark/lib/hadoop-client-1.0.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/shark/lib/httpclient-4.3.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/shark/lib/httpcore-4.3.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/shark/lib/javax.servlet-2.5.0.v201103041518.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/shark/lib/jets3t-0.7.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/shark/lib/shark_2.10-0.9.0.1-DSP-3062-SNAPSHOT.jar::/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/conf:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/elephant-bird-hadoop-compat-4.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/hadoop-core-1.0.4.10.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/hadoop-examples-1.0.4.10.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/hadoop-fairscheduler-1.0.4.10.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/hadoop-streaming-1.0.4.10.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/hadoop-test-1.0.4.10.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/hadoop-tools-1.0.4.10.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/ant-1.6.5.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/automaton-1.11-8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-beanutils-1.7.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-beanutils-core-1.8.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-cli-1.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-codec-1.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-collections-3.2.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-configuration-1.6.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-digester-1.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-el-1.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-httpclient-3.0.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-lang-2.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-logging-1.1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-math-2.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/commons-net-1.4.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/core-3.1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/ftplet-api-1.0.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/ftpserver-core-1.0.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/ftpserver-deprecated-1.0.0-M2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/hsqldb-1.8.0.10.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/httpclient-4.1.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/httpcore-4.1.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/jackson-core-asl-1.8.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/jasper-compiler-5.5.12.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/jasper-runtime-5.5.12.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/java-xmlbuilder-0.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/jets3t-0.9.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/jetty-6.1.26.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/jetty-util-6.1.26.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/jsp-2.1-6.1.14.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/jsp-api-2.1-6.1.14.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/kfs-0.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/mina-core-2.0.0-M5.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/oro-2.0.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/servlet-api-2.5-20081211.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/servlet-api-2.5-6.1.14.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/snappy-java-1.0.5.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/lib/xmlenc-0.52.jar::/home/pkolaczk/Projekty/datastax/bdp/build/dse-4.5.0-SNAPSHOT.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/antlr-runtime-3.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop:/home/pkolaczk/Projekty/datastax/bdp/resources/hadoop/conf::/home/pkolaczk/Projekty/datastax/bdp/build/dse-4.5.0-SNAPSHOT.jar:/home/pkolaczk/Projekty/datastax/bdp/build/dse-4.5.0-SNAPSHOT.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/cassandra/lib/antlr-runtime-3.2.jar::/home/pkolaczk/Projekty/datastax/bdp/resources/hive/conf:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/antlr-2.7.7.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/antlr-3.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/antlr-runtime-3.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/asm-4.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/avro-1.7.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/avro-ipc-1.7.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/avro-mapred-1.7.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/bonecp-0.7.1.RELEASE.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-beanutils-1.7.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-beanutils-core-1.8.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-cli-1.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-codec-1.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-collections-3.2.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-compress-1.4.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-configuration-1.6.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-digester-1.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-io-2.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-lang-2.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-lang3-3.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-logging-1.1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-logging-api-1.0.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/commons-pool-1.5.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/datanucleus-api-jdo-3.2.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/datanucleus-core-3.2.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/datanucleus-rdbms-3.2.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/derby-10.4.2.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/guava-15.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/hive-cli-0.12.0.3-20140319.091653-2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/hive-common-0.12.0.3-20140319.091659-2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/hive-exec-0.12.0.3-20140319.091716-2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/hive-hwi-0.12.0.3-20140319.091745-2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/hive-jdbc-0.12.0.3-20140319.091754-2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/hive-metastore-0.12.0.3-20140319.091801-2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/hive-serde-0.12.0.3-20140319.091811-2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/hive-service-0.12.0.3-20140319.091817-2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/hive-shims-0.12.0.3-20140319.091825-2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/httpclient-4.2.5.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/httpcore-4.2.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/jackson-core-asl-1.8.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/jackson-mapper-asl-1.8.8.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/JavaEWAH-0.3.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/java-xmlbuilder-0.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/javolution-5.5.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/jdo-api-3.0.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/jets3t-0.9.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/jetty-util-6.1.26.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/json-20090211.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/jta-1.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/libfb303-0.9.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/libthrift-0.9.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/log4j-1.2.16.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/netty-3.5.9.Final.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/paranamer-2.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/protobuf-java-2.4.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/servlet-api-2.5-20081211.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/slf4j-api-1.6.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/snappy-0.2.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/snappy-java-1.0.5.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/ST4-4.0.4.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/stringtemplate-3.2.1.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/velocity-1.7.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/xz-1.0.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/lib/zookeeper-3.4.3.jar:/home/pkolaczk/Projekty/datastax/bdp/resources/hive/conf"
"-XX:MaxPermSize=256M" "-XX:MaxPermSize=256M" "-Xms2048M" "-Xmx2048M" "org.apache.spark.executor.CoarseGrainedExecutorBackend"
"akka.tcp://spark@m4600.local:45753/user/CoarseGrainedScheduler" "0" "127.0.0.1" "1" "akka.tcp://sparkWorker@127.0.0.1:45858/user/Worker"
"app-20140505213300-0000"
{noformat}

> ParallelCollectionRDD operations hanging forever without any error messages 
> ----------------------------------------------------------------------------
>
>                 Key: SPARK-1712
>                 URL: https://issues.apache.org/jira/browse/SPARK-1712
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 0.9.0
>         Environment: Linux Ubuntu 14.04, a single spark node; standalone mode.
>            Reporter: Piotr Kołaczkowski
>         Attachments: executor.jstack.txt, master.jstack.txt, repl.jstack.txt, spark-hang.png,
worker.jstack.txt
>
>
> {noformat}
> scala> val collection = (1 to 1000000).map(i => ("foo" + i, i)).toVector
> collection: Vector[(String, Int)] = Vector((foo1,1), (foo2,2), (foo3,3), (foo4,4), (foo5,5),
(foo6,6), (foo7,7), (foo8,8), (foo9,9), (foo10,10), (foo11,11), (foo12,12), (foo13,13), (foo14,14),
(foo15,15), (foo16,16), (foo17,17), (foo18,18), (foo19,19), (foo20,20), (foo21,21), (foo22,22),
(foo23,23), (foo24,24), (foo25,25), (foo26,26), (foo27,27), (foo28,28), (foo29,29), (foo30,30),
(foo31,31), (foo32,32), (foo33,33), (foo34,34), (foo35,35), (foo36,36), (foo37,37), (foo38,38),
(foo39,39), (foo40,40), (foo41,41), (foo42,42), (foo43,43), (foo44,44), (foo45,45), (foo46,46),
(foo47,47), (foo48,48), (foo49,49), (foo50,50), (foo51,51), (foo52,52), (foo53,53), (foo54,54),
(foo55,55), (foo56,56), (foo57,57), (foo58,58), (foo59,59), (foo60,60), (foo61,61), (foo62,62),
(foo63,63), (foo64,64), (foo...
> scala> val rdd = sc.parallelize(collection)
> rdd: org.apache.spark.rdd.RDD[(String, Int)] = ParallelCollectionRDD[0] at parallelize
at <console>:24
> scala> rdd.first
> res4: (String, Int) = (foo1,1)
> scala> rdd.map(_._2).sum
> // nothing happens
> {noformat}
> CPU and I/O idle. 
> Memory usage reported by JVM, after manually triggered GC:
> repl: 216 MB / 2 GB
> executor: 67 MB / 2 GB
> worker: 6 MB / 128 MB
> master: 6 MB / 128 MB
> No errors found in worker's stderr/stdout. 
> It works fine with 700,000 elements and then it takes about 1 second to process the request
and calculate the sum. With 700,000 items the spark executor memory doesn't even exceed 300
MB out of 2GB available. It fails with 800,000 items.
> Multiple parralelized collections of size 700,000 items at the same time in the same
session work fine.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message