spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michel Hubert <mich...@phact.nl>
Subject RE: apache spark errors
Date Thu, 24 Mar 2016 13:40:19 GMT
<dependencies>
    <dependency> <!-- Spark dependency -->
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.10</artifactId>
        <version>1.6.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming_2.10</artifactId>
        <version>1.6.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming-kafka_2.10</artifactId>
        <version>1.6.1</version>
    </dependency>

    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>2.2.0</version>
    </dependency>

    <dependency>
        <groupId>org.apache.kafka</groupId>
       <artifactId>kafka_2.10</artifactId>
        <version>0.8.2.2</version>
    </dependency>


    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch-spark_2.10</artifactId>
        <version>2.2.0</version>
    </dependency>
    <dependency>
        <groupId>redis.clients</groupId>
        <artifactId>jedis</artifactId>
        <version>2.8.0</version>
        <type>jar</type>
        <scope>compile</scope>
    </dependency>
</dependencies>


How can I look at those tasks?

Van: Ted Yu [mailto:yuzhihong@gmail.com]
Verzonden: donderdag 24 maart 2016 14:33
Aan: Michel Hubert <michelh@phact.nl>
CC: user@spark.apache.org
Onderwerp: Re: apache spark errors

Which release of Spark are you using ?

Have you looked the tasks whose Ids were printed to see if there was more clue ?

Thanks

On Thu, Mar 24, 2016 at 6:12 AM, Michel Hubert <michelh@phact.nl<mailto:michelh@phact.nl>>
wrote:
HI,

I constantly get these errors:

0    [Executor task launch worker-15] ERROR org.apache.spark.executor.Executor  - Managed
memory leak detected; size = 6564500 bytes, TID = 38969
310002 [Executor task launch worker-12] ERROR org.apache.spark.executor.Executor  - Managed
memory leak detected; size = 5523550 bytes, TID = 43270
318445 [Executor task launch worker-12] ERROR org.apache.spark.executor.Executor  - Managed
memory leak detected; size = 6879566 bytes, TID = 43408
388893 [Executor task launch worker-12] ERROR org.apache.spark.executor.Executor  - Managed
memory leak detected; size = 5572546 bytes, TID = 44382
418186 [Executor task launch worker-13] ERROR org.apache.spark.executor.Executor  - Managed
memory leak detected; size = 5289222 bytes, TID = 44795
488421 [Executor task launch worker-4] ERROR org.apache.spark.executor.Executor  - Managed
memory leak detected; size = 8738142 bytes, TID = 45769
619276 [Executor task launch worker-4] ERROR org.apache.spark.executor.Executor  - Managed
memory leak detected; size = 5759312 bytes, TID = 47571
632275 [Executor task launch worker-12] ERROR org.apache.spark.executor.Executor  - Managed
memory leak detected; size = 5602240 bytes, TID = 47709
644989 [Executor task launch worker-13] ERROR org.apache.spark.executor.Executor  - Managed
memory leak detected; size = 5326260 bytes, TID = 47863
720701 [Executor task launch worker-12] ERROR org.apache.spark.executor.Executor  - Managed
memory leak detected; size = 5399578 bytes, TID = 48959
1147961 [Executor task launch worker-16] ERROR org.apache.spark.executor.Executor  - Managed
memory leak detected; size = 5251872 bytes, TID = 54922


How can I fix this?

With kind regard,

Michel

Mime
View raw message