spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tathagata Das <t...@databricks.com>
Subject Re: spark streaming performance
Date Thu, 09 Jul 2015 22:41:38 GMT
I am not sure why you are getting node_local and not process_local. Also
there is probably not a good documentation other than that configuration
page - http://spark.apache.org/docs/latest/configuration.html (search for
locality)

On Thu, Jul 9, 2015 at 5:51 AM, Michel Hubert <michelh@vsnsystemen.nl>
wrote:

>
>
>
>
>
>
> StorageLevel.MEMORY_ONLY
>
>
>
> Streaming / Batch Processing Statistics / Processing Time:
>
> Executor-cores NOT specified: 5s
>
> Executor-cores 8: 400  ms
>
>
>
> Processing time has significantly reduced with executor-cores set to 8.
>
> But what is the general rule of thumb for a good excecutor-cores settings?
>
>
>
> But 257 records/sec still isn’t very, or not?
>
> Statistics over last 95 processed batches Receiver Statistics
>
> ·         *Receiver*
>
> ·         *Status*
>
> ·         *Location*
>
> ·         *Records in last batch*
>
> ·         *[2015/07/09 14:49:47]*
>
> ·         *Minimum rate*
>
> ·         *[records/sec]*
>
> ·         *Median rate*
>
> ·         *[records/sec]*
>
> ·         *Maximum rate*
>
> ·         *[records/sec]*
>
> ·         *Last Error*
>
> SocketReceiver-0
>
> ACTIVE
>
> bfravicsvr81440-cld.opentsp.com
>
> 1352
>
> 0
>
> 257
>
> 279
>
> -
>
>
>
>
>
>
>
> “After that the somehow the system is not able to launch any process
> local task”
>
> Where can I look for an answer why some tasks are NODE_LOCAL and others
> are PROCESS_LOCAL?
>
> Where should I look for the reason the locality wait expires?
>
>
>  Details for Stage 147
>
> ·         *Total task time across all tasks:* 0.4 s
>
> ·         *Shuffle write:* 704.0 B / 26
>
>  *Show additional metrics*
>  Summary Metrics for 26 Completed Tasks
>
> *Metric*
>
> *Min*
>
> *25th percentile*
>
> *Median*
>
> *75th percentile*
>
> *Max*
>
> Duration
>
> 9 ms
>
> 11 ms
>
> 12 ms
>
> 24 ms
>
> 30 ms
>
> GC Time
>
> 0 ms
>
> 0 ms
>
> 0 ms
>
> 0 ms
>
> 0 ms
>
> Shuffle Write Size / Records
>
> 27.0 B / 1
>
> 27.0 B / 1
>
> 27.0 B / 1
>
> 27.0 B / 1
>
> 28.0 B / 1
>    Aggregated Metrics by Executor
>
> *Executor ID*
>
> *Address*
>
> *Task Time*
>
> *Total Tasks*
>
> *Failed Tasks*
>
> *Succeeded Tasks*
>
> *Shuffle Write Size / Records*
>
> 1
>
> xxx81440xxx:38882
>
> 0.8 s
>
> 25
>
> 0
>
> 25
>
> 677.0 B / 25
>
> 2
>
> xxx81441xxx:46832
>
> 45 ms
>
> 1
>
> 0
>
> 1
>
> 27.0 B / 1
>    Tasks
>
> *Index*
>
> *ID*
>
> *Attempt*
>
> *Status*
>
> *Locality Level*
>
> *Executor ID / Host*
>
> *Launch Time*
>
> *Duration*
>
> *GC Time*
>
> *Write Time*
>
> *Shuffle Write Size / Records*
>
> *Errors*
>
> 0
>
> 1029
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 30 ms
>
> 27.0 B / 1
>
> 2
>
> 1031
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 29 ms
>
> 27.0 B / 1
>
> 1
>
> 1030
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 30 ms
>
> 27.0 B / 1
>
> 5
>
> 1034
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 25 ms
>
> 27.0 B / 1
>
> 4
>
> 1033
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 28 ms
>
> 27.0 B / 1
>
> 3
>
> 1032
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 27 ms
>
> 27.0 B / 1
>
> 25
>
> 1036
>
> 0
>
> SUCCESS
>
> PROCESS_LOCAL
>
> 2 / xxx81441xxx
>
> 2015/07/09 14:43:55
>
> 23 ms
>
> 27.0 B / 1
>
> 6
>
> 1035
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 24 ms
>
> 27.0 B / 1
>
> 7
>
> 1037
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 10 ms
>
> 27.0 B / 1
>
> 8
>
> 1038
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 12 ms
>
> 27.0 B / 1
>
> 9
>
> 1039
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 11 ms
>
> 27.0 B / 1
>
> 10
>
> 1040
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 11 ms
>
> 27.0 B / 1
>
> 11
>
> 1041
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>
> 2015/07/09 14:43:55
>
> 11 ms
>
> 27.0 B / 1
>
> 12
>
> 1042
>
> 0
>
> SUCCESS
>
> NODE_LOCAL
>
> 1 / xxx81440xxx
>  ...
>
> [Message clipped]

Mime
View raw message