spark-user mailing list archives: March 2021

Site index · List index
Message list1 · 2 · 3 · Next »Thread · Author · Date
Re: Spark structured streaming Stuck on Batch = 0 on spark 3.1.1, Dataproc cluster
Mich Talebzadeh   Re: Spark structured streaming Stuck on Batch = 0 on spark 3.1.1, Dataproc cluster Mon, 01 Mar, 15:25
Mich Talebzadeh What is the latest stable release of Spark Mon, 01 Mar, 16:32
Sean Owen   Re: What is the latest stable release of Spark Mon, 01 Mar, 16:33
Mich Talebzadeh     Re: What is the latest stable release of Spark Mon, 01 Mar, 16:41
Sean Owen       Re: What is the latest stable release of Spark Mon, 01 Mar, 17:21
Mich Talebzadeh         Re: What is the latest stable release of Spark Mon, 01 Mar, 17:35
Sean Owen           Re: What is the latest stable release of Spark Mon, 01 Mar, 17:59
Mich Talebzadeh             Re: What is the latest stable release of Spark Mon, 01 Mar, 20:27
Jungtaek Lim               Re: What is the latest stable release of Spark Mon, 01 Mar, 23:02
Mich Talebzadeh                 Re: What is the latest stable release of Spark Mon, 01 Mar, 23:10
Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3
shiva   Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3 Mon, 01 Mar, 17:25
Mich Talebzadeh   Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3 Mon, 01 Mar, 17:50
shiva     Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3 Mon, 01 Mar, 17:59
Mich Talebzadeh       Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3 Mon, 01 Mar, 19:02
shiva         Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3 Tue, 02 Mar, 07:22
Mich Talebzadeh           Re: s3a staging committer(directory committer )not writing data to s3 bucket (final output directory) in spark3 Tue, 02 Mar, 12:08
Alma Maria Rinasz [Virtual meetup] March 11th 10am PST Mon, 01 Mar, 23:14
Ranju Jain Spark Version 3.0.1 Gui Display Query Tue, 02 Mar, 05:31
Kapil Garg   Re: Spark Version 3.0.1 Gui Display Query Tue, 02 Mar, 06:08
Ranju Jain     RE: Spark Version 3.0.1 Gui Display Query Tue, 02 Mar, 06:22
Ranju Jain       RE: Spark Version 3.0.1 Gui Display Query Tue, 02 Mar, 06:26
Kapil Garg         Re: Spark Version 3.0.1 Gui Display Query Tue, 02 Mar, 07:09
Ranju Jain           RE: Spark Version 3.0.1 Gui Display Query Wed, 03 Mar, 15:48
Attila Zsolt Piros             RE: Spark Version 3.0.1 Gui Display Query Wed, 03 Mar, 17:07
Ranju Jain               RE: Spark Version 3.0.1 Gui Display Query Thu, 04 Mar, 17:53
Attila Zsolt Piros                 RE: Spark Version 3.0.1 Gui Display Query Thu, 04 Mar, 18:08
Ranju Jain                   RE: Spark Version 3.0.1 Gui Display Query Fri, 05 Mar, 04:04
Kapil Garg                     Re: Spark Version 3.0.1 Gui Display Query Fri, 05 Mar, 04:30
Ranju Jain                   RE: Spark Version 3.0.1 Gui Display Query Sun, 07 Mar, 12:18
Attila Zsolt Piros                     Re: Spark Version 3.0.1 Gui Display Query Mon, 08 Mar, 13:56
Attila Zsolt Piros                     Re: Spark Version 3.0.1 Gui Display Query Mon, 08 Mar, 14:15
Ranju Jain                       RE: Spark Version 3.0.1 Gui Display Query Tue, 09 Mar, 02:07
Attila Zsolt Piros                         Re: Spark Version 3.0.1 Gui Display Query Tue, 09 Mar, 08:58
Ranju Jain                           RE: Spark Version 3.0.1 Gui Display Query Wed, 10 Mar, 08:50
Attila Zsolt Piros                             Re: Spark Version 3.0.1 Gui Display Query Wed, 10 Mar, 10:09
Ranju Jain                               RE: Spark Version 3.0.1 Gui Display Query Wed, 10 Mar, 10:19
Mich Talebzadeh   Re: Spark Version 3.0.1 Gui Display Query Thu, 04 Mar, 18:29
Sachit Murarka Spark job crashing - Spark Structured Streaming with Kafka Tue, 02 Mar, 08:25
Jungtaek Lim   Re: Spark job crashing - Spark Structured Streaming with Kafka Tue, 02 Mar, 09:11
Sachit Murarka     Re: Spark job crashing - Spark Structured Streaming with Kafka Tue, 02 Mar, 10:07
Jungtaek Lim       Re: Spark job crashing - Spark Structured Streaming with Kafka Wed, 03 Mar, 10:19
Mich Talebzadeh         Re: Spark job crashing - Spark Structured Streaming with Kafka Wed, 03 Mar, 10:37
Oldrich Vlasic [Spark SQL, intermediate+] possible bug or weird behavior of insertInto Tue, 02 Mar, 11:37
Sean Owen   Re: [Spark SQL, intermediate+] possible bug or weird behavior of insertInto Wed, 03 Mar, 14:29
Russell Spitzer     Re: [Spark SQL, intermediate+] possible bug or weird behavior of insertInto Wed, 03 Mar, 14:31
Oldrich Vlasic     Re: [Spark SQL, intermediate+] possible bug or weird behavior of insertInto Thu, 04 Mar, 09:15
Jeff Evans       Re: [Spark SQL, intermediate+] possible bug or weird behavior of insertInto Thu, 04 Mar, 13:55
Oldrich Vlasic       Re: [Spark SQL, intermediate+] possible bug or weird behavior of insertInto Thu, 04 Mar, 18:17
Re: Structured Streaming With Kafka - processing each event
Sachit Murarka   Re: Structured Streaming With Kafka - processing each event Tue, 02 Mar, 12:55
Gourav Sengupta     Re: Structured Streaming With Kafka - processing each event Wed, 03 Mar, 01:05
Mich Talebzadeh Please update this notification on Spark download Site Tue, 02 Mar, 15:09
Sean Owen   Re: Please update this notification on Spark download Site Tue, 02 Mar, 15:21
Hyukjin Kwon [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 01:41
Yuming Wang   Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 02:53
Mridul Muralidharan     Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 05:18
Takeshi Yamamuro       Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 07:04
Mich Talebzadeh         Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 07:14
Kent Yao       Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 07:50
Wenchen Fan         Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 09:01
angers zhu           Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 09:54
Jungtaek Lim             Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 10:14
Gabor Somogyi               Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 11:21
Dongjoon Hyun                 Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 17:26
Hyukjin Kwon                   Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Thu, 04 Mar, 00:09
Felix Cheung                   Re: [ANNOUNCE] Announcing Apache Spark 3.1.1 Sat, 06 Mar, 02:07
Bode, Meikel, NMA-CFD   AW: [ANNOUNCE] Announcing Apache Spark 3.1.1 Wed, 03 Mar, 07:23
Mich Talebzadeh Spark 3.1.1 Preliminary results (mainly to do with Spark Structured Streaming) Wed, 03 Mar, 09:35
rajat kumar Yaml for google spark kubernetes configmap Wed, 03 Mar, 14:32
Mich Talebzadeh Spark structured streaming seems to work on local mode only Wed, 03 Mar, 17:21
Sachit Murarka   Re: Spark structured streaming seems to work on local mode only Wed, 03 Mar, 17:29
Mich Talebzadeh     Re: Spark structured streaming seems to work on local mode only Wed, 03 Mar, 17:34
Mich Talebzadeh       Re: Spark structured streaming seems to work on local mode only Wed, 03 Mar, 20:56
Mich Talebzadeh Possible upgrade path from Spark 3.1.1-RC2 to Spark 3.1.1 GA Thu, 04 Mar, 14:15
Sean Owen   Re: Possible upgrade path from Spark 3.1.1-RC2 to Spark 3.1.1 GA Thu, 04 Mar, 14:39
Mich Talebzadeh     Re: Possible upgrade path from Spark 3.1.1-RC2 to Spark 3.1.1 GA Thu, 04 Mar, 14:44
吃完药感觉自己萌萌哒 退订 Fri, 05 Mar, 07:08
韩天罡   Re:退订 Fri, 05 Mar, 13:21
Thomas     Re: 退订 Sat, 06 Mar, 03:38
Dipl.-Inf. Rico Bergmann Structured Streaming Microbatch Semantics Fri, 05 Mar, 08:06
Roland Johann   Re: Structured Streaming Microbatch Semantics Fri, 05 Mar, 08:18
Dipl.-Inf. Rico Bergmann     Re: Structured Streaming Microbatch Semantics Fri, 05 Mar, 13:54
Mich Talebzadeh   Re: Structured Streaming Microbatch Semantics Fri, 05 Mar, 09:58
Dipl.-Inf. Rico Bergmann     Re: Structured Streaming Microbatch Semantics Fri, 05 Mar, 13:43
Mich Talebzadeh       Re: Structured Streaming Microbatch Semantics Fri, 05 Mar, 15:07
Dipl.-Inf. Rico Bergmann         Re: Structured Streaming Microbatch Semantics Mon, 08 Mar, 10:24
Mich Talebzadeh           Re: Structured Streaming Microbatch Semantics Mon, 08 Mar, 12:07
Mich Talebzadeh             Re: Structured Streaming Microbatch Semantics Mon, 08 Mar, 14:25
(无主题)
Sophia   (无主题) Fri, 05 Mar, 11:10
Sophia   (无主题) Sun, 07 Mar, 11:38
韩天罡   (无主题) Mon, 08 Mar, 08:58
lalitha bandaru Spark streaming with multiple Kafka topics Fri, 05 Mar, 19:28
Fwd: [jira] [Commented] (SPARK-34648) Reading Parquet Files in Spark Extremely Slow for Large Number of Files?
Pankaj Bhootra   Fwd: [jira] [Commented] (SPARK-34648) Reading Parquet Files in Spark Extremely Slow for Large Number of Files? Sat, 06 Mar, 17:22
Pankaj Bhootra     Re: [jira] [Commented] (SPARK-34648) Reading Parquet Files in Spark Extremely Slow for Large Number of Files? Wed, 10 Mar, 04:25
钟雨       Re: [jira] [Commented] (SPARK-34648) Reading Parquet Files in Spark Extremely Slow for Large Number of Files? Wed, 10 Mar, 13:59
Kent Yao       Re: [jira] [Commented] (SPARK-34648) Reading Parquet Files in Spark Extremely Slow for Large Number of Files? Wed, 10 Mar, 14:54
Sandeep Varma [No Subject] Sun, 07 Mar, 11:40
Ranju Jain Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Sun, 07 Mar, 16:22
Jacek Laskowski   Re: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Mon, 08 Mar, 10:43
Ranju Jain     RE: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Mon, 08 Mar, 11:33
Jacek Laskowski       Re: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Mon, 08 Mar, 14:03
Mich Talebzadeh   Re: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Mon, 08 Mar, 14:35
Ranju Jain     RE: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Mon, 08 Mar, 14:46
Mich Talebzadeh       Re: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Mon, 08 Mar, 15:01
Ranju Jain         RE: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Mon, 08 Mar, 15:19
Mich Talebzadeh           Re: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Mon, 08 Mar, 15:33
Mich Talebzadeh             Re: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Mon, 08 Mar, 15:37
Ranju Jain               RE: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Mon, 08 Mar, 16:18
Mich Talebzadeh                 Re: Spark 3.0.1 | Volume to use For Spark Kubernetes Executor Part Files Storage Mon, 08 Mar, 16:25
zenglong chen 退订 Mon, 08 Mar, 01:59
Sachit Murarka com.esotericsoftware.kryo.KryoException: java.io.IOException: No space left on device\n\t Mon, 08 Mar, 08:01
Gourav Sengupta   Re: com.esotericsoftware.kryo.KryoException: java.io.IOException: No space left on device\n\t Mon, 08 Mar, 08:23
Sachit Murarka     Re: com.esotericsoftware.kryo.KryoException: java.io.IOException: No space left on device\n\t Mon, 08 Mar, 08:25
Sean Owen   Re: com.esotericsoftware.kryo.KryoException: java.io.IOException: No space left on device\n\t Mon, 08 Mar, 12:52
Sachit Murarka     Re: com.esotericsoftware.kryo.KryoException: java.io.IOException: No space left on device\n\t Mon, 08 Mar, 17:23
韩天罡 退订 Mon, 08 Mar, 09:02
Re: How to control count / size of output files for
m li   Re: How to control count / size of output files for Mon, 08 Mar, 15:58
Gourav Sengupta     Re: How to control count / size of output files for Mon, 08 Mar, 16:31
m li       Re: How to control count / size of output files for Wed, 10 Mar, 15:04
Sachit Murarka Single executor processing all tasks in spark structured streaming kafka Mon, 08 Mar, 17:26
Kapil Garg   Re: Single executor processing all tasks in spark structured streaming kafka Mon, 08 Mar, 18:48
Sachit Murarka     Re: Single executor processing all tasks in spark structured streaming kafka Fri, 12 Mar, 03:17
Rich Bowen Call for Presentations for ApacheCon 2021 now open Mon, 08 Mar, 21:01
Mich Talebzadeh Creating spark context outside of the driver throws error Mon, 08 Mar, 21:02
Sean Owen   Re: Creating spark context outside of the driver throws error Mon, 08 Mar, 21:48
Mich Talebzadeh     Re: Creating spark context outside of the driver throws error Mon, 08 Mar, 22:31
Sergey Oboguev Detecting latecomer events in Spark structured streaming Mon, 08 Mar, 21:14
Jungtaek Lim   Re: Detecting latecomer events in Spark structured streaming Fri, 12 Mar, 04:36
Spark Streaming - Routing rdd to Executor based on Key
forece85   Spark Streaming - Routing rdd to Executor based on Key Tue, 09 Mar, 07:50
forece85   Spark Streaming - Routing rdd to Executor based on Key Tue, 09 Mar, 07:51
Ali Gouta     Re: Spark Streaming - Routing rdd to Executor based on Key Tue, 09 Mar, 13:28
Sean Owen       Re: Spark Streaming - Routing rdd to Executor based on Key Tue, 09 Mar, 13:32
forece85         Re: Spark Streaming - Routing rdd to Executor based on Key Tue, 09 Mar, 13:39
forece85       Re: Spark Streaming - Routing rdd to Executor based on Key Tue, 09 Mar, 13:38
Mich Talebzadeh Sounds like Structured streaming with foreach, can only run on one executor Tue, 09 Mar, 22:41
Sean Owen   Re: Sounds like Structured streaming with foreach, can only run on one executor Tue, 09 Mar, 22:50
Mich Talebzadeh     Re: Sounds like Structured streaming with foreach, can only run on one executor Tue, 09 Mar, 23:01
SRK Speed up Spark writes to Google Cloud storage Tue, 09 Mar, 23:40
jiahong li spark 3.1.1 support hive 1.2 Wed, 10 Mar, 02:49
Kent Yao   Re:spark 3.1.1 support hive 1.2 Wed, 10 Mar, 03:11
jiahong li     Re: spark 3.1.1 support hive 1.2 Wed, 10 Mar, 03:20
Muthu Jayakumar [spark-core] docker-image-tool.sh question... Wed, 10 Mar, 04:43
Attila Zsolt Piros   Re: [spark-core] docker-image-tool.sh question... Wed, 10 Mar, 17:41
Muthu Jayakumar     Re: [spark-core] docker-image-tool.sh question... Thu, 11 Mar, 04:26
jiahong li compile spark 3.1.1 error Wed, 10 Mar, 10:26
Attila Zsolt Piros   Re: compile spark 3.1.1 error Wed, 10 Mar, 10:55
jiahong li     Re: compile spark 3.1.1 error Wed, 10 Mar, 11:25
Attila Zsolt Piros       Re: compile spark 3.1.1 error Wed, 10 Mar, 12:56
jiahong li         Re: compile spark 3.1.1 error Thu, 11 Mar, 03:14
jiahong li           Re: compile spark 3.1.1 error Thu, 11 Mar, 04:04
Attila Zsolt Piros             Fwd: compile spark 3.1.1 error Fri, 12 Mar, 13:06
Kuttaiah Robin FlatMapGroupsWithStateFunction is called thrice - Production use case. Thu, 11 Mar, 07:54
Jungtaek Lim   Re: FlatMapGroupsWithStateFunction is called thrice - Production use case. Fri, 12 Mar, 03:13
Kuttaiah Robin     Re: FlatMapGroupsWithStateFunction is called thrice - Production use case. Thu, 18 Mar, 14:19
Ranju Jain Spark on Kubernetes | 3.0.1 | Shared Volume or NFS Thu, 11 Mar, 11:28
Mich Talebzadeh   Re: Spark on Kubernetes | 3.0.1 | Shared Volume or NFS Thu, 11 Mar, 11:52
Ranju Jain     RE: Spark on Kubernetes | 3.0.1 | Shared Volume or NFS Thu, 11 Mar, 12:01
Mich Talebzadeh       Re: Spark on Kubernetes | 3.0.1 | Shared Volume or NFS Thu, 11 Mar, 12:08
Ranju Jain         RE: Spark on Kubernetes | 3.0.1 | Shared Volume or NFS Thu, 11 Mar, 12:53
Mich Talebzadeh           Re: Spark on Kubernetes | 3.0.1 | Shared Volume or NFS Thu, 11 Mar, 17:37
Ranju Jain             RE: Spark on Kubernetes | 3.0.1 | Shared Volume or NFS Fri, 12 Mar, 03:08
yxl040840219 spark on k8s driver pod exception Thu, 11 Mar, 12:05
Attila Zsolt Piros   Re: spark on k8s driver pod exception Thu, 11 Mar, 21:05
Attila Zsolt Piros     Re: spark on k8s driver pod exception Thu, 11 Mar, 21:42
040840219       Re: spark on k8s driver pod exception Mon, 15 Mar, 07:44
Attila Zsolt Piros         Re: spark on k8s driver pod exception Mon, 15 Mar, 09:07
Renu Yadav How to upgrade kafka client in spark_streaming_kafka 2.2 Fri, 12 Mar, 08:44
Gabor Somogyi   Re: How to upgrade kafka client in spark_streaming_kafka 2.2 Fri, 12 Mar, 12:06
Renu Yadav     Re: How to upgrade kafka client in spark_streaming_kafka 2.2 Fri, 12 Mar, 13:55
Renu Yadav       Re: How to upgrade kafka client in spark_streaming_kafka 2.2 Fri, 12 Mar, 14:02
Gabor Somogyi         Re: How to upgrade kafka client in spark_streaming_kafka 2.2 Fri, 12 Mar, 14:07
Renu Yadav           Re: How to upgrade kafka client in spark_streaming_kafka 2.2 Fri, 12 Mar, 14:18
Sachit Murarka Issue while consuming message in kafka using structured streaming Fri, 12 Mar, 11:28
Gabor Somogyi   Re: Issue while consuming message in kafka using structured streaming Fri, 12 Mar, 12:00
Sachit Murarka     Re: Issue while consuming message in kafka using structured streaming Fri, 12 Mar, 12:02
Gabor Somogyi       Re: Issue while consuming message in kafka using structured streaming Fri, 12 Mar, 12:13
Sachit Murarka         Re: Issue while consuming message in kafka using structured streaming Tue, 23 Mar, 16:02
Message list1 · 2 · 3 · Next »Thread · Author · Date
Box list
Oct 202177
Sep 2021126
Aug 2021171
Jul 2021158
Jun 2021179
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137