spark-user mailing list archives: April 2021

Site index · List index
Message list1 · 2 · 3 · Next »Thread · Author · Date
JB Data31 Re: Ubuntu 18.04: Docker: start-master.sh: command not found Thu, 01 Apr, 05:33
GUINKO Ferdinand Re: Ubuntu 18.04: Docker: start-master.sh: command not found Thu, 01 Apr, 06:31
Williams, David (Risk Value Stream) RE: FW: Email to Spark Org please Thu, 01 Apr, 15:35
Sean Owen Re: FW: Email to Spark Org please Thu, 01 Apr, 21:18
Mich Talebzadeh PySpark functions for various sources and sinks Fri, 02 Apr, 11:29
Alexei [SPARK SQL] Sometimes spark does not scale down on k8s Fri, 02 Apr, 14:47
Jacek Laskowski Writing to Google Cloud Storage with v2 algorithm safe? Sat, 03 Apr, 16:12
Mich Talebzadeh Re: Writing to Google Cloud Storage with v2 algorithm safe? Sat, 03 Apr, 17:49
Ali Gouta Spark structured streaming + offset management in kafka + kafka headers Sat, 03 Apr, 19:02
Jacek Laskowski Re: Writing to Google Cloud Storage with v2 algorithm safe? Sat, 03 Apr, 21:43
Jacek Laskowski Re: Source.getBatch and schema vs qe.analyzed.schema? Sat, 03 Apr, 21:45
vaquar khan Re: Writing to Google Cloud Storage with v2 algorithm safe? Sun, 04 Apr, 05:27
Gabor Somogyi Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 10:28
Jacek Laskowski Re: Writing to Google Cloud Storage with v2 algorithm safe? Sun, 04 Apr, 10:42
Jacek Laskowski Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 10:52
Ali Gouta Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 15:26
Mich Talebzadeh Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 16:11
Ali Gouta Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 16:27
Mich Talebzadeh Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 16:44
Ali Gouta Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 17:16
Gabor Somogyi Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 20:23
Kohki Nishio Ordering pushdown for Spark Datasources Sun, 04 Apr, 22:53
Mich Talebzadeh Re: Ordering pushdown for Spark Datasources Mon, 05 Apr, 08:01
Eric Beabes Spark doesn't add _SUCCESS file when 'partitionBy' is used Mon, 05 Apr, 16:48
Alexei Re: [SPARK SQL] Sometimes spark does not scale down on k8s Mon, 05 Apr, 16:52
Kohki Nishio Re: Ordering pushdown for Spark Datasources Tue, 06 Apr, 06:17
Krishna Chakka Tuning spark job to make count faster. Tue, 06 Apr, 06:59
Mich Talebzadeh Re: Ordering pushdown for Spark Datasources Tue, 06 Apr, 07:54
Mich Talebzadeh jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 08:14
Mich Talebzadeh Re: Spark Structured Streaming with PySpark throwing error in execution Tue, 06 Apr, 10:07
Latha Appanna unsubscribe Tue, 06 Apr, 10:09
Gabor Somogyi Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 10:54
Gabor Somogyi Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 11:26
Mich Talebzadeh Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 11:41
Ranju Jain Dynamic Allocation Backlog Property in Spark on Kubernetes Tue, 06 Apr, 11:58
Sean Owen Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 12:20
Sean Owen Re: Tuning spark job to make count faster. Tue, 06 Apr, 12:23
Mich Talebzadeh Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 12:39
Gabor Somogyi Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 14:44
Mich Talebzadeh Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 16:35
Sean Owen Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 16:39
Mich Talebzadeh Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 16:52
Tzahi File Spark performance over S3 Tue, 06 Apr, 18:44
Gourav Sengupta Re: Spark performance over S3 Tue, 06 Apr, 19:24
Mich Talebzadeh Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 19:56
Boris Litvak RE: Spark performance over S3 Wed, 07 Apr, 06:18
Boris Litvak Data Lakes using Spark Wed, 07 Apr, 06:32
Gabor Somogyi Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 06:50
Mich Talebzadeh Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 09:11
Hariharan Re: Spark performance over S3 Wed, 07 Apr, 11:12
Vladimir Prus Re: Spark performance over S3 Wed, 07 Apr, 11:55
Sean Owen Mesos + Spark users going forward? Wed, 07 Apr, 12:11
Gabor Somogyi Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 12:15
Sean Owen Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 12:17
Gabor Somogyi Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 12:19
Mridul Muralidharan Re: Mesos + Spark users going forward? Wed, 07 Apr, 12:31
Tzahi File Re: Spark performance over S3 Wed, 07 Apr, 13:01
dmcwhorter Re: Mesos + Spark users going forward? Wed, 07 Apr, 14:09
Dianjin Wang Re: Invite Spark community as Pulsar Summit NA 2021 Community Partner Wed, 07 Apr, 14:17
Boris Litvak RE: Spark performance over S3 Wed, 07 Apr, 14:21
Mich Talebzadeh Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 14:43
Amit Joshi Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 16:02
SRITHALAM, ANUPAMA (Risk Value Stream) Apache ML Agorithm Solution Wed, 07 Apr, 16:12
Mich Talebzadeh Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 16:13
Adi Polak Re: Apache ML Agorithm Solution Wed, 07 Apr, 16:43
Sean Owen Re: Apache ML Agorithm Solution Wed, 07 Apr, 16:53
Mich Talebzadeh Re: Apache ML Agorithm Solution Wed, 07 Apr, 17:33
iashiq5 How to adapt PySpark to optimize handling of large no. of partitions? Wed, 07 Apr, 20:29
Tomas Bartalos Big Broadcast Hash Join with Dynamic Partition Pruning gives wrong results Wed, 07 Apr, 23:51
Wenchen Fan Re: Big Broadcast Hash Join with Dynamic Partition Pruning gives wrong results Thu, 08 Apr, 02:24
maziyar Why is Spark 3.0.x is faster than Spark 3.1.x Thu, 08 Apr, 10:38
maziyar Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 10:47
Mich Talebzadeh Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 11:58
maziyar Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 12:12
Weiand, Markus, NMA-CFD possible bug Thu, 08 Apr, 12:52
Mich Talebzadeh Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 12:57
Sean Owen Re: possible bug Thu, 08 Apr, 13:00
Weiand, Markus, NMA-CFD AW: possible bug Thu, 08 Apr, 13:06
Russell Spitzer Re: possible bug Thu, 08 Apr, 13:24
Russell Spitzer Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 13:28
Russell Spitzer Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 13:33
maziyar Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 13:52
Sean Owen Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 13:54
Maziyar Panahi Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 14:04
Attila Zsolt Piros Re: possible bug Thu, 08 Apr, 14:12
Mich Talebzadeh Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 14:47
Maziyar Panahi Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 15:19
Mich Talebzadeh Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 15:27
Attila Zsolt Piros Re: Dynamic Allocation Backlog Property in Spark on Kubernetes Thu, 08 Apr, 18:42
Bin Fan Evaluating Apache Spark with Data Orchestration using TPC-DS Thu, 08 Apr, 20:25
Ranju Jain RE: Dynamic Allocation Backlog Property in Spark on Kubernetes Fri, 09 Apr, 04:03
Weiand, Markus, NMA-CFD AW: possible bug Fri, 09 Apr, 05:03
Attila Zsolt Piros Re: Dynamic Allocation Backlog Property in Spark on Kubernetes Fri, 09 Apr, 05:41
Maziyar Panahi Re: Why is Spark 3.0.x faster than Spark 3.1.x Fri, 09 Apr, 07:58
Mich Talebzadeh Re: Why is Spark 3.0.x faster than Spark 3.1.x Fri, 09 Apr, 08:41
Rao Bandaru [Spark SQL]:to calculate distance between four coordinates(Latitude1, Longtitude1, Latitude2, Longtitude2) in the pysaprk dataframe Fri, 09 Apr, 11:24
ayan guha Re: [Spark SQL]:to calculate distance between four coordinates(Latitude1, Longtitude1, Latitude2, Longtitude2) in the pysaprk dataframe Fri, 09 Apr, 12:31
Muhammed Favas How to use spark steaming data to plot live line chart Fri, 09 Apr, 12:33
Sean Owen Re: [Spark SQL]:to calculate distance between four coordinates(Latitude1, Longtitude1, Latitude2, Longtitude2) in the pysaprk dataframe Fri, 09 Apr, 12:41
Mich Talebzadeh Re: How to use spark steaming data to plot live line chart Fri, 09 Apr, 12:47
Message list1 · 2 · 3 · Next »Thread · Author · Date
Box list
May 202138
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137