spark-user mailing list archives: April 2021

Site index · List index
Message list1 · 2 · Next »Thread · Author · Date
Re: Ubuntu 18.04: Docker: start-master.sh: command not found
JB Data31   Re: Ubuntu 18.04: Docker: start-master.sh: command not found Thu, 01 Apr, 05:33
GUINKO Ferdinand Re: Ubuntu 18.04: Docker: start-master.sh: command not found Thu, 01 Apr, 06:31
RE: FW: Email to Spark Org please
Williams, David (Risk Value Stream)   RE: FW: Email to Spark Org please Thu, 01 Apr, 15:35
Sean Owen     Re: FW: Email to Spark Org please Thu, 01 Apr, 21:18
Mich Talebzadeh PySpark functions for various sources and sinks Fri, 02 Apr, 11:29
Alexei [SPARK SQL] Sometimes spark does not scale down on k8s Fri, 02 Apr, 14:47
Alexei   Re: [SPARK SQL] Sometimes spark does not scale down on k8s Mon, 05 Apr, 16:52
Jacek Laskowski Writing to Google Cloud Storage with v2 algorithm safe? Sat, 03 Apr, 16:12
Mich Talebzadeh   Re: Writing to Google Cloud Storage with v2 algorithm safe? Sat, 03 Apr, 17:49
Jacek Laskowski     Re: Writing to Google Cloud Storage with v2 algorithm safe? Sat, 03 Apr, 21:43
vaquar khan   Re: Writing to Google Cloud Storage with v2 algorithm safe? Sun, 04 Apr, 05:27
Jacek Laskowski     Re: Writing to Google Cloud Storage with v2 algorithm safe? Sun, 04 Apr, 10:42
Ali Gouta Spark structured streaming + offset management in kafka + kafka headers Sat, 03 Apr, 19:02
Gabor Somogyi   Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 10:28
Jacek Laskowski     Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 10:52
Ali Gouta       Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 15:26
Mich Talebzadeh         Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 16:11
Ali Gouta           Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 16:27
Mich Talebzadeh             Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 16:44
Ali Gouta               Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 17:16
Gabor Somogyi                 Re: Spark structured streaming + offset management in kafka + kafka headers Sun, 04 Apr, 20:23
Re: Source.getBatch and schema vs qe.analyzed.schema?
Jacek Laskowski   Re: Source.getBatch and schema vs qe.analyzed.schema? Sat, 03 Apr, 21:45
Kohki Nishio Ordering pushdown for Spark Datasources Sun, 04 Apr, 22:53
Mich Talebzadeh   Re: Ordering pushdown for Spark Datasources Mon, 05 Apr, 08:01
Kohki Nishio     Re: Ordering pushdown for Spark Datasources Tue, 06 Apr, 06:17
Mich Talebzadeh       Re: Ordering pushdown for Spark Datasources Tue, 06 Apr, 07:54
Eric Beabes Spark doesn't add _SUCCESS file when 'partitionBy' is used Mon, 05 Apr, 16:48
Krishna Chakka Tuning spark job to make count faster. Tue, 06 Apr, 06:59
Sean Owen   Re: Tuning spark job to make count faster. Tue, 06 Apr, 12:23
Mich Talebzadeh jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 08:14
Gabor Somogyi   Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 10:54
Gabor Somogyi     Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 11:26
Mich Talebzadeh     Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 11:41
Sean Owen       Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 12:20
Mich Talebzadeh         Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 12:39
Gabor Somogyi           Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 14:44
Mich Talebzadeh             Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 16:35
Sean Owen               Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 16:39
Mich Talebzadeh                 Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 16:52
Mich Talebzadeh                   Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Tue, 06 Apr, 19:56
Gabor Somogyi                     Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 06:50
Mich Talebzadeh                       Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 09:11
Gabor Somogyi                         Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 12:15
Sean Owen                           Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 12:17
Gabor Somogyi                             Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 12:19
Mich Talebzadeh                               Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 14:43
Amit Joshi                                 Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 16:02
Mich Talebzadeh                                   Re: jar incompatibility with Spark 3.1.1 for structured streaming with kafka Wed, 07 Apr, 16:13
Re: Spark Structured Streaming with PySpark throwing error in execution
Mich Talebzadeh   Re: Spark Structured Streaming with PySpark throwing error in execution Tue, 06 Apr, 10:07
Latha Appanna unsubscribe Tue, 06 Apr, 10:09
Ranju Jain Dynamic Allocation Backlog Property in Spark on Kubernetes Tue, 06 Apr, 11:58
Attila Zsolt Piros   Re: Dynamic Allocation Backlog Property in Spark on Kubernetes Thu, 08 Apr, 18:42
Ranju Jain     RE: Dynamic Allocation Backlog Property in Spark on Kubernetes Fri, 09 Apr, 04:03
Attila Zsolt Piros       Re: Dynamic Allocation Backlog Property in Spark on Kubernetes Fri, 09 Apr, 05:41
ranju goel   Re: Dynamic Allocation Backlog Property in Spark on Kubernetes Sat, 10 Apr, 17:01
Attila Zsolt Piros     Re: Dynamic Allocation Backlog Property in Spark on Kubernetes Sat, 10 Apr, 18:19
ranju goel       Re: Dynamic Allocation Backlog Property in Spark on Kubernetes Sun, 11 Apr, 00:36
Tzahi File Spark performance over S3 Tue, 06 Apr, 18:44
Gourav Sengupta   Re: Spark performance over S3 Tue, 06 Apr, 19:24
Boris Litvak     RE: Spark performance over S3 Wed, 07 Apr, 06:18
Hariharan   Re: Spark performance over S3 Wed, 07 Apr, 11:12
Vladimir Prus     Re: Spark performance over S3 Wed, 07 Apr, 11:55
Tzahi File     Re: Spark performance over S3 Wed, 07 Apr, 13:01
Boris Litvak       RE: Spark performance over S3 Wed, 07 Apr, 14:21
Boris Litvak Data Lakes using Spark Wed, 07 Apr, 06:32
Sean Owen Mesos + Spark users going forward? Wed, 07 Apr, 12:11
Mridul Muralidharan   Re: Mesos + Spark users going forward? Wed, 07 Apr, 12:31
dmcwhorter   Re: Mesos + Spark users going forward? Wed, 07 Apr, 14:09
Re: Invite Spark community as Pulsar Summit NA 2021 Community Partner
Dianjin Wang   Re: Invite Spark community as Pulsar Summit NA 2021 Community Partner Wed, 07 Apr, 14:17
SRITHALAM, ANUPAMA (Risk Value Stream) Apache ML Agorithm Solution Wed, 07 Apr, 16:12
Adi Polak   Re: Apache ML Agorithm Solution Wed, 07 Apr, 16:43
Sean Owen   Re: Apache ML Agorithm Solution Wed, 07 Apr, 16:53
Mich Talebzadeh     Re: Apache ML Agorithm Solution Wed, 07 Apr, 17:33
iashiq5 How to adapt PySpark to optimize handling of large no. of partitions? Wed, 07 Apr, 20:29
Tomas Bartalos Big Broadcast Hash Join with Dynamic Partition Pruning gives wrong results Wed, 07 Apr, 23:51
Wenchen Fan   Re: Big Broadcast Hash Join with Dynamic Partition Pruning gives wrong results Thu, 08 Apr, 02:24
maziyar Why is Spark 3.0.x is faster than Spark 3.1.x Thu, 08 Apr, 10:38
maziyar Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 10:47
Mich Talebzadeh   Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 11:58
maziyar     Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 12:12
Mich Talebzadeh       Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 12:57
Russell Spitzer         Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 13:28
Russell Spitzer           Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 13:33
maziyar         Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 13:52
Sean Owen           Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 13:54
Maziyar Panahi             Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 14:04
Mich Talebzadeh               Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 14:47
Maziyar Panahi                 Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 15:19
Mich Talebzadeh                   Re: Why is Spark 3.0.x faster than Spark 3.1.x Thu, 08 Apr, 15:27
Maziyar Panahi                     Re: Why is Spark 3.0.x faster than Spark 3.1.x Fri, 09 Apr, 07:58
Mich Talebzadeh                       Re: Why is Spark 3.0.x faster than Spark 3.1.x Fri, 09 Apr, 08:41
Weiand, Markus, NMA-CFD possible bug Thu, 08 Apr, 12:52
Sean Owen   Re: possible bug Thu, 08 Apr, 13:00
Weiand, Markus, NMA-CFD     AW: possible bug Thu, 08 Apr, 13:06
Russell Spitzer       Re: possible bug Thu, 08 Apr, 13:24
Attila Zsolt Piros         Re: possible bug Thu, 08 Apr, 14:12
Weiand, Markus, NMA-CFD         AW: possible bug Fri, 09 Apr, 05:03
Attila Zsolt Piros           Re: possible bug Fri, 09 Apr, 15:42
Sean Owen             Re: possible bug Fri, 09 Apr, 15:54
Attila Zsolt Piros               Re: possible bug Fri, 09 Apr, 16:19
Sean Owen                 Re: possible bug Fri, 09 Apr, 16:31
Mich Talebzadeh                   Re: possible bug Fri, 09 Apr, 16:36
Mich Talebzadeh                     Re: possible bug Fri, 09 Apr, 16:36
Mich Talebzadeh             Re: possible bug Fri, 09 Apr, 15:58
Bin Fan Evaluating Apache Spark with Data Orchestration using TPC-DS Thu, 08 Apr, 20:25
Rao Bandaru [Spark SQL]:to calculate distance between four coordinates(Latitude1, Longtitude1, Latitude2, Longtitude2) in the pysaprk dataframe Fri, 09 Apr, 11:24
ayan guha   Re: [Spark SQL]:to calculate distance between four coordinates(Latitude1, Longtitude1, Latitude2, Longtitude2) in the pysaprk dataframe Fri, 09 Apr, 12:31
Sean Owen     Re: [Spark SQL]:to calculate distance between four coordinates(Latitude1, Longtitude1, Latitude2, Longtitude2) in the pysaprk dataframe Fri, 09 Apr, 12:41
Rao Bandaru     Re: [Spark SQL]:to calculate distance between four coordinates(Latitude1, Longtitude1, Latitude2, Longtitude2) in the pysaprk dataframe Fri, 09 Apr, 12:55
ayan guha       Re: [Spark SQL]:to calculate distance between four coordinates(Latitude1, Longtitude1, Latitude2, Longtitude2) in the pysaprk dataframe Fri, 09 Apr, 14:14
Sean Owen         Re: [Spark SQL]:to calculate distance between four coordinates(Latitude1, Longtitude1, Latitude2, Longtitude2) in the pysaprk dataframe Fri, 09 Apr, 16:36
ayan guha           Re: [Spark SQL]:to calculate distance between four coordinates(Latitude1, Longtitude1, Latitude2, Longtitude2) in the pysaprk dataframe Sat, 10 Apr, 01:51
Muhammed Favas How to use spark steaming data to plot live line chart Fri, 09 Apr, 12:33
Mich Talebzadeh   Re: How to use spark steaming data to plot live line chart Fri, 09 Apr, 12:47
Muhammed Favas     RE: How to use spark steaming data to plot live line chart Fri, 09 Apr, 13:30
Mich Talebzadeh       Re: How to use spark steaming data to plot live line chart Fri, 09 Apr, 14:11
Martin Somers GPU job in Spark 3 Fri, 09 Apr, 16:18
Sean Owen   Re: GPU job in Spark 3 Fri, 09 Apr, 16:32
Tom Graves   Re: GPU job in Spark 3 Fri, 09 Apr, 16:52
HaoZ   Re: GPU job in Spark 3 Fri, 09 Apr, 17:09
Sean Owen     Re: GPU job in Spark 3 Fri, 09 Apr, 17:27
Gourav Sengupta     Re: GPU job in Spark 3 Thu, 15 Apr, 17:06
KhajaAsmath Mohammed Spark Hbase Hive error in EMR Fri, 09 Apr, 18:15
Dhruv Kumar Automated setup of a multi-node cluster for Apache Spark Sat, 10 Apr, 08:51
Hariharan   Re: Automated setup of a multi-node cluster for Apache Spark Sun, 11 Apr, 02:57
András Kolbert Tasks are skewed to one executor Sat, 10 Apr, 13:26
Mich Talebzadeh   Re: Tasks are skewed to one executor Sat, 10 Apr, 14:42
András Kolbert     Re: Tasks are skewed to one executor Sun, 11 Apr, 07:38
Mich Talebzadeh       Re: Tasks are skewed to one executor Sun, 11 Apr, 08:39
András Kolbert         Re: Tasks are skewed to one executor Sun, 11 Apr, 09:16
Mich Talebzadeh           Re: Tasks are skewed to one executor Sun, 11 Apr, 09:56
András Kolbert             Re: Tasks are skewed to one executor Sun, 11 Apr, 10:09
Mich Talebzadeh               Re: Tasks are skewed to one executor Sun, 11 Apr, 10:30
Gourav Sengupta       Re: Tasks are skewed to one executor Mon, 12 Apr, 08:01
András Kolbert         Re: Tasks are skewed to one executor Mon, 12 Apr, 08:43
Mich Talebzadeh           Re: Tasks are skewed to one executor Mon, 12 Apr, 13:57
Mich Talebzadeh             Re: Tasks are skewed to one executor Mon, 12 Apr, 14:00
András Kolbert               Re: Tasks are skewed to one executor Mon, 12 Apr, 14:35
Mich Talebzadeh                 Re: Tasks are skewed to one executor Mon, 12 Apr, 14:54
Gourav Sengupta                 Re: Tasks are skewed to one executor Mon, 12 Apr, 16:19
András Kolbert                   Re: Tasks are skewed to one executor Sat, 17 Apr, 06:25
Mich Talebzadeh                     Re: Tasks are skewed to one executor Sun, 18 Apr, 09:30
András Kolbert                       Re: Tasks are skewed to one executor Wed, 21 Apr, 21:56
Mich Talebzadeh                         Re: Tasks are skewed to one executor Thu, 22 Apr, 13:55
András Kolbert                           Re: Tasks are skewed to one executor Mon, 26 Apr, 20:25
Mich Talebzadeh                             Re: Tasks are skewed to one executor Mon, 26 Apr, 20:44
András Kolbert                               Re: Tasks are skewed to one executor Mon, 26 Apr, 21:06
Mich Talebzadeh                                 Re: Tasks are skewed to one executor Mon, 26 Apr, 21:26
András Kolbert                                   Re: Tasks are skewed to one executor Tue, 27 Apr, 06:56
Mich Talebzadeh                                     Re: Tasks are skewed to one executor Tue, 27 Apr, 14:47
Florian CASTELAIN Spark and Bintray's shutdown Mon, 12 Apr, 16:05
Sean Owen   Re: Spark and Bintray's shutdown Mon, 12 Apr, 16:28
Xiao Li     Re: Spark and Bintray's shutdown Mon, 12 Apr, 16:37
Grigory Pomadchin       Re: Spark and Bintray's shutdown Mon, 12 Apr, 19:20
KhajaAsmath Mohammed Spark Session error with 30s Mon, 12 Apr, 18:55
Sean Owen   Re: Spark Session error with 30s Mon, 12 Apr, 19:15
KhajaAsmath Mohammed     Re: Spark Session error with 30s Mon, 12 Apr, 19:20
KhajaAsmath Mohammed       Re: Spark Session error with 30s Mon, 12 Apr, 19:25
Sean Owen         Re: Spark Session error with 30s Mon, 12 Apr, 19:29
Peng Lei           Re: Spark Session error with 30s Tue, 13 Apr, 01:35
KhajaAsmath Mohammed             Re: Spark Session error with 30s Tue, 13 Apr, 23:44
Mohamadreza Rostami [Spark Core][Advanced]: Problem with data locality when running Spark query with local nature on apache Hadoop Tue, 13 Apr, 08:47
Russell Spitzer   Re: [Spark Core][Advanced]: Problem with data locality when running Spark query with local nature on apache Hadoop Tue, 13 Apr, 13:15
Russell Spitzer     Re: [Spark Core][Advanced]: Problem with data locality when running Spark query with local nature on apache Hadoop Tue, 13 Apr, 13:50
KhajaAsmath Mohammed Spark2.4 json Jackson errors Tue, 13 Apr, 23:46
KhajaAsmath Mohammed Overirde Jackson jar - Spark Submit Wed, 14 Apr, 15:57
Python level of knowledge for Spark and PySpark
ashok34...@yahoo.com.INVALID   Python level of knowledge for Spark and PySpark Wed, 14 Apr, 16:18
ayan guha     Re: Python level of knowledge for Spark and PySpark Wed, 14 Apr, 19:54
Lalwani, Jayesh       Re: Python level of knowledge for Spark and PySpark Wed, 14 Apr, 20:04
Mich Talebzadeh         Re: Python level of knowledge for Spark and PySpark Wed, 14 Apr, 20:39
KhajaAsmath Mohammed Spark submit hbase issue Wed, 14 Apr, 21:38
Mich Talebzadeh   Re: Spark submit hbase issue Wed, 14 Apr, 21:54
Kalin Stoyanov full SQL query graph not shown in monitoring when using cache Thu, 15 Apr, 12:54
Kalin Stoyanov   Re: full SQL query graph not shown in monitoring when using cache Thu, 15 Apr, 13:37
Mich Talebzadeh   Re: full SQL query graph not shown in monitoring when using cache Thu, 15 Apr, 13:44
Kalin Stoyanov     Re: full SQL query graph not shown in monitoring when using cache Thu, 15 Apr, 13:53
Mich Talebzadeh       Re: full SQL query graph not shown in monitoring when using cache Thu, 15 Apr, 14:10
Message list1 · 2 · Next »Thread · Author · Date
Box list
Jul 2021124
Jun 2021179
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137