spark-user mailing list archives: January 2021

Site index · List index
Message list1 · 2 · Next »Thread · Author · Date
somni fourfiveone hello Sun, 03 Jan, 04:40
Renganathan M How Spark Framework works a Compiler Sun, 03 Jan, 09:52
Sean Owen   Re: How Spark Framework works a Compiler Sun, 03 Jan, 16:23
Sachit Murarka Error while running Spark on K8s Mon, 04 Jan, 11:55
Prashant Sharma   Re: Error while running Spark on K8s Mon, 04 Jan, 12:15
Sachit Murarka     Re: Error while running Spark on K8s Mon, 04 Jan, 12:48
Prashant Sharma       Re: Error while running Spark on K8s Mon, 04 Jan, 13:14
Mich Talebzadeh Spark DF does not rename the column Mon, 04 Jan, 18:05
Lalwani, Jayesh   Re: Spark DF does not rename the column Mon, 04 Jan, 18:09
Mich Talebzadeh     Re: Spark DF does not rename the column Mon, 04 Jan, 18:16
German Schiavon     Re: Spark DF does not rename the column Mon, 04 Jan, 19:49
Mich Talebzadeh       Re: Spark DF does not rename the column Tue, 05 Jan, 11:55
Sachit Murarka Suggestion on Spark 2.4.7 vs Spark 3 for Kubernetes Mon, 04 Jan, 19:10
Prashant Sharma   Re: Suggestion on Spark 2.4.7 vs Spark 3 for Kubernetes Tue, 05 Jan, 09:38
Sachit Murarka     Re: Suggestion on Spark 2.4.7 vs Spark 3 for Kubernetes Tue, 05 Jan, 18:48
Mich Talebzadeh A question on extrapolation of a nonlinear curve fit beyond x value Tue, 05 Jan, 12:03
Sean Owen   Re: A question on extrapolation of a nonlinear curve fit beyond x value Tue, 05 Jan, 14:43
Mich Talebzadeh     Re: A question on extrapolation of a nonlinear curve fit beyond x value Tue, 05 Jan, 15:18
Sean Owen       Re: A question on extrapolation of a nonlinear curve fit beyond x value Tue, 05 Jan, 15:22
Mich Talebzadeh         Re: A question on extrapolation of a nonlinear curve fit beyond x value Tue, 05 Jan, 15:38
Sean Owen           Re: A question on extrapolation of a nonlinear curve fit beyond x value Tue, 05 Jan, 15:41
Mich Talebzadeh             Re: A question on extrapolation of a nonlinear curve fit beyond x value Tue, 05 Jan, 17:00
Michal Monselise Extending GraphFrames without running into serialization issues Tue, 05 Jan, 19:08
Sean Owen   Re: Extending GraphFrames without running into serialization issues Tue, 05 Jan, 21:25
Eric Beabes Understanding Executors UI Tue, 05 Jan, 23:19
Luca Canali   RE: Understanding Executors UI Wed, 06 Jan, 08:59
Eric Beabes     Re: Understanding Executors UI Fri, 08 Jan, 03:22
Luca Canali       RE: Understanding Executors UI Fri, 08 Jan, 20:59
Amit Sharma         Re: Understanding Executors UI Sat, 09 Jan, 19:02
Eric Beabes           Re: Understanding Executors UI Tue, 12 Jan, 16:13
brettplarson Impact of .localCheckpoint() and executor dying Wed, 06 Jan, 17:14
Jacek Laskowski   Re: Impact of .localCheckpoint() and executor dying Wed, 06 Jan, 19:09
Brett Larson     Re: Impact of .localCheckpoint() and executor dying Wed, 06 Jan, 19:30
Jacek Laskowski       Re: Impact of .localCheckpoint() and executor dying Wed, 06 Jan, 19:36
Ramki Ram Library to read health care EDI files in pyspark Wed, 06 Jan, 20:33
Dennis Suhari   Re: Library to read health care EDI files in pyspark Thu, 07 Jan, 06:51
Pradyumn Agrawal Spark 3.0.1 not connecting with Hive 2.1.1 Thu, 07 Jan, 07:40
michael.yang   Re: Spark 3.0.1 not connecting with Hive 2.1.1 Sat, 09 Jan, 13:21
Pradyumn Agrawal     Re: Spark 3.0.1 not connecting with Hive 2.1.1 Sat, 09 Jan, 17:53
DB Tsai       Re: Spark 3.0.1 not connecting with Hive 2.1.1 Sun, 10 Jan, 05:13
Pradyumn Agrawal         Re: Spark 3.0.1 not connecting with Hive 2.1.1 Sun, 10 Jan, 06:59
michael.yang       Re: Spark 3.0.1 not connecting with Hive 2.1.1 Sun, 10 Jan, 15:54
Pradyumn Agrawal         Re: Spark 3.0.1 not connecting with Hive 2.1.1 Tue, 12 Jan, 17:40
djdillon         Re: Spark 3.0.1 not connecting with Hive 2.1.1 Thu, 14 Jan, 14:36
Varun kumar Does Spark dynamic allocation work with more than one workers? Thu, 07 Jan, 10:46
Sean Owen   Re: Does Spark dynamic allocation work with more than one workers? Thu, 07 Jan, 14:43
mhd wrk Converting spark batch to spark streaming Thu, 07 Jan, 17:55
Jacek Laskowski   Re: Converting spark batch to spark streaming Fri, 08 Jan, 15:41
Mich Talebzadeh PyCharm, Running spark-submit calling jars and a package at run time Fri, 08 Jan, 14:11
Riccardo Ferrari   Re: PyCharm, Running spark-submit calling jars and a package at run time Fri, 08 Jan, 15:50
Mich Talebzadeh     Re: PyCharm, Running spark-submit calling jars and a package at run time Fri, 08 Jan, 16:13
Sean Owen       Re: PyCharm, Running spark-submit calling jars and a package at run time Fri, 08 Jan, 16:27
Riccardo Ferrari         Re: PyCharm, Running spark-submit calling jars and a package at run time Fri, 08 Jan, 16:38
Mich Talebzadeh           Re: PyCharm, Running spark-submit calling jars and a package at run time Fri, 08 Jan, 17:15
Sean Owen             Re: PyCharm, Running spark-submit calling jars and a package at run time Fri, 08 Jan, 17:18
Mich Talebzadeh               Re: PyCharm, Running spark-submit calling jars and a package at run time Fri, 08 Jan, 17:32
Mich Talebzadeh                 Re: PyCharm, Running spark-submit calling jars and a package at run time Sat, 09 Jan, 01:46
Mich Talebzadeh         Re: PyCharm, Running spark-submit calling jars and a package at run time Fri, 08 Jan, 16:41
Sudhir Babu Pothineni Spark standalone - reading kerberos hdfs Fri, 08 Jan, 17:48
Gabor Somogyi   Re: Spark standalone - reading kerberos hdfs Fri, 08 Jan, 18:12
Sudhir Babu Pothineni     Re: Spark standalone - reading kerberos hdfs Fri, 08 Jan, 19:53
Sudhir Babu Pothineni       Re: Spark standalone - reading kerberos hdfs Thu, 21 Jan, 15:24
Gábor Rőczei         Re: Spark standalone - reading kerberos hdfs Sun, 24 Jan, 00:12
jelmer   Re: Spark standalone - reading kerberos hdfs Sun, 24 Jan, 16:37
Re: how to integrate hbase and hive in spark3.0.1?
michael.yang   Re: how to integrate hbase and hive in spark3.0.1? Sat, 09 Jan, 13:11
András Kolbert Use case advice Sat, 09 Jan, 14:35
Artemis User   Re: Use case advice Sat, 09 Jan, 17:17
András Kolbert     Re: Use case advice Sat, 09 Jan, 20:30
muru       Re: Use case advice Sun, 10 Jan, 02:33
András Kolbert         Re: Use case advice Thu, 14 Jan, 12:54
András Kolbert           Re: Use case advice Thu, 14 Jan, 12:58
muru             Re: Use case advice Fri, 15 Jan, 01:45
Dilip Desavali   Re: Use case advice Fri, 15 Jan, 04:20
purav aggarwal     Re: Use case advice Thu, 21 Jan, 05:00
Sachit Murarka HPA - Kubernetes for Spark Sun, 10 Jan, 19:22
Sandish Kumar HN   Re: HPA - Kubernetes for Spark Sun, 10 Jan, 20:47
Sachit Murarka     Re: HPA - Kubernetes for Spark Mon, 11 Jan, 03:54
Ying Zhou [Spark SQL]HiveQL and Spark SQL producing different results Sun, 10 Jan, 21:48
Terry Kim   Re: [Spark SQL]HiveQL and Spark SQL producing different results Tue, 12 Jan, 18:40
Eric Beabes Data source v2 streaming sinks does not support Update mode Tue, 12 Jan, 05:35
Jacek Laskowski   Re: Data source v2 streaming sinks does not support Update mode Tue, 12 Jan, 12:30
Eric Beabes     Re: Data source v2 streaming sinks does not support Update mode Tue, 12 Jan, 14:04
Jungtaek Lim       Re: Data source v2 streaming sinks does not support Update mode Wed, 13 Jan, 06:04
Eric Beabes         Re: Data source v2 streaming sinks does not support Update mode Wed, 13 Jan, 06:46
Jungtaek Lim           Re: Data source v2 streaming sinks does not support Update mode Wed, 13 Jan, 07:53
Gabor Somogyi             Re: Data source v2 streaming sinks does not support Update mode Wed, 13 Jan, 10:26
Eric Beabes               Re: Data source v2 streaming sinks does not support Update mode Wed, 13 Jan, 17:56
Eric Beabes                 Re: Data source v2 streaming sinks does not support Update mode Mon, 18 Jan, 13:14
Gabor Somogyi                   Re: Data source v2 streaming sinks does not support Update mode Mon, 18 Jan, 13:37
Eric Beabes                     Re: Data source v2 streaming sinks does not support Update mode Mon, 18 Jan, 14:45
German Schiavon                       Re: Data source v2 streaming sinks does not support Update mode Mon, 18 Jan, 15:10
Jungtaek Lim                         Re: Data source v2 streaming sinks does not support Update mode Mon, 18 Jan, 23:50
Jungtaek Lim                           Re: Data source v2 streaming sinks does not support Update mode Mon, 18 Jan, 23:59
Eric Beabes                             Re: Data source v2 streaming sinks does not support Update mode Tue, 19 Jan, 03:15
Eric Beabes                               Re: Data source v2 streaming sinks does not support Update mode Tue, 19 Jan, 06:41
Gabor Somogyi                                 Re: Data source v2 streaming sinks does not support Update mode Tue, 19 Jan, 08:09
Eric Beabes                                   Re: Data source v2 streaming sinks does not support Update mode Tue, 19 Jan, 08:38
Artemis User Customizing K-Means for Anomaly Detection Tue, 12 Jan, 18:58
Sean Owen   Re: Customizing K-Means for Anomaly Detection Tue, 12 Jan, 19:11
Vineet Mishra Connection to Presto via Spark Wed, 13 Jan, 12:45
Gourav Sengupta   Re: Connection to Presto via Spark Thu, 21 Jan, 17:14
raymond.tan Spark Event Log Forwarding and Offset Tracking Wed, 13 Jan, 16:08
Jacek Laskowski   Re: Spark Event Log Forwarding and Offset Tracking Sun, 17 Jan, 14:08
Koert Kuipers understanding spark shuffle file re-use better Wed, 13 Jan, 16:38
Jacek Laskowski   Re: understanding spark shuffle file re-use better Sun, 17 Jan, 14:06
Rahul Kumar Insertable records in Datasource v2. Thu, 14 Jan, 02:06
Jacek Laskowski   Re: Insertable records in Datasource v2. Thu, 14 Jan, 11:48
Rahul Kumar     Re: Insertable records in Datasource v2. Fri, 15 Jan, 00:37
Mich Talebzadeh Adding third party specific jars to Spark Thu, 14 Jan, 16:51
Spark 3.0.1 giving warning while running with Java 11
Sachit Murarka   Spark 3.0.1 giving warning while running with Java 11 Fri, 15 Jan, 05:24
Sean Owen     Re: Spark 3.0.1 giving warning while running with Java 11 Fri, 15 Jan, 05:27
Sachit Murarka       Re: Spark 3.0.1 giving warning while running with Java 11 Fri, 15 Jan, 09:22
Mich Talebzadeh PySpark, setting spark conf values in a function and catching for errors Fri, 15 Jan, 23:52
Yuri Oleynikov (יורי אולייניקוב Dynamic Spark metrics creation Sat, 16 Jan, 13:20
Jacek Laskowski   Re: Dynamic Spark metrics creation Sun, 17 Jan, 13:56
Yuri_Oleynikov_=28=E2=80=AB=D7=99=D7=95=D7=A8=D7=99_?= אולייניקו=?utf-8?Q?=D7=91=E2=80=AC=E2=80=8E=29     Re: Dynamic Spark metrics creation Sun, 17 Jan, 14:15
Ivan Petrov       Re: Dynamic Spark metrics creation Sun, 17 Jan, 14:58
rajat kumar Running pyspark job from virtual environment Sat, 16 Jan, 18:16
rajat kumar   Re: Running pyspark job from virtual environment Sun, 17 Jan, 13:40
Mich Talebzadeh     Re: Running pyspark job from virtual environment Sun, 17 Jan, 17:02
rajat kumar       Re: Running pyspark job from virtual environment Sun, 17 Jan, 17:22
Mich Talebzadeh         Re: Running pyspark job from virtual environment Sun, 17 Jan, 18:18
Re: Correctness bug on Shuffle+Repartition scenario
Shiao-An Yuan   Re: Correctness bug on Shuffle+Repartition scenario Sun, 17 Jan, 12:21
Sean Owen     Re: Correctness bug on Shuffle+Repartition scenario Sun, 17 Jan, 20:26
Shiao-An Yuan       Re: Correctness bug on Shuffle+Repartition scenario Mon, 18 Jan, 02:35
王长春         Re: Correctness bug on Shuffle+Repartition scenario Mon, 18 Jan, 14:00
Gourav Sengupta     Re: Correctness bug on Shuffle+Repartition scenario Sun, 17 Jan, 20:49
Mich Talebzadeh   Re: Correctness bug on Shuffle+Repartition scenario Sun, 17 Jan, 23:17
刘 欢 [SparkSQL] Full Join Return Null Value For Funtion-Based Column Mon, 18 Jan, 05:39
刘 欢   Re: [SparkSQL] Full Join Return Null Value For Funtion-Based Column Mon, 18 Jan, 08:30
Amit Joshi [Spark Structured Streaming] Processing the data path coming from kafka. Mon, 18 Jan, 13:03
Boris Litvak   RE: [Spark Structured Streaming] Processing the data path coming from kafka. Mon, 18 Jan, 13:45
Amit Joshi     Re: [Spark Structured Streaming] Processing the data path coming from kafka. Mon, 18 Jan, 15:09
Boris Litvak       RE: [Spark Structured Streaming] Processing the data path coming from kafka. Mon, 18 Jan, 17:13
Amit Joshi         Re: [Spark Structured Streaming] Processing the data path coming from kafka. Mon, 18 Jan, 18:22
Brian Wylie           Re: [Spark Structured Streaming] Processing the data path coming from kafka. Mon, 18 Jan, 18:48
Boris Litvak           RE: [Spark Structured Streaming] Processing the data path coming from kafka. Mon, 18 Jan, 19:52
Alex Rehnby [Datasource API V2] Creating datasource - no step for final cleanup on read Mon, 18 Jan, 19:38
Peter Podlovics subscribe user@spark.apache.org Tue, 19 Jan, 17:29
Sean Owen   Re: subscribe user@spark.apache.org Tue, 19 Jan, 18:29
Artemis User Persisting Customized Transformer Tue, 19 Jan, 19:29
Brett Spark Application Timeout Tue, 19 Jan, 22:27
Jacek Laskowski   Re: Application Timeout Thu, 21 Jan, 11:39
Gabriel Magno Spark with Hadoop 3.3 distribution release in Download page Wed, 20 Jan, 03:34
Gabriel Magno   Re: Spark with Hadoop 3.3 distribution release in Download page Wed, 20 Jan, 03:59
Chao Sun     Re: Spark with Hadoop 3.3 distribution release in Download page Wed, 20 Jan, 04:12
rajat kumar Process each kafka record for structured streaming Wed, 20 Jan, 09:43
Jacek Laskowski   Re: Process each kafka record for structured streaming Thu, 21 Jan, 11:41
Vikas Garg Issue with executer Wed, 20 Jan, 12:04
Sachit Murarka   Re: Issue with executer Wed, 20 Jan, 12:07
Vikas Garg     Re: Issue with executer Wed, 20 Jan, 12:30
Mich Talebzadeh       Re: Issue with executer Wed, 20 Jan, 12:37
Vikas Garg         Re: Issue with executer Wed, 20 Jan, 13:49
Vikas Garg   Re: Issue with executer Wed, 20 Jan, 12:30
Message list1 · 2 · Next »Thread · Author · Date
Box list
Jul 2021125
Jun 2021179
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137