spark-user mailing list archives: May 2021

Site index · List index
Message list1 · 2 · Next »Thread · Author · Date
RE: Recursive Queries or Recursive UDF?
Bode, Meikel, NMA-CFD   RE: Recursive Queries or Recursive UDF? Sat, 01 May, 13:17
Re: Delivery Status Notification (Failure)
Mich Talebzadeh   Re: Delivery Status Notification (Failure) Sun, 02 May, 09:45
Mich Talebzadeh     Re: Delivery Status Notification (Failure) Sun, 02 May, 09:49
Re: Spark JDBC errors out
Farhan Misarwala   Re: Spark JDBC errors out Sun, 02 May, 11:46
Sachit Murarka How to handle auto-restart in Kubernetes Spark application Sun, 02 May, 16:08
Ali Gouta   Re: How to handle auto-restart in Kubernetes Spark application Sun, 02 May, 17:05
Bode, Meikel, NMA-CFD Broadcast Variable Mon, 03 May, 12:54
Sean Owen   Re: Broadcast Variable Mon, 03 May, 13:40
Eric Beabes Stream which needs to be “joined” with another Stream of “Reference” data. Mon, 03 May, 16:36
Mich Talebzadeh   Re: Stream which needs to be “joined” with another Stream of “Reference” data. Mon, 03 May, 16:51
Eric Beabes     Re: Stream which needs to be “joined” with another Stream of “Reference” data. Mon, 03 May, 17:01
Mich Talebzadeh       Re: Stream which needs to be “joined” with another Stream of “Reference” data. Mon, 03 May, 17:19
Yuri_Oleynikov_=28=E2=80=AB=D7=99=D7=95=D7=A8=D7=99_?= אולייניקו=?utf-8?Q?=D7=91=E2=80=AC=E2=80=8E=29       Re: Stream which needs to be “joined” with another Stream of “Reference” data. Mon, 03 May, 17:27
Mich Talebzadeh         Re: Stream which needs to be “joined” with another Stream of “Reference” data. Mon, 03 May, 17:53
Yuri_Oleynikov_=28=E2=80=AB=D7=99=D7=95=D7=A8=D7=99_?= אולייניקו=?utf-8?Q?=D7=91=E2=80=AC=E2=80=8E=29           Re: Stream which needs to be “joined” with another Stream of “Reference” data. Mon, 03 May, 18:02
Mich Talebzadeh             Re: Stream which needs to be “joined” with another Stream of “Reference” data. Mon, 03 May, 18:07
Eric Beabes               Re: Stream which needs to be “joined” with another Stream of “Reference” data. Mon, 03 May, 18:48
Mich Talebzadeh                 Re: Stream which needs to be “joined” with another Stream of “Reference” data. Mon, 03 May, 19:06
Tianchen Zhang [No Subject] Mon, 03 May, 18:37
Tianchen Zhang [Spark Catalog API] Support for metadata Backup/Restore Mon, 03 May, 18:38
Wenchen Fan   Re: [Spark Catalog API] Support for metadata Backup/Restore Fri, 07 May, 19:27
Tianchen Zhang     Re: [Spark Catalog API] Support for metadata Backup/Restore Fri, 07 May, 21:02
ayan guha       Re: [Spark Catalog API] Support for metadata Backup/Restore Fri, 07 May, 22:54
Wenchen Fan         Re: [Spark Catalog API] Support for metadata Backup/Restore Tue, 11 May, 07:57
Mich Talebzadeh           Re: [Spark Catalog API] Support for metadata Backup/Restore Tue, 11 May, 08:13
Tianchen Zhang             Re: [Spark Catalog API] Support for metadata Backup/Restore Tue, 11 May, 20:27
Scott Gerard [PySpark][apache-spark]: Pass Pyspark SparkSession instance to Scala Tue, 04 May, 17:02
Taylor Schneider Windows Jupyter Notebook Cannot Connect to Kubernetes Master Tue, 04 May, 19:23
Priyanka Kakkar Does Pyspark script support Sonarqube Wed, 05 May, 08:08
Kapil Garg How to read multiple HDFS directories Wed, 05 May, 14:45
Mich Talebzadeh   Re: How to read multiple HDFS directories Wed, 05 May, 15:04
Kapil Garg     Re: How to read multiple HDFS directories Wed, 05 May, 15:22
Mich Talebzadeh       Re: How to read multiple HDFS directories Wed, 05 May, 15:50
Kapil Garg         Re: How to read multiple HDFS directories Wed, 05 May, 16:03
Mich Talebzadeh           Re: How to read multiple HDFS directories Wed, 05 May, 16:52
Kapil Garg             Re: How to read multiple HDFS directories Wed, 05 May, 17:04
Lalwani, Jayesh   Re: How to read multiple HDFS directories Wed, 05 May, 17:11
Kapil Garg     Re: How to read multiple HDFS directories Wed, 05 May, 17:18
Kapil Garg       Re: How to read multiple HDFS directories Wed, 05 May, 17:29
joerg.strebel         Re: How to read multiple HDFS directories Wed, 05 May, 18:06
Kapil Garg           Re: How to read multiple HDFS directories Wed, 05 May, 18:30
Kapil Garg             Re: How to read multiple HDFS directories Wed, 05 May, 18:37
Fwd: Graceful shutdown SPARK Structured Streaming
Gourav Sengupta   Fwd: Graceful shutdown SPARK Structured Streaming Wed, 05 May, 16:29
Mich Talebzadeh     Re: Graceful shutdown SPARK Structured Streaming Wed, 05 May, 17:04
Gourav Sengupta       Re: Graceful shutdown SPARK Structured Streaming Thu, 06 May, 07:12
ayan guha         Re: Graceful shutdown SPARK Structured Streaming Thu, 06 May, 12:28
Mich Talebzadeh           Re: Graceful shutdown SPARK Structured Streaming Thu, 06 May, 19:07
Chetan Khatri Performance Improvement: Collect in spark taking huge time Thu, 06 May, 02:15
Chetan Khatri   Re: Performance Improvement: Collect in spark taking huge time Thu, 06 May, 02:52
Tang Jinxin (无主题) Thu, 06 May, 09:32
Re: Updating spark-env.sh per application
Renu Yadav   Re: Updating spark-env.sh per application Fri, 07 May, 10:33
Mich Talebzadeh     Re: Updating spark-env.sh per application Fri, 07 May, 13:21
Renu Yadav       Re: Updating spark-env.sh per application Sun, 09 May, 03:31
Jacek Laskowski         Re: Updating spark-env.sh per application Sun, 09 May, 17:10
Mich Talebzadeh         Re: Updating spark-env.sh per application Mon, 10 May, 16:57
rajat kumar Issue while calling foreach in Pyspark Fri, 07 May, 15:06
Mich Talebzadeh   Re: Issue while calling foreach in Pyspark Fri, 07 May, 16:03
Sean Owen     Re: Issue while calling foreach in Pyspark Fri, 07 May, 16:05
rajat kumar       Re: Issue while calling foreach in Pyspark Fri, 07 May, 18:16
Mich Talebzadeh         Re: Issue while calling foreach in Pyspark Fri, 07 May, 20:08
Sean Owen           Re: Issue while calling foreach in Pyspark Fri, 07 May, 21:10
Mich Talebzadeh             Re: Issue while calling foreach in Pyspark Fri, 07 May, 21:32
rajat kumar               Re: Issue while calling foreach in Pyspark Sat, 08 May, 18:35
Sean Owen                 Re: Issue while calling foreach in Pyspark Sat, 08 May, 19:08
Sebastian Schere How to have map_from_arrays() in Spark 2.3 Sat, 08 May, 22:01
Giuseppe Ricci Calculate average from Spark stream Mon, 10 May, 14:47
Mich Talebzadeh   Re: Calculate average from Spark stream Mon, 10 May, 15:43
Lalwani, Jayesh   Re: Calculate average from Spark stream Mon, 10 May, 16:14
Giuseppe Ricci     Re: Calculate average from Spark stream Tue, 11 May, 13:39
Mich Talebzadeh       Re: Calculate average from Spark stream Wed, 12 May, 16:11
Mich Talebzadeh       Re: Calculate average from Spark stream Sat, 15 May, 21:47
Mich Talebzadeh         Re: Calculate average from Spark stream Mon, 17 May, 14:32
Mich Talebzadeh           Re: Calculate average from Spark stream Mon, 17 May, 17:01
Mich Talebzadeh             Re: Calculate average from Spark stream Tue, 18 May, 13:25
Mich Talebzadeh               Re: Calculate average from Spark stream Tue, 18 May, 14:58
Mich Talebzadeh                 Re: Calculate average from Spark stream Fri, 21 May, 14:36
Mich Talebzadeh                   Re: Calculate average from Spark stream Mon, 24 May, 14:57
Mich Talebzadeh                     Re: Calculate average from Spark stream Wed, 26 May, 14:48
Mich Talebzadeh                       Re: Calculate average from Spark stream Thu, 27 May, 19:56
Re: compile spark 3.1.1 error
jason_xu   Re: compile spark 3.1.1 error Mon, 10 May, 23:46
Talha Javed Installation Error - Please Help! Tue, 11 May, 20:21
Sean Owen   Re: Installation Error - Please Help! Tue, 11 May, 21:34
kushagra deep Merge two dataframes Wed, 12 May, 12:50
Raghavendra Ganesh   Re: Merge two dataframes Wed, 12 May, 15:30
kushagra deep     Re: Merge two dataframes Wed, 12 May, 15:47
Sean Owen       Re: Merge two dataframes Wed, 12 May, 16:06
Andrew Melo         Re: Merge two dataframes Wed, 12 May, 16:32
Andrew Melo           Re: Merge two dataframes Mon, 17 May, 18:03
Sean Owen             Re: Merge two dataframes Mon, 17 May, 18:10
Andrew Melo               Re: Merge two dataframes Mon, 17 May, 18:55
Lalwani, Jayesh                 Re: Merge two dataframes Mon, 17 May, 19:31
Andrew Melo                   Re: Merge two dataframes Mon, 17 May, 19:55
Mich Talebzadeh           Re: Merge two dataframes Wed, 19 May, 09:20
ayan guha             Re: Merge two dataframes Wed, 19 May, 12:14
Mich Talebzadeh   Re: Merge two dataframes Tue, 18 May, 15:09
kushagra deep     Re: Merge two dataframes Tue, 18 May, 15:38
Mich Talebzadeh       Re: Merge two dataframes Tue, 18 May, 17:17
kushagra deep         Re: Merge two dataframes Tue, 18 May, 19:51
Mich Talebzadeh           Re: Merge two dataframes Wed, 19 May, 09:06
Chris Thomas Spark with External Shuffle Service - using saved shuffle files in the event of executor failure Wed, 12 May, 14:56
Attila Zsolt Piros   Re: Spark with External Shuffle Service - using saved shuffle files in the event of executor failure Wed, 12 May, 16:38
abhilash.kr Understanding what happens when a job is submitted to a cluster Thu, 13 May, 14:13
Mich Talebzadeh   Re: Understanding what happens when a job is submitted to a cluster Thu, 13 May, 14:54
Lalwani, Jayesh   Re: Understanding what happens when a job is submitted to a cluster Thu, 13 May, 15:56
abhilash.kr     Re: Understanding what happens when a job is submitted to a cluster Thu, 13 May, 16:38
Lalwani, Jayesh       Re: Understanding what happens when a job is submitted to a cluster Thu, 13 May, 18:07
Suryansh Agnihotri beeline spark thrift server issue Thu, 13 May, 15:22
Yuming Wang   Re: beeline spark thrift server issue Fri, 14 May, 02:17
Bode, Meikel, NMA-CFD Thrift2 Server on Kubernetes? Fri, 14 May, 08:43
mykidong   Re: Thrift2 Server on Kubernetes? Fri, 14 May, 12:12
Bode, Meikel, NMA-CFD     RE: Thrift2 Server on Kubernetes? Sun, 16 May, 15:46
abhilash.kr Multiple destination single source Fri, 14 May, 17:15
KhajaAsmath Mohammed Urgent Help - Py Spark submit error Fri, 14 May, 21:49
KhajaAsmath Mohammed   Re: [EXTERNAL] Urgent Help - Py Spark submit error Fri, 14 May, 22:03
KhajaAsmath Mohammed     Re: [EXTERNAL] Urgent Help - Py Spark submit error Fri, 14 May, 22:05
KhajaAsmath Mohammed       Re: [EXTERNAL] Urgent Help - Py Spark submit error Fri, 14 May, 23:19
KhajaAsmath Mohammed         Re: [EXTERNAL] Urgent Help - Py Spark submit error Fri, 14 May, 23:43
Amit Joshi           Re: [EXTERNAL] Urgent Help - Py Spark submit error Sat, 15 May, 04:02
Gourav Sengupta             Re: [EXTERNAL] Urgent Help - Py Spark submit error Sat, 15 May, 06:41
Sean Owen               Re: [EXTERNAL] Urgent Help - Py Spark submit error Sat, 15 May, 13:12
KhajaAsmath Mohammed                 Re: [EXTERNAL] Urgent Help - Py Spark submit error Sat, 15 May, 17:31
Mich Talebzadeh                   Re: [EXTERNAL] Urgent Help - Py Spark submit error Sat, 15 May, 18:03
dwgw Unable to create direct stream with SSL enabled Kafka cluster Sat, 15 May, 14:49
Tianbin Jiang Spark History Server to S3 doesn't show up incomplete jobs Mon, 17 May, 17:48
RE: Why is Spark 3.0.x faster than Spark 3.1.x
Rao, Abhishek (Nokia - IN/Bangalore)   RE: Why is Spark 3.0.x faster than Spark 3.1.x Tue, 18 May, 05:42
Maziyar Panahi     Re: Why is Spark 3.0.x faster than Spark 3.1.x Tue, 18 May, 07:30
KhajaAsmath Mohammed S3 Access Issues - Spark Tue, 18 May, 23:11
石鹏磊 Does spark3.1.1 support parquet nested column predicate pushdown for array type and map type column Wed, 19 May, 06:43
Philipp Kraus Spark Executor dies in K8 cluster Wed, 19 May, 09:18
unresolved dependency: graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found
Wensheng Deng   unresolved dependency: graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found Wed, 19 May, 17:41
Sean Owen     Re: unresolved dependency: graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found Wed, 19 May, 18:03
Wensheng Deng       Re: unresolved dependency: graphframes#graphframes;0.8.1-spark2.4-s_2.11: not found Wed, 19 May, 18:26
Clay McDonald PySpark Write File Container exited with a non-zero exit code 143 Wed, 19 May, 19:09
Mich Talebzadeh   Re: PySpark Write File Container exited with a non-zero exit code 143 Wed, 19 May, 21:44
Clay McDonald     RE: PySpark Write File Container exited with a non-zero exit code 143 Thu, 20 May, 02:01
ayan guha       Re: PySpark Write File Container exited with a non-zero exit code 143 Thu, 20 May, 04:22
Clay McDonald         RE: PySpark Write File Container exited with a non-zero exit code 143 Thu, 20 May, 09:01
Bulldog20630405 spark 3.1.1 history server fails to boot with scala/MatchError Thu, 20 May, 17:34
Mithalee Mohapatra Question on spark on Kubernetes Thu, 20 May, 20:25
Gourav Sengupta   Re: Question on spark on Kubernetes Thu, 20 May, 21:27
Praneeth Shishtla Memory issues in 3.0.2 but works well on 2.4.4 Fri, 21 May, 10:57
Femi Anthony   Re: [External Sender] Memory issues in 3.0.2 but works well on 2.4.4 Fri, 21 May, 11:54
Bode, Meikel, NMA-CFD DF blank value fill Fri, 21 May, 11:27
ayan guha   Re: DF blank value fill Fri, 21 May, 13:35
Message list1 · 2 · Next »Thread · Author · Date
Box list
Jul 2021131
Jun 2021179
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137