spark-user mailing list archives: August 2019

Site index · List index
Message list1 · 2 · Next »Thread · Author · Date
Rishi Shah [Pyspark 2.4] not able to partition the data frame by dates Thu, 01 Aug, 02:55
Gourav Sengupta Re: [Pyspark 2.4] not able to partition the data frame by dates Thu, 01 Aug, 02:58
Rishi Shah Re: [Pyspark 2.4] not able to partition the data frame by dates Thu, 01 Aug, 03:32
raman gugnani How to get logging right for Spark applications in the YARN ecosystem Thu, 01 Aug, 16:47
Tathagata Das Announcing Delta Lake 0.3.0 Fri, 02 Aug, 01:44
zenglong chen spark stream kafka wait for all data process done Fri, 02 Aug, 01:59
刘 勇 Re: spark stream kafka wait for all data process done Fri, 02 Aug, 02:37
Srinath C Re: How to get logging right for Spark applications in the YARN ecosystem Fri, 02 Aug, 04:46
Girish bhat m Re: How to get logging right for Spark applications in the YARN ecosystem Fri, 02 Aug, 10:17
raman gugnani Re: How to get logging right for Spark applications in the YARN ecosystem Fri, 02 Aug, 10:58
Gourav Sengupta Re: Announcing Delta Lake 0.3.0 Fri, 02 Aug, 16:42
Mich Talebzadeh Reading configuration file in Spark Scala throws error Sat, 03 Aug, 19:28
Chris Teoh Re: Reading configuration file in Spark Scala throws error Sun, 04 Aug, 00:55
zenglong chen where can i see the spark app output in cluster mode? Sun, 04 Aug, 10:54
Mich Talebzadeh Re: Reading configuration file in Spark Scala throws error Sun, 04 Aug, 21:32
Mich Talebzadeh Re: Reading configuration file in Spark Scala throws error Mon, 05 Aug, 00:01
chagas Incremental (online) machine learning algorithms on ML Mon, 05 Aug, 12:56
Stephen Boesch Re: Incremental (online) machine learning algorithms on ML Mon, 05 Aug, 13:18
Dennis Suhari Re: Spark and Oozie Mon, 05 Aug, 16:03
Amit Sharma spark job getting hang Mon, 05 Aug, 20:16
kant kodali How to programmatically pause and resume Spark/Kafka structured streaming? Tue, 06 Aug, 02:42
Gourav Sengupta Re: How to programmatically pause and resume Spark/Kafka structured streaming? Tue, 06 Aug, 05:53
Rishikesh Gawade Hive external table not working in sparkSQL when subdirectories are present Tue, 06 Aug, 06:57
Mich Talebzadeh Re: Hive external table not working in sparkSQL when subdirectories are present Tue, 06 Aug, 07:33
Imran Rashid CVE-2019-10099: Apache Spark unencrypted data on local disk Tue, 06 Aug, 15:17
kant kodali Re: How to programmatically pause and resume Spark/Kafka structured streaming? Tue, 06 Aug, 20:49
Peter Willis unsubscribe Tue, 06 Aug, 20:55
Information Technologies unsubscribe Tue, 06 Aug, 20:59
Mich Talebzadeh How to read configuration file parameters in Spark without mapping each parameter Tue, 06 Aug, 21:34
Nicolas Paris Re: Announcing Delta Lake 0.3.0 Tue, 06 Aug, 21:37
Information Technologies unsubscribe Tue, 06 Aug, 22:20
Gourav Sengupta Re: How to programmatically pause and resume Spark/Kafka structured streaming? Wed, 07 Aug, 05:32
Rishikesh Gawade Re: Hive external table not working in sparkSQL when subdirectories are present Wed, 07 Aug, 06:50
Jörn Franke Re: Hive external table not working in sparkSQL when subdirectories are present Wed, 07 Aug, 07:13
Hao Ren Spark SQL reads all leaf directories on a partitioned Hive table Wed, 07 Aug, 14:27
anbutech Spark scala/Hive scenario Wed, 07 Aug, 18:40
Rishikesh Gawade Re: Hive external table not working in sparkSQL when subdirectories are present Wed, 07 Aug, 19:14
Jörn Franke Re: Spark scala/Hive scenario Wed, 07 Aug, 19:32
Patrick McCarthy Re: Hive external table not working in sparkSQL when subdirectories are present Wed, 07 Aug, 20:06
Mich Talebzadeh Re: Hive external table not working in sparkSQL when subdirectories are present Wed, 07 Aug, 20:50
Mich Talebzadeh Sharing ideas on using Databricks Delta Lake Wed, 07 Aug, 21:06
Pravinkumar vp Pyspark - Kerberos authentication error Thu, 08 Aug, 03:06
Gourav Sengupta Re: Spark SQL reads all leaf directories on a partitioned Hive table Thu, 08 Aug, 11:23
Hao Ren Fwd: Spark SQL reads all leaf directories on a partitioned Hive table Thu, 08 Aug, 14:16
Mich Talebzadeh Re: Spark SQL reads all leaf directories on a partitioned Hive table Thu, 08 Aug, 19:05
Mohit Jaggi Dataset -- Schema for type scala.collection.Set[scala.Int] is not supported Fri, 09 Aug, 03:41
Hao Ren Re: Spark SQL reads all leaf directories on a partitioned Hive table Fri, 09 Aug, 10:48
Debabrata Ghosh Unable to write data from Spark into a Hive Managed table Fri, 09 Aug, 11:04
Mich Talebzadeh Re: Unable to write data from Spark into a Hive Managed table Fri, 09 Aug, 16:51
Mohit Jaggi Re: Dataset -- Schema for type scala.collection.Set[scala.Int] is not supported Fri, 09 Aug, 18:10
Sandish Kumar HN Using StreamingQueryListener.OnTerminate for Kafka Offset restore Fri, 09 Aug, 20:13
zenglong chen How to add spark structured streaming kafka source receiver Sat, 10 Aug, 04:22
zenglong chen Spark streaming dataframe extract message to new columns Sat, 10 Aug, 08:23
Gourav Sengupta Re: Spark streaming dataframe extract message to new columns Mon, 12 Aug, 12:51
Andrew Leverentz Custom aggregations: modular and lightweight solutions? Tue, 13 Aug, 00:31
zenglong chen Continuous processing mode and python udf Tue, 13 Aug, 03:36
Subash Prabakar Re: Spark SQL reads all leaf directories on a partitioned Hive table Tue, 13 Aug, 03:58
Tianlang Re: Spark streaming dataframe extract message to new columns Tue, 13 Aug, 06:04
zenglong chen how to specify which partition each record send on spark structured streaming kafka sink? Tue, 13 Aug, 06:39
霍战锋 Sreate temp table or view by using location Tue, 13 Aug, 12:00
霍战锋 Re: Sreate temp table or view by using location Tue, 13 Aug, 12:03
Marcelo Valle help understanding physical plan Tue, 13 Aug, 14:45
Amit Sharma Spark Streaming concurrent calls Tue, 13 Aug, 14:53
Andrew Leverentz Re: Custom aggregations: modular and lightweight solutions? Tue, 13 Aug, 19:59
Dongjoon Hyun Release Apache Spark 2.4.4 Tue, 13 Aug, 23:16
DB Tsai Re: Release Apache Spark 2.4.4 Tue, 13 Aug, 23:24
Takeshi Yamamuro Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 00:13
Hyukjin Kwon Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 00:20
Sean Owen Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 00:21
霍战锋 Re: Sreate temp table or view by using location Wed, 14 Aug, 02:17
Shyam P Any advice how to do this usecase in spark sql ? Wed, 14 Aug, 03:08
Kazuaki Ishizaki RE: Release Apache Spark 2.4.4 Wed, 14 Aug, 03:47
Holden Karau Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 04:51
Hyukjin Kwon Re: Continuous processing mode and python udf Wed, 14 Aug, 05:00
Jörn Franke Re: Any advice how to do this usecase in spark sql ? Wed, 14 Aug, 05:08
Wenchen Fan Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 05:24
Terry Kim Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 05:30
Hyukjin Kwon Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 06:58
Hao Ren Re: Spark SQL reads all leaf directories on a partitioned Hive table Wed, 14 Aug, 10:07
Holden Karau Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 16:13
Nick Dawes Spark Structured Streaming XML content Wed, 14 Aug, 19:05
Dongjoon Hyun Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 21:36
Tianlang Re: help understanding physical plan Thu, 15 Aug, 06:18
Tianlang Re: Spark Streaming concurrent calls Thu, 15 Aug, 06:25
ans Spark streaming kafka source delay occasionally Thu, 15 Aug, 09:11
Dennis Suhari Memory Limits error Thu, 15 Aug, 15:21
rajat kumar Call Oracle Sequence using Spark Thu, 15 Aug, 17:28
Mikhail Pryakhin Stream is corrupted in ShuffleBlockFetcherIterator Thu, 15 Aug, 20:47
Marcelo Valle Re: help understanding physical plan Fri, 16 Aug, 09:16
Amit Sharma Re: Memory Limits error Fri, 16 Aug, 11:16
Kazuaki Ishizaki Release Spark 2.3.4 Fri, 16 Aug, 14:48
Tak-Lon (Stephen) Wu Spark on YARN with private Docker repositories/registries Fri, 16 Aug, 17:12
Vadim Semenov Re: Stream is corrupted in ShuffleBlockFetcherIterator Fri, 16 Aug, 17:14
Nicolas Paris Re: Call Oracle Sequence using Spark Fri, 16 Aug, 17:25
Antoine DUBOIS Spark and ZStandard Mon, 19 Aug, 15:38
Alex Landa Spark Standalone - Failing to pass extra java options to the driver in cluster mode Mon, 19 Aug, 18:42
Jungtaek Lim Re: Spark Standalone - Failing to pass extra java options to the driver in cluster mode Mon, 19 Aug, 19:01
Rishikesh Gawade How to combine all rows into a single row in DataFrame Mon, 19 Aug, 20:23
Marcin Tustin Re: How to combine all rows into a single row in DataFrame Mon, 19 Aug, 20:35
Sandish Kumar HN Running Spark history Server at Context localhost:18080/sparkhistory Tue, 20 Aug, 00:52
Message list1 · 2 · Next »Thread · Author · Date
Box list
Sep 2019127
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137