spark-user mailing list archives: August 2019

Site index · List index
Message list1 · 2 · Next »Thread · Author · Date
Rishi Shah [Pyspark 2.4] not able to partition the data frame by dates Thu, 01 Aug, 02:55
Gourav Sengupta   Re: [Pyspark 2.4] not able to partition the data frame by dates Thu, 01 Aug, 02:58
Rishi Shah     Re: [Pyspark 2.4] not able to partition the data frame by dates Thu, 01 Aug, 03:32
raman gugnani How to get logging right for Spark applications in the YARN ecosystem Thu, 01 Aug, 16:47
Srinath C   Re: How to get logging right for Spark applications in the YARN ecosystem Fri, 02 Aug, 04:46
Girish bhat m     Re: How to get logging right for Spark applications in the YARN ecosystem Fri, 02 Aug, 10:17
raman gugnani       Re: How to get logging right for Spark applications in the YARN ecosystem Fri, 02 Aug, 10:58
Tathagata Das Announcing Delta Lake 0.3.0 Fri, 02 Aug, 01:44
Gourav Sengupta   Re: Announcing Delta Lake 0.3.0 Fri, 02 Aug, 16:42
Nicolas Paris   Re: Announcing Delta Lake 0.3.0 Tue, 06 Aug, 21:37
zenglong chen spark stream kafka wait for all data process done Fri, 02 Aug, 01:59
刘 勇   Re: spark stream kafka wait for all data process done Fri, 02 Aug, 02:37
Mich Talebzadeh Reading configuration file in Spark Scala throws error Sat, 03 Aug, 19:28
Chris Teoh   Re: Reading configuration file in Spark Scala throws error Sun, 04 Aug, 00:55
Mich Talebzadeh     Re: Reading configuration file in Spark Scala throws error Sun, 04 Aug, 21:32
Mich Talebzadeh       Re: Reading configuration file in Spark Scala throws error Mon, 05 Aug, 00:01
zenglong chen where can i see the spark app output in cluster mode? Sun, 04 Aug, 10:54
Incremental (online) machine learning algorithms on ML
chagas   Incremental (online) machine learning algorithms on ML Mon, 05 Aug, 12:56
Stephen Boesch     Re: Incremental (online) machine learning algorithms on ML Mon, 05 Aug, 13:18
Re: Spark and Oozie
Dennis Suhari   Re: Spark and Oozie Mon, 05 Aug, 16:03
Amit Sharma spark job getting hang Mon, 05 Aug, 20:16
kant kodali How to programmatically pause and resume Spark/Kafka structured streaming? Tue, 06 Aug, 02:42
Gourav Sengupta   Re: How to programmatically pause and resume Spark/Kafka structured streaming? Tue, 06 Aug, 05:53
kant kodali     Re: How to programmatically pause and resume Spark/Kafka structured streaming? Tue, 06 Aug, 20:49
Gourav Sengupta       Re: How to programmatically pause and resume Spark/Kafka structured streaming? Wed, 07 Aug, 05:32
Rishikesh Gawade Hive external table not working in sparkSQL when subdirectories are present Tue, 06 Aug, 06:57
Mich Talebzadeh   Re: Hive external table not working in sparkSQL when subdirectories are present Tue, 06 Aug, 07:33
Rishikesh Gawade     Re: Hive external table not working in sparkSQL when subdirectories are present Wed, 07 Aug, 06:50
Jörn Franke       Re: Hive external table not working in sparkSQL when subdirectories are present Wed, 07 Aug, 07:13
Rishikesh Gawade         Re: Hive external table not working in sparkSQL when subdirectories are present Wed, 07 Aug, 19:14
Patrick McCarthy           Re: Hive external table not working in sparkSQL when subdirectories are present Wed, 07 Aug, 20:06
Mich Talebzadeh             Re: Hive external table not working in sparkSQL when subdirectories are present Wed, 07 Aug, 20:50
Imran Rashid CVE-2019-10099: Apache Spark unencrypted data on local disk Tue, 06 Aug, 15:17
unsubscribe
Peter Willis   unsubscribe Tue, 06 Aug, 20:55
Information Technologies   unsubscribe Tue, 06 Aug, 20:59
Information Technologies   unsubscribe Tue, 06 Aug, 22:20
Mich Talebzadeh How to read configuration file parameters in Spark without mapping each parameter Tue, 06 Aug, 21:34
Hao Ren Spark SQL reads all leaf directories on a partitioned Hive table Wed, 07 Aug, 14:27
Gourav Sengupta   Re: Spark SQL reads all leaf directories on a partitioned Hive table Thu, 08 Aug, 11:23
Hao Ren     Fwd: Spark SQL reads all leaf directories on a partitioned Hive table Thu, 08 Aug, 14:16
Mich Talebzadeh       Re: Spark SQL reads all leaf directories on a partitioned Hive table Thu, 08 Aug, 19:05
Hao Ren         Re: Spark SQL reads all leaf directories on a partitioned Hive table Fri, 09 Aug, 10:48
Subash Prabakar           Re: Spark SQL reads all leaf directories on a partitioned Hive table Tue, 13 Aug, 03:58
Hao Ren             Re: Spark SQL reads all leaf directories on a partitioned Hive table Wed, 14 Aug, 10:07
anbutech Spark scala/Hive scenario Wed, 07 Aug, 18:40
Jörn Franke   Re: Spark scala/Hive scenario Wed, 07 Aug, 19:32
Mich Talebzadeh Sharing ideas on using Databricks Delta Lake Wed, 07 Aug, 21:06
Pravinkumar vp Pyspark - Kerberos authentication error Thu, 08 Aug, 03:06
Mohit Jaggi Dataset -- Schema for type scala.collection.Set[scala.Int] is not supported Fri, 09 Aug, 03:41
Mohit Jaggi   Re: Dataset -- Schema for type scala.collection.Set[scala.Int] is not supported Fri, 09 Aug, 18:10
Debabrata Ghosh Unable to write data from Spark into a Hive Managed table Fri, 09 Aug, 11:04
Mich Talebzadeh   Re: Unable to write data from Spark into a Hive Managed table Fri, 09 Aug, 16:51
HARSH TAKKAR     Re: Unable to write data from Spark into a Hive Managed table Tue, 20 Aug, 11:00
Using StreamingQueryListener.OnTerminate for Kafka Offset restore
Sandish Kumar HN   Using StreamingQueryListener.OnTerminate for Kafka Offset restore Fri, 09 Aug, 20:13
zenglong chen How to add spark structured streaming kafka source receiver Sat, 10 Aug, 04:22
zenglong chen Spark streaming dataframe extract message to new columns Sat, 10 Aug, 08:23
Gourav Sengupta   Re: Spark streaming dataframe extract message to new columns Mon, 12 Aug, 12:51
Tianlang     Re: Spark streaming dataframe extract message to new columns Tue, 13 Aug, 06:04
Andrew Leverentz Custom aggregations: modular and lightweight solutions? Tue, 13 Aug, 00:31
Andrew Leverentz   Re: Custom aggregations: modular and lightweight solutions? Tue, 13 Aug, 19:59
zenglong chen Continuous processing mode and python udf Tue, 13 Aug, 03:36
Hyukjin Kwon   Re: Continuous processing mode and python udf Wed, 14 Aug, 05:00
zenglong chen how to specify which partition each record send on spark structured streaming kafka sink? Tue, 13 Aug, 06:39
霍战锋 Sreate temp table or view by using location Tue, 13 Aug, 12:00
霍战锋   Re: Sreate temp table or view by using location Tue, 13 Aug, 12:03
霍战锋     Re: Sreate temp table or view by using location Wed, 14 Aug, 02:17
Marcelo Valle help understanding physical plan Tue, 13 Aug, 14:45
Tianlang   Re: help understanding physical plan Thu, 15 Aug, 06:18
Marcelo Valle     Re: help understanding physical plan Fri, 16 Aug, 09:16
Amit Sharma Spark Streaming concurrent calls Tue, 13 Aug, 14:53
Tianlang   Re: Spark Streaming concurrent calls Thu, 15 Aug, 06:25
Dongjoon Hyun Release Apache Spark 2.4.4 Tue, 13 Aug, 23:16
DB Tsai   Re: Release Apache Spark 2.4.4 Tue, 13 Aug, 23:24
Takeshi Yamamuro     Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 00:13
Hyukjin Kwon       Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 00:20
Kazuaki Ishizaki         RE: Release Apache Spark 2.4.4 Wed, 14 Aug, 03:47
Sean Owen   Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 00:21
Holden Karau     Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 04:51
Wenchen Fan       Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 05:24
Terry Kim         Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 05:30
Hyukjin Kwon           Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 06:58
Holden Karau             Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 16:13
Dongjoon Hyun               Re: Release Apache Spark 2.4.4 Wed, 14 Aug, 21:36
Shyam P Any advice how to do this usecase in spark sql ? Wed, 14 Aug, 03:08
Jörn Franke   Re: Any advice how to do this usecase in spark sql ? Wed, 14 Aug, 05:08
Nick Dawes Spark Structured Streaming XML content Wed, 14 Aug, 19:05
ans Spark streaming kafka source delay occasionally Thu, 15 Aug, 09:11
Dennis Suhari Memory Limits error Thu, 15 Aug, 15:21
Amit Sharma   Re: Memory Limits error Fri, 16 Aug, 11:16
rajat kumar Call Oracle Sequence using Spark Thu, 15 Aug, 17:28
Nicolas Paris   Re: Call Oracle Sequence using Spark Fri, 16 Aug, 17:25
Mikhail Pryakhin Stream is corrupted in ShuffleBlockFetcherIterator Thu, 15 Aug, 20:47
Vadim Semenov   Re: Stream is corrupted in ShuffleBlockFetcherIterator Fri, 16 Aug, 17:14
Darshan Pandya     Re: Stream is corrupted in ShuffleBlockFetcherIterator Wed, 28 Aug, 00:44
Mikhail Pryakhin       Re: Stream is corrupted in ShuffleBlockFetcherIterator Wed, 28 Aug, 05:54
Kazuaki Ishizaki Release Spark 2.3.4 Fri, 16 Aug, 14:48
Kazuaki Ishizaki   RE: Release Spark 2.3.4 Thu, 22 Aug, 16:52
Tak-Lon (Stephen) Wu Spark on YARN with private Docker repositories/registries Fri, 16 Aug, 17:12
Antoine DUBOIS Spark and ZStandard Mon, 19 Aug, 15:38
Alex Landa Spark Standalone - Failing to pass extra java options to the driver in cluster mode Mon, 19 Aug, 18:42
Jungtaek Lim   Re: Spark Standalone - Failing to pass extra java options to the driver in cluster mode Mon, 19 Aug, 19:01
Alex Landa     Re: Spark Standalone - Failing to pass extra java options to the driver in cluster mode Tue, 20 Aug, 05:28
Message list1 · 2 · Next »Thread · Author · Date
Box list
Sep 2019103
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137