spark-user mailing list archives: February 2020

Site index · List index
Message list1 · 2 · Next »Thread · Author · Date
Ashutosh Joshi [Structured Streaming] Domain data refresh with flatMapGroupsWithState Sat, 01 Feb, 03:19
Re: Extract value from streaming Dataframe to a variable
Jungtaek Lim   Re: Extract value from streaming Dataframe to a variable Mon, 03 Feb, 02:21
Re: How to prevent and track data loss/dropped due to watermark during structure streaming aggregation
Jungtaek Lim   Re: How to prevent and track data loss/dropped due to watermark during structure streaming aggregation Mon, 03 Feb, 03:06
stevech.hu     Re: How to prevent and track data loss/dropped due to watermark during structure streaming aggregation Fri, 07 Feb, 09:05
Ruijing Li Best way to read batch from Kafka and Offsets Mon, 03 Feb, 08:38
Chris Teoh   Re: Best way to read batch from Kafka and Offsets Mon, 03 Feb, 09:36
Ruijing Li     Re: Best way to read batch from Kafka and Offsets Mon, 03 Feb, 19:13
Chris Teoh       Re: Best way to read batch from Kafka and Offsets Mon, 03 Feb, 21:50
Anil Kulkarni   Re: Best way to read batch from Kafka and Offsets Mon, 03 Feb, 23:14
Burak Yavuz     Re: Best way to read batch from Kafka and Offsets Wed, 05 Feb, 00:20
Ruijing Li       Re: Best way to read batch from Kafka and Offsets Wed, 05 Feb, 00:41
Burak Yavuz         Re: Best way to read batch from Kafka and Offsets Wed, 05 Feb, 00:59
Gourav Sengupta           Re: Best way to read batch from Kafka and Offsets Wed, 05 Feb, 08:48
Ruijing Li             Re: Best way to read batch from Kafka and Offsets Thu, 06 Feb, 02:13
Ruijing Li               Re: Best way to read batch from Kafka and Offsets Thu, 06 Feb, 03:07
Ruijing Li                 Re: Best way to read batch from Kafka and Offsets Sat, 15 Feb, 15:41
Burak Yavuz                   Re: Best way to read batch from Kafka and Offsets Sun, 16 Feb, 04:16
asma zgolli shuffle mathematic formulat Tue, 04 Feb, 11:57
Aironman DirtDiver   Re: shuffle mathematic formulat Tue, 04 Feb, 12:10
edge7 Committer to use if "spark.sql.sources.partitionOverwriteMode": 'dynamic' Tue, 04 Feb, 16:39
Karthik Srinivas Data locality Wed, 05 Feb, 00:33
Karthik Srinivas [ spark-streaming ] - Data Locality issue Wed, 05 Feb, 04:59
Zhang Victor SparkAppHandle can not stop application in yarn client mode Wed, 05 Feb, 09:02
Cool Joe subscribe Thu, 06 Feb, 01:35
Marcelo Valle dataframe null safe joins given a list of columns Thu, 06 Feb, 12:45
Enrico Minack   [SPARK-30957][SQL] Null-safe variant of Dataset.join(Dataset[_], Seq[String]) Wed, 26 Feb, 09:07
Vijayant Kumar Spark Application Dynamic IP and Path assign. Fri, 07 Feb, 11:06
Dongjoon Hyun [ANNOUNCE] Announcing Apache Spark 2.4.5 Sun, 09 Feb, 01:22
Dongjoon Hyun   Re: [ANNOUNCE] Announcing Apache Spark 2.4.5 Sun, 09 Feb, 01:27
Takeshi Yamamuro     Re: [ANNOUNCE] Announcing Apache Spark 2.4.5 Sun, 09 Feb, 01:49
Hyukjin Kwon       Re: [ANNOUNCE] Announcing Apache Spark 2.4.5 Mon, 10 Feb, 08:17
Wenchen Fan         Re: [ANNOUNCE] Announcing Apache Spark 2.4.5 Tue, 11 Feb, 03:39
Jungtaek Lim           Re: [ANNOUNCE] Announcing Apache Spark 2.4.5 Tue, 11 Feb, 03:43
Nicolas PARIS Ceph / Lustre VS hdfs comparison Wed, 12 Feb, 13:31
Ashley Hoff Questions about count() performance with dataframes and parquet files Thu, 13 Feb, 04:08
David Edwards   Re: Questions about count() performance with dataframes and parquet files Thu, 13 Feb, 05:45
Ashley Hoff     Re: Questions about count() performance with dataframes and parquet files Thu, 13 Feb, 06:08
David Edwards       Re: Questions about count() performance with dataframes and parquet files Thu, 13 Feb, 06:29
Enrico Minack   Re: Questions about count() performance with dataframes and parquet files Thu, 13 Feb, 18:13
Ashley Hoff     Re: Questions about count() performance with dataframes and parquet files Fri, 14 Feb, 01:39
Nicolas PARIS     Re: Questions about count() performance with dataframes and parquet files Mon, 17 Feb, 21:06
Enrico Minack       Re: Questions about count() performance with dataframes and parquet files Tue, 18 Feb, 07:38
Nicolas PARIS         Re: Questions about count() performance with dataframes and parquet files Tue, 18 Feb, 08:26
Re: Start a standalone server as root and use it with user accounts
WranglingData   Re: Start a standalone server as root and use it with user accounts Thu, 13 Feb, 04:51
Debabrata Ghosh Environment variable for deleting .sparkStaging Thu, 13 Feb, 13:06
mailford...@gmail.com   Re: Environment variable for deleting .sparkStaging Thu, 13 Feb, 18:05
Vinod Kancharana Spark 2.4.4 with Hive 2.3.6 Fri, 14 Feb, 17:09
Ruijing Li Spark 2.4.4 has bigger memory impact than 2.3? Sat, 15 Feb, 15:37
kant kodali Connected components using GraphFrames is significantly slower than GraphX? Sun, 16 Feb, 11:41
Subash Prabakar Apache Arrow support for Apache Spark Mon, 17 Feb, 07:41
Chris Teoh   Re: Apache Arrow support for Apache Spark Mon, 17 Feb, 13:31
Zhefu PENG [ML] [How-to]: How to unload the loaded W2V model in Pyspark? Mon, 17 Feb, 13:10
Mich Talebzadeh Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods Mon, 17 Feb, 14:43
Muthu Jayakumar   Re: Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods Mon, 17 Feb, 20:27
Mich Talebzadeh     Re: Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods Mon, 17 Feb, 21:15
Muthu Jayakumar       Re: Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods Mon, 17 Feb, 21:33
Jörn Franke       Re: Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods Mon, 17 Feb, 21:33
Mich Talebzadeh         Re: Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods Mon, 17 Feb, 21:37
Mich Talebzadeh           Re: Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods Mon, 17 Feb, 22:27
Mich Talebzadeh             Re: Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods Sun, 23 Feb, 19:13
Sean Busbey               Re: Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods Sun, 23 Feb, 19:54
Jörn Franke             Re: Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods Sun, 23 Feb, 20:15
Ruijing Li Better way to debug serializable issues Tue, 18 Feb, 10:01
Maxim Gekk   Re: Better way to debug serializable issues Tue, 18 Feb, 18:24
Ruijing Li     Re: Better way to debug serializable issues Fri, 21 Feb, 00:29
julio.ces...@free.fr unsubscribe Wed, 19 Feb, 13:40
Srinivasaraghavan, Vinod   RE: unsubscribe Wed, 19 Feb, 13:41
Alexey Kovyazin   unsubscribe Wed, 19 Feb, 16:00
Pramod Biligiri Spark Streaming job having issue with Java Flight Recorder (JFR) Thu, 20 Feb, 12:12
Aelur Sadgod CBO not working? Thu, 20 Feb, 13:36
Sol Rodriguez   Re: CBO not working? Thu, 20 Feb, 14:38
Ruijing Li Integration testing Framework Spark SQL Scala Fri, 21 Feb, 02:09
Ruijing Li   Re: Integration testing Framework Spark SQL Scala Tue, 25 Feb, 18:46
ard3nte Spark RDD ouput path for data lineage Fri, 21 Feb, 09:40
Nikhil Goyal Serialization error when using scala kernel with Jupyter Fri, 21 Feb, 19:28
Apostolos N. Papadopoulos   Re: Serialization error when using scala kernel with Jupyter Fri, 21 Feb, 19:54
Monish R PowerIterationClustering Sat, 22 Feb, 00:48
Kshitij Does dataframe spark API write/create a single file instead of directory as a result of write operation. Sat, 22 Feb, 05:20
Kshitij   Re: Does dataframe spark API write/create a single file instead of directory as a result of write operation. Sat, 22 Feb, 10:38
Sebastian Piu     Re: Does dataframe spark API write/create a single file instead of directory as a result of write operation. Sat, 22 Feb, 10:45
Kshitij       Re: Does dataframe spark API write/create a single file instead of directory as a result of write operation. Sat, 22 Feb, 10:47
JARDIN Yohann         Re: Does dataframe spark API write/create a single file instead of directory as a result of write operation. Sat, 22 Feb, 12:01
Nicolas PARIS     Re: Does dataframe spark API write/create a single file instead of directory as a result of write operation. Sat, 22 Feb, 22:52
Proust (Feng Guizhou) [Travel Search & Discovery] [Spark SQL] NegativeArraySizeException When Parse InternalRow to DTO Field with Type Array[String] Sun, 23 Feb, 08:08
Sandeep Patra   Re: [Spark SQL] NegativeArraySizeException When Parse InternalRow to DTO Field with Type Array[String] Sun, 23 Feb, 12:04
Proust (Feng Guizhou) [Travel Search & Discovery]   Re: [Spark SQL] NegativeArraySizeException When Parse InternalRow to DTO Field with Type Array[String] Tue, 25 Feb, 15:28
Proust (Feng Guizhou) [Travel Search & Discovery]   Re: [Spark SQL] NegativeArraySizeException When Parse InternalRow to DTO Field with Type Array[String] Tue, 25 Feb, 15:30
dpristin setting initial state for mapGroupsWithState Mon, 24 Feb, 15:41
Anthony PONCET [SPARK Dependencies] Security Vulnerability with Xerces version < 2.12 Mon, 24 Feb, 17:05
Jianneng Li [Spark SQL] Memory problems with packing too many joins into the same WholeStageCodegen Tue, 25 Feb, 02:15
Liu Genie   Re: [Spark SQL] Memory problems with packing too many joins into the same WholeStageCodegen Tue, 25 Feb, 02:39
Jianneng Li   Re: [Spark SQL] Memory problems with packing too many joins into the same WholeStageCodegen Tue, 25 Feb, 03:06
yeikel valdes     Re: [Spark SQL] Memory problems with packing too many joins into the same WholeStageCodegen Tue, 25 Feb, 13:48
Jianneng Li     Re: [Spark SQL] Memory problems with packing too many joins into the same WholeStageCodegen Wed, 26 Feb, 01:32
Liu Genie       Re: [Spark SQL] Memory problems with packing too many joins into the same WholeStageCodegen Wed, 26 Feb, 11:58
yeikel valdes What options do I have to handle third party classes that are not serializable? Tue, 25 Feb, 16:23
Jeff Evans   Re: What options do I have to handle third party classes that are not serializable? Tue, 25 Feb, 16:25
Aniruddha P Tekade Standard practices for building dashboards for spark processed data Wed, 26 Feb, 01:23
Roland Johann   Re: Standard practices for building dashboards for spark processed data Wed, 26 Feb, 07:36
Aniruddha P Tekade     Re: [External Email] Re: Standard practices for building dashboards for spark processed data Wed, 26 Feb, 19:16
Breno Arosa   Re: Standard practices for building dashboards for spark processed data Wed, 26 Feb, 19:40
Spark join: grouping of records having same value for a particular column in same partition
ARAVIND ARUMUGHAM SETHURATHNAM   Spark join: grouping of records having same value for a particular column in same partition Wed, 26 Feb, 21:54
lec ssmi dropDuplicates and watermark in structured streaming Thu, 27 Feb, 10:30
Phillip Pienaar   Unsubscribe Thu, 27 Feb, 10:31
Tathagata Das   Re: dropDuplicates and watermark in structured streaming Fri, 28 Feb, 02:25
lec ssmi     Re: dropDuplicates and watermark in structured streaming Fri, 28 Feb, 05:11
lec ssmi       Re: dropDuplicates and watermark in structured streaming Fri, 28 Feb, 05:13
Tathagata Das       Re: dropDuplicates and watermark in structured streaming Fri, 28 Feb, 10:00
Message list1 · 2 · Next »Thread · Author · Date
Box list
Jul 2021140
Jun 2021179
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137