spark-user mailing list archives: September 2019

Site index · List index
Message list1 · 2 · Next »Thread · Author · Date
Dongjoon Hyun [ANNOUNCE] Announcing Apache Spark 2.4.4 Sun, 01 Sep, 21:54
Wenchen Fan   Re: [ANNOUNCE] Announcing Apache Spark 2.4.4 Mon, 02 Sep, 04:27
Hyukjin Kwon     Re: [ANNOUNCE] Announcing Apache Spark 2.4.4 Mon, 02 Sep, 05:16
Wesley Peng   Re: [ANNOUNCE] Announcing Apache Spark 2.4.4 Mon, 02 Sep, 06:04
Peter Liu   Re: read image or binary files / spark 2.3 Thu, 05 Sep, 18:13
Peter Liu     Re: read binary files (for stream reader) / spark 2.3 Mon, 09 Sep, 14:07
Re: Control Sqoop job from Spark job
Chetan Khatri   Re: Control Sqoop job from Spark job Mon, 02 Sep, 11:11
Chris Teoh     Re: Control Sqoop job from Spark job Mon, 02 Sep, 21:43
Chetan Khatri   Re: Control Sqoop job from Spark job Mon, 02 Sep, 11:12
Mich Talebzadeh     Re: Control Sqoop job from Spark job Mon, 02 Sep, 18:05
Shyam P   Re: Control Sqoop job from Spark job Tue, 03 Sep, 07:30
Jörn Franke     Re: Control Sqoop job from Spark job Tue, 03 Sep, 07:29
Shyam P       Re: Control Sqoop job from Spark job Tue, 03 Sep, 13:05
Rahul Nandi Unit testing PySpark Code and doing assertion Tue, 03 Sep, 15:04
Re: EMR Spark 2.4.3 executor hang
Vadim Semenov   Re: EMR Spark 2.4.3 executor hang Tue, 03 Sep, 20:45
Natalie Ruiz Structured Streaming: How to add a listener for when a batch is complete Tue, 03 Sep, 22:25
Tathagata Das   Re: Structured Streaming: How to add a listener for when a batch is complete Wed, 04 Sep, 01:28
Gourav Sengupta     Re: Structured Streaming: How to add a listener for when a batch is complete Wed, 04 Sep, 15:02
Re: Even after VO fields are mapped using @Table and @Column annotations get error NoSuchElementException
Shyam P   Re: Even after VO fields are mapped using @Table and @Column annotations get error NoSuchElementException Wed, 04 Sep, 18:41
stevech.hu Issue with structured streaming custom data source V2 Thu, 05 Sep, 02:43
kyunam How to query StructField's metadata in spark sql? Thu, 05 Sep, 07:09
[Spark Streaming Kafka 0-10] - What was the reason for adding "spark-executor-" prefix to group id in executor configurations
Sethupathi T   [Spark Streaming Kafka 0-10] - What was the reason for adding "spark-executor-" prefix to group id in executor configurations Thu, 05 Sep, 08:05
Gabor Somogyi     Re: [Spark Streaming Kafka 0-10] - What was the reason for adding "spark-executor-" prefix to group id in executor configurations Thu, 05 Sep, 16:13
Sethupathi T       Re: [Spark Streaming Kafka 0-10] - What was the reason for adding "spark-executor-" prefix to group id in executor configurations Thu, 05 Sep, 18:30
Sethupathi T       Re: [Spark Streaming Kafka 0-10] - What was the reason for adding "spark-executor-" prefix to group id in executor configurations Thu, 05 Sep, 18:33
Gabor Somogyi         Re: [Spark Streaming Kafka 0-10] - What was the reason for adding "spark-executor-" prefix to group id in executor configurations Fri, 06 Sep, 07:38
Sethupathi T           Re: [Spark Streaming Kafka 0-10] - What was the reason for adding "spark-executor-" prefix to group id in executor configurations Fri, 06 Sep, 07:43
Himali Patel Test mail Thu, 05 Sep, 10:20
Himali Patel Tune hive query launched thru spark-yarn job. Thu, 05 Sep, 12:10
Sathi Chowdhury   Re: Tune hive query launched thru spark-yarn job. Thu, 05 Sep, 14:40
Himali Patel     Re: Tune hive query launched thru spark-yarn job. Thu, 05 Sep, 17:32
Rishikesh Gawade Collecting large dataset Thu, 05 Sep, 18:22
Marcin Tustin   Re: Collecting large dataset Thu, 05 Sep, 18:27
da zhou Start point to read source codes Thu, 05 Sep, 20:11
Hichame El Khalfi   Re: Start point to read source codes Thu, 05 Sep, 20:30
David Zhou     Re: Start point to read source codes Thu, 05 Sep, 20:33
kyunam org.apache.spark.sql.AnalysisException: Detected implicit cartesian product Thu, 05 Sep, 23:58
Shyam P how to refresh the loaded non-streaming dataframe for each steaming batch ? Fri, 06 Sep, 04:29
David Zhou   Re: how to refresh the loaded non-streaming dataframe for each steaming batch ? Fri, 06 Sep, 18:10
Shyam P     Re: how to refresh the loaded non-streaming dataframe for each steaming batch ? Fri, 06 Sep, 21:28
David Zhou       Re: how to refresh the loaded non-streaming dataframe for each steaming batch ? Fri, 06 Sep, 21:18
Shyam P         Re: how to refresh the loaded non-streaming dataframe for each steaming batch ? Fri, 06 Sep, 21:32
Shubham Chaurasia DataSourceV2: pushFilters() is not invoked for each read call - spark 2.3.2 Fri, 06 Sep, 06:24
Hyukjin Kwon   Re: DataSourceV2: pushFilters() is not invoked for each read call - spark 2.3.2 Fri, 06 Sep, 07:20
Yuta Morisawa Anonymous functions cannot be found Fri, 06 Sep, 07:06
David Zhou Question on streaming job wait and re-run Fri, 06 Sep, 21:07
Ankit Khettry OOM Error Fri, 06 Sep, 23:33
Upasana Sharma   Re: OOM Error Sat, 07 Sep, 01:22
Ankit Khettry     Re: OOM Error Sat, 07 Sep, 06:18
Chris Teoh       Re: OOM Error Sat, 07 Sep, 07:35
Ankit Khettry         Re: OOM Error Sat, 07 Sep, 09:19
Chris Teoh           Re: OOM Error Sat, 07 Sep, 09:26
Ankit Khettry             Re: OOM Error Sat, 07 Sep, 09:56
Chris Teoh               Re: OOM Error Sat, 07 Sep, 10:35
Sunil Kalra                 Re: OOM Error Sat, 07 Sep, 13:26
Ankit Khettry                   Re: OOM Error Sat, 07 Sep, 13:56
Nathan Kronenfeld Problem upgrading from 2.3.1 to 2.4.3 with gradle Mon, 09 Sep, 20:48
Kazuaki Ishizaki [ANNOUNCE] Announcing Apache Spark 2.3.4 Tue, 10 Sep, 04:37
jelmer Custom encoders and udf's Tue, 10 Sep, 10:32
csmith Deadlock using Barrier Execution Tue, 10 Sep, 15:12
Dhrubajyoti Hati script running in jupyter 6-7x faster than spark submit Tue, 10 Sep, 18:32
Patrick McCarthy   Re: script running in jupyter 6-7x faster than spark submit Tue, 10 Sep, 19:14
Dhrubajyoti Hati     Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 03:05
Stephen Boesch       Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 03:16
Dhrubajyoti Hati         Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 03:25
Stephen Boesch           Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 03:32
Abdeali Kothari         Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 03:28
Dhrubajyoti Hati           Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 04:15
Dhrubajyoti Hati             Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 07:17
Patrick McCarthy               Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 13:36
Dhrubajyoti Hati                 Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 14:03
Abdeali Kothari                   Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 14:52
Dhrubajyoti Hati                     Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 16:58
Dhrubajyoti Hati                       Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 17:02
Abdeali Kothari                         Re: script running in jupyter 6-7x faster than spark submit Wed, 11 Sep, 17:40
Re: question about pyarrow.Table to pyspark.DataFrame conversion
Bryan Cutler   Re: question about pyarrow.Table to pyspark.DataFrame conversion Tue, 10 Sep, 19:17
Natalie Ruiz Access all of the custom streaming query listeners that were registered to spark session Tue, 10 Sep, 20:18
Gabor Somogyi   Re: Access all of the custom streaming query listeners that were registered to spark session Wed, 11 Sep, 07:55
Charles vinodh Spark Kafka Streaming making progress but there is no data to be consumed Wed, 11 Sep, 21:38
Burak Yavuz   Re: Spark Kafka Streaming making progress but there is no data to be consumed Wed, 11 Sep, 22:12
Charles vinodh     Re: Spark Kafka Streaming making progress but there is no data to be consumed Wed, 11 Sep, 22:24
Sandish Kumar HN       Re: Spark Kafka Streaming making progress but there is no data to be consumed Wed, 11 Sep, 22:44
Charles vinodh         Re: Spark Kafka Streaming making progress but there is no data to be consumed Thu, 12 Sep, 01:08
Burak Yavuz           Re: Spark Kafka Streaming making progress but there is no data to be consumed Thu, 12 Sep, 01:29
Dhaval Patel             Re: Spark Kafka Streaming making progress but there is no data to be consumed Thu, 12 Sep, 02:03
Charles vinodh               Re: Spark Kafka Streaming making progress but there is no data to be consumed Thu, 12 Sep, 02:49
Kumaresh AK Exception when reading multiline JSON file Thu, 12 Sep, 17:03
Kevin Mellott   Re: Exception when reading multiline JSON file Thu, 12 Sep, 22:55
Dean Arnold Inconsistent dataset behavior between file and in-memory versions Thu, 12 Sep, 18:41
raman gugnani Monitor Spark Applications Fri, 13 Sep, 04:57
Alex Landa   Re: Monitor Spark Applications Fri, 13 Sep, 05:17
raman gugnani     Re: Monitor Spark Applications Fri, 13 Sep, 05:45
Alex Landa       Re: Monitor Spark Applications Sun, 15 Sep, 09:00
Riccardo Ferrari Cluster sizing Fri, 13 Sep, 07:39
Abhinesh Hada [Spark SQL]: Does Union operation followed by drop duplicate follows "keep first" Fri, 13 Sep, 15:43
Patrick McCarthy   Re: [Spark SQL]: Does Union operation followed by drop duplicate follows "keep first" Fri, 13 Sep, 17:20
Nathan Kronenfeld   Re: [Spark SQL]: Does Union operation followed by drop duplicate follows "keep first" Fri, 13 Sep, 19:28
Abhinesh Hada     Re: [Spark SQL]: Does Union operation followed by drop duplicate follows "keep first" Sat, 14 Sep, 20:41
Dhaval Patel       Re: [Spark SQL]: Does Union operation followed by drop duplicate follows "keep first" Sat, 14 Sep, 21:58
☼ R Nair Partitioning query Fri, 13 Sep, 19:47
grp Conflicting PySpark Storage Level Defaults? Mon, 16 Sep, 00:07
Jörn Franke   Re: Conflicting PySpark Storage Level Defaults? Mon, 16 Sep, 07:02
grp     Re: [EXTERNAL] Re: Conflicting PySpark Storage Level Defaults? Mon, 16 Sep, 22:21
RE: Classloading issues when using connectors with Uber jars with improper Shading in single Spark job
Sharma, Praneet   RE: Classloading issues when using connectors with Uber jars with improper Shading in single Spark job Mon, 16 Sep, 02:30
G R Unable to verify in-transit encryption Mon, 16 Sep, 18:25
Shyam P Can anyone suggest what is wrong with my spark job here? Mon, 16 Sep, 18:39
lk_spark how can I dynamic parse json in kafka when using Structured Streaming Tue, 17 Sep, 02:39
Tathagata Das   Re: how can I dynamic parse json in kafka when using Structured Streaming Tue, 17 Sep, 08:13
lk_spark     Re: Re: how can I dynamic parse json in kafka when using Structured Streaming Tue, 17 Sep, 08:58
Praful Rana How to integrates MLeap to Spark Structured Streaming Tue, 17 Sep, 13:32
Praful Rana How to Integrate Spark mllib Streaming Training Models To Spark Structured Streaming Tue, 17 Sep, 13:38
Message list1 · 2 · Next »Thread · Author · Date
Box list
Oct 201981
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137