spark-user mailing list archives: January 2020

Site index · List index
Message list1 · 2 · Next »Thread · Author · Date
Sinha, Breeta (Nokia - IN/Bangalore) Java 11 support in Spark 2.5 Thu, 02 Jan, 07:18
Jatin Puri   Re: Java 11 support in Spark 2.5 Thu, 02 Jan, 11:59
Re: 'ExecutorTaskSummary' alternative in Spark 2.3 onwards
Ninja Coder   Re: 'ExecutorTaskSummary' alternative in Spark 2.3 onwards Thu, 02 Jan, 19:39
unsubscribe
Amit Jain   unsubscribe Thu, 02 Jan, 19:42
David Aspegren   unsubscribe Fri, 03 Jan, 09:27
Anton Kravchenko   unsubscribe Sat, 04 Jan, 16:21
Wan, Patric     unsubscribe Sun, 05 Jan, 05:41
SV   unsubscribe Sat, 04 Jan, 20:40
Alex Pajaron   unsubscribe Sat, 04 Jan, 21:00
Rishabh Pugalia   unsubscribe Sun, 05 Jan, 12:39
Bruno S. de Barros   unsubscribe Sun, 05 Jan, 13:05
Bruno S. de Barros   unsubscribe Mon, 06 Jan, 15:01
Pingxiao Ye   unsubscribe Fri, 17 Jan, 11:14
vijay krishna   unsubscribe Fri, 17 Jan, 11:30
Sethupathi T   unsubscribe Fri, 17 Jan, 16:14
Christian Acuña   unsubscribe Fri, 17 Jan, 18:20
Bruno S. de Barros   unsubscribe Fri, 17 Jan, 19:38
Puneet Saha     Re: unsubscribe Thu, 30 Jan, 06:48
Arcadius Ahouansou   unsubscribe Fri, 24 Jan, 02:16
femibyte MLeap and Spark ML SQLTransformer Fri, 03 Jan, 05:52
Re: How more than one spark job can write to same partition in the parquet file
Iqbal Singh   Re: How more than one spark job can write to same partition in the parquet file Sun, 05 Jan, 16:47
Mich Talebzadeh OrderBy Year and Month is not displaying correctly Sun, 05 Jan, 23:39
Enrico Minack   Re: OrderBy Year and Month is not displaying correctly Mon, 06 Jan, 09:22
Gourav Sengupta     Re: OrderBy Year and Month is not displaying correctly Mon, 06 Jan, 09:35
Mich Talebzadeh       Re: OrderBy Year and Month is not displaying correctly Mon, 06 Jan, 12:05
Rishi Shah [pyspark2.4+] A lot of tasks failed, but job eventually completes Mon, 06 Jan, 00:16
hemant singh   Re: [pyspark2.4+] A lot of tasks failed, but job eventually completes Mon, 06 Jan, 04:06
Rishi Shah     Re: [pyspark2.4+] A lot of tasks failed, but job eventually completes Mon, 06 Jan, 04:19
hemant singh       Re: [pyspark2.4+] A lot of tasks failed, but job eventually completes Mon, 06 Jan, 05:27
Enrico Minack         Re: [pyspark2.4+] A lot of tasks failed, but job eventually completes Mon, 06 Jan, 09:18
Rishi Shah           Re: [pyspark2.4+] A lot of tasks failed, but job eventually completes Mon, 06 Jan, 13:35
Enrico Minack             Re: [pyspark2.4+] When to choose RDD over Dataset, was: A lot of tasks failed, but job eventually completes Mon, 06 Jan, 19:24
Fwd: [Spark Streaming]: Why my Spark Direct stream is sending multiple offset commits to Kafka?
Raghu B   Fwd: [Spark Streaming]: Why my Spark Direct stream is sending multiple offset commits to Kafka? Mon, 06 Jan, 18:18
Unsubscribe
Rishabh Pugalia   Unsubscribe Mon, 06 Jan, 18:20
Dylan Hogg   Unsubscribe Wed, 15 Jan, 01:51
Re: Fail to use SparkR of 3.0 preview 2
Xiao Li   Re: Fail to use SparkR of 3.0 preview 2 Tue, 07 Jan, 18:47
act_coder How to disable 'spark.security.credentials.${service}.enabled' in Structured streaming while connecting to a kafka cluster Wed, 08 Jan, 14:37
Gabor Somogyi   Re: How to disable 'spark.security.credentials.${service}.enabled' in Structured streaming while connecting to a kafka cluster Fri, 10 Jan, 11:00
act_coder     Re: How to disable 'spark.security.credentials.${service}.enabled' in Structured streaming while connecting to a kafka cluster Mon, 13 Jan, 03:20
act_coder     Re: How to disable 'spark.security.credentials.${service}.enabled' in Structured streaming while connecting to a kafka cluster Mon, 13 Jan, 03:21
anbutech Merge multiple different s3 logs using pyspark 2.4.3 Thu, 09 Jan, 02:20
Gourav Sengupta   Re: Merge multiple different s3 logs using pyspark 2.4.3 Thu, 09 Jan, 12:05
Shraddha Shah     Re: Merge multiple different s3 logs using pyspark 2.4.3 Thu, 09 Jan, 13:18
Gourav Sengupta       Re: Merge multiple different s3 logs using pyspark 2.4.3 Thu, 09 Jan, 15:23
Patrick Spark Mllib logistic regression setWeightCol illegal argument exception Fri, 10 Jan, 04:01
FW: [Spark Structured Streaming] Getting all the data in flatMapGroup
Shaji U   FW: [Spark Structured Streaming] Getting all the data in flatMapGroup Fri, 10 Jan, 16:04
Shaji U   RE: [Spark Structured Streaming] Getting all the data in flatMapGroup Sun, 12 Jan, 19:09
Re: High level explanation of dropDuplicates
Rishi Shah   Re: High level explanation of dropDuplicates Sat, 11 Jan, 19:14
Miguel Morales     Re: High level explanation of dropDuplicates Sat, 11 Jan, 20:01
HARSH TAKKAR Reading 7z file in spark Mon, 13 Jan, 12:31
Enrico Minack   Re: Reading 7z file in spark Tue, 14 Jan, 20:02
Someshwar Kale     Re: Reading 7z file in spark Wed, 15 Jan, 01:36
Andrew Melo       Re: Reading 7z file in spark Wed, 15 Jan, 01:40
newroyker Why Apache Spark doesn't use Calcite? Mon, 13 Jan, 14:24
Jason Nerothin   Re: Why Apache Spark doesn't use Calcite? Mon, 13 Jan, 14:45
Michael Mior     Re: Why Apache Spark doesn't use Calcite? Mon, 13 Jan, 16:41
Matei Zaharia       Re: Why Apache Spark doesn't use Calcite? Tue, 14 Jan, 01:37
Debajyoti Roy         Re: Why Apache Spark doesn't use Calcite? Wed, 15 Jan, 17:23
Xiao Li           Re: Why Apache Spark doesn't use Calcite? Wed, 15 Jan, 17:39
Debajyoti Roy             Re: Why Apache Spark doesn't use Calcite? Wed, 15 Jan, 18:02
[No Subject]
@Sanjiv Singh   [No Subject] Tue, 14 Jan, 12:35
@Sanjiv Singh   [No Subject] Tue, 14 Jan, 16:02
Andrew A Reading Dataset from DB2 over JDBC Tue, 14 Jan, 14:28
William Briggs Structured Streaming - HDFS State Store Performance Issues Wed, 15 Jan, 05:17
Gourav Sengupta   Re: Structured Streaming - HDFS State Store Performance Issues Wed, 15 Jan, 06:43
Kalin Stoyanov Spark 2.4.4 having worse performance than 2.4.2 when running the same code [pyspark][sql] Wed, 15 Jan, 17:53
Xiao Li   Re: Spark 2.4.4 having worse performance than 2.4.2 when running the same code [pyspark][sql] Wed, 15 Jan, 17:58
Kalin Stoyanov     Re: Spark 2.4.4 having worse performance than 2.4.2 when running the same code [pyspark][sql] Wed, 15 Jan, 18:53
Xiao Li       Re: Spark 2.4.4 having worse performance than 2.4.2 when running the same code [pyspark][sql] Wed, 15 Jan, 20:20
Kalin Stoyanov         Re: Spark 2.4.4 having worse performance than 2.4.2 when running the same code [pyspark][sql] Wed, 15 Jan, 21:10
Gourav Sengupta           Re: Spark 2.4.4 having worse performance than 2.4.2 when running the same code [pyspark][sql] Wed, 15 Jan, 22:15
Gourav Sengupta   Re: Spark 2.4.4 having worse performance than 2.4.2 when running the same code [pyspark][sql] Wed, 15 Jan, 19:15
Luca Toscano Spark 2.4.4, RPC encryption and Python Thu, 16 Jan, 08:16
oripwk Cannot read case-sensitive Glue table backed by Parquet Thu, 16 Jan, 16:54
oripwk   Re: Cannot read case-sensitive Glue table backed by Parquet Fri, 17 Jan, 12:34
oripwk     Re: Cannot read case-sensitive Glue table backed by Parquet Fri, 17 Jan, 14:31
Gourav Sengupta Pytorch support in SPARK 3.0 Fri, 17 Jan, 01:45
Arwin Tio Spark Executor OOMs when writing Parquet Fri, 17 Jan, 15:11
Chris Teoh   Re: Spark Executor OOMs when writing Parquet Fri, 17 Jan, 15:30
Arwin Tio     Re: Spark Executor OOMs when writing Parquet Fri, 17 Jan, 15:37
Chris Teoh       Re: Spark Executor OOMs when writing Parquet Sat, 18 Jan, 00:32
Arwin Tio       Re: Spark Executor OOMs when writing Parquet Sat, 18 Jan, 02:06
Chris Teoh         Re: Spark Executor OOMs when writing Parquet Sat, 18 Jan, 02:15
Vadim Semenov   Re: Spark Executor OOMs when writing Parquet Fri, 17 Jan, 17:53
Arwin Tio     Re: Spark Executor OOMs when writing Parquet Sat, 18 Jan, 02:12
Phillip Henry       Re: Spark Executor OOMs when writing Parquet Tue, 21 Jan, 11:18
Arwin Tio         Re: Spark Executor OOMs when writing Parquet Tue, 21 Jan, 13:35
Vadim Semenov           Re: Spark Executor OOMs when writing Parquet Tue, 21 Jan, 17:59
anbutech Record count query parallel processing in databricks spark delta lake Fri, 17 Jan, 18:18
Farhan Misarwala   Re: Record count query parallel processing in databricks spark delta lake Sun, 19 Jan, 13:30
anbutech     Re: Record count query parallel processing in databricks spark delta lake Mon, 20 Jan, 09:24
Jeff Evans Is there a way to get the final web URL from an active Spark context Fri, 17 Jan, 22:09
Jeff Evans   Re: Is there a way to get the final web URL from an active Spark context Wed, 22 Jan, 22:31
Nick Dawes Extract value from streaming Dataframe to a variable Fri, 17 Jan, 23:27
Nick Dawes   Re: Extract value from streaming Dataframe to a variable Mon, 20 Jan, 11:42
Jungtaek Lim     Re: Extract value from streaming Dataframe to a variable Mon, 20 Jan, 22:09
Nick Dawes       Re: Extract value from streaming Dataframe to a variable Tue, 21 Jan, 16:12
stevech.hu How to prevent and track data loss/dropped due to watermark during structure streaming aggregation Sat, 18 Jan, 08:57
stevech.hu   Re: How to prevent and track data loss/dropped due to watermark during structure streaming aggregation Thu, 23 Jan, 22:19
kineret M How to implement "getPreferredLocations" in Data source v2? Sat, 18 Jan, 19:44
Russell Spitzer   Re: How to implement "getPreferredLocations" in Data source v2? Sat, 18 Jan, 21:36
V0lleyBallJunki3 Does explode lead to more usage of memory Sun, 19 Jan, 00:50
Chris Teoh   Re: Does explode lead to more usage of memory Sun, 19 Jan, 02:03
Jörn Franke   Re: Does explode lead to more usage of memory Sun, 19 Jan, 06:19
Chris Teoh     Re: Does explode lead to more usage of memory Sun, 19 Jan, 11:48
hamishberri...@tutanota.com RESTful Operations Sun, 19 Jan, 22:55
Chris Teoh   Re: RESTful Operations Mon, 20 Jan, 00:26
hamishberri...@tutanota.com   Re: RESTful Operations Mon, 20 Jan, 08:34
hemant singh     Re: RESTful Operations Mon, 20 Jan, 09:47
hamishberri...@tutanota.com       Re: RESTful Operations Mon, 20 Jan, 10:41
Jason Dai [Announcement] Analytics Zoo 0.7.0 release Mon, 20 Jan, 23:52
Jason Dai   Re: [Announcement] Analytics Zoo 0.7.0 release Tue, 21 Jan, 02:05
hamishberri...@tutanota.com Parallelism in custom Receiver Tue, 21 Jan, 13:36
Rich Bowen Call for presentations for ApacheCon North America 2020 now open Tue, 21 Jan, 15:04
anbutech Performance tuning on the Databricks pyspark 2.4.4 Tue, 21 Jan, 16:50
ayan guha   Re: Performance tuning on the Databricks pyspark 2.4.4 Tue, 21 Jan, 21:41
Nicolas Paris Best approach to write UDF Tue, 21 Jan, 17:29
Bryan Jeffrey Accumulator v2 Tue, 21 Jan, 20:29
bsikander Problems during upgrade 2.2.2 -> 2.4.4 Wed, 22 Jan, 14:37
bsikander   Re: Problems during upgrade 2.2.2 -> 2.4.4 Wed, 22 Jan, 16:20
bsikander     Re: Problems during upgrade 2.2.2 -> 2.4.4 Fri, 24 Jan, 10:19
bsikander       Re: Problems during upgrade 2.2.2 -> 2.4.4 Wed, 29 Jan, 10:12
Shixiong(Ryan) Zhu         Re: Problems during upgrade 2.2.2 -> 2.4.4 Fri, 31 Jan, 17:08
Shixiong(Ryan) Zhu           Re: Problems during upgrade 2.2.2 -> 2.4.4 Fri, 31 Jan, 17:14
bsikander             Re: Problems during upgrade 2.2.2 -> 2.4.4 Fri, 31 Jan, 22:34
Jeff Evans Possible to limit number of IPC retries on spark-submit? Wed, 22 Jan, 23:02
Jeff Evans   Re: Possible to limit number of IPC retries on spark-submit? Fri, 31 Jan, 21:53
Message list1 · 2 · Next »Thread · Author · Date
Box list
Jun 2021145
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137