spark-user mailing list archives: November 2018

Site index · List index
Message list1 · 2 · 3 · Next »Thread · Author · Date
☼ R Nair DB2 Sequence - Error while invoking Wed, 07 Nov, 13:37
☼ R Nair Re: Testing Apache Spark applications Thu, 15 Nov, 18:42
张万新 Re: how to use cluster sparkSession like localSession Fri, 02 Nov, 06:36
François Sarradin [Spark SQL] [Spark 2.4.0] v1 -> struct(v1.e) fails Thu, 15 Nov, 14:46
965 回复:Do we need to kill a spark job every time we change and deploy it? Fri, 30 Nov, 14:31
965 回复:Java: pass parameters in spark sql query Fri, 30 Nov, 14:39
Ирина Шершукова FW: Spark2 and Hive metastore Mon, 12 Nov, 08:12
崔苗(数据与人工智能产品开发部) use spark cluster in java web service Thu, 01 Nov, 07:22
崔苗(数据与人工智能产品开发部) how to use cluster sparkSession like localSession Fri, 02 Nov, 03:01
崔苗(数据与人工智能产品开发部) Re: how to use cluster sparkSession like localSession Fri, 02 Nov, 05:52
崔苗(数据与人工智能产品开发部) Re: how to use cluster sparkSession like localSession Fri, 02 Nov, 06:01
崔苗(数据与人工智能产品开发部) spark historyserver web ui Thu, 08 Nov, 10:59
崔苗(数据与人工智能产品开发部) programmatically set hadoop_conf_dir for spark Fri, 16 Nov, 01:37
Bjørnar Jensen [Spark ORC | SQL | Hive] Buffer size too small when using filterPushdown predicate=True (ref.: SPARK-25145) Fri, 23 Nov, 10:36
Jörn Franke Re: Apache Spark orc read performance when reading large number of small files Thu, 01 Nov, 07:19
Jörn Franke Re: How to avoid long-running jobs blocking short-running jobs Sat, 03 Nov, 09:16
Jörn Franke Re: [Spark SQL] INSERT OVERWRITE to a hive partitioned table (pointing to s3) from spark is too slow. Mon, 05 Nov, 07:08
Jörn Franke Re: writing to local files on a worker Mon, 12 Nov, 06:51
Jörn Franke Re: streaming pdf Mon, 19 Nov, 06:23
Jörn Franke Re: streaming pdf Tue, 20 Nov, 07:06
Jörn Franke Re: streaming pdf Tue, 20 Nov, 07:07
Jörn Franke Re: Zookeeper and Spark deployment for standby master Mon, 26 Nov, 08:06
Abdeali Kothari Show function name in Logs for PythonUDFRunner Thu, 22 Nov, 09:04
Abdeali Kothari Re: Show function name in Logs for PythonUDFRunner Thu, 22 Nov, 13:18
Abhijeet Kumar Spark Streaming join taking long to process Tue, 27 Nov, 08:15
Abhijeet Kumar Re: Spark Streaming join taking long to process Tue, 27 Nov, 14:16
Abhijeet Kumar Spark streaming join on yarn Wed, 28 Nov, 22:25
Akila Wajirasena Zookeeper and Spark deployment for standby master Mon, 26 Nov, 06:25
Alessandro Solimando Re: Re: spark-sql force parallel union Wed, 21 Nov, 11:02
Alex [PySpark Profiler]: Does empty profile mean no execution in Python Interpreter? Fri, 02 Nov, 03:00
Alexander Czech How to use the Graphframe PageRank method with dangling edges? Mon, 05 Nov, 10:20
Ankur Gupta Monthly Apache Spark Newsletter Wed, 21 Nov, 03:30
Arbab Khalil Re: how to use cluster sparkSession like localSession Fri, 02 Nov, 05:55
Arijit Tarafdar Questions on Python support with Spark Fri, 09 Nov, 22:04
Arun Manivannan Equivalent of emptyDataFrame in StructuredStreaming Mon, 05 Nov, 23:29
Arun Manivannan Re: Equivalent of emptyDataFrame in StructuredStreaming Sat, 17 Nov, 09:33
Bartosz Konieczny Spark 2.4.0 artifact in Maven repository Sun, 04 Nov, 15:14
Bartosz Konieczny Re: Spark 2.4.0 artifact in Maven repository Tue, 06 Nov, 11:10
Bhaskar Ebbur Re: [Spark SQL] INSERT OVERWRITE to a hive partitioned table (pointing to s3) from spark is too slow. Mon, 05 Nov, 07:30
Bhaskar Ebbur Re: [Spark SQL] INSERT OVERWRITE to a hive partitioned table (pointing to s3) from spark is too slow. Mon, 05 Nov, 23:17
Biplob Biswas Re: [Spark-Core] Long scheduling delays (1+ hour) Wed, 07 Nov, 10:53
Brandon Geise Re: How to address seemingly low core utilization on a spark workload? Thu, 15 Nov, 15:27
Chetan Khatri Spark 2.3.0 with HDP Got completely successfully but status FAILED with error Wed, 21 Nov, 18:38
Chetan Khatri How to Keep Null values in Parquet Thu, 22 Nov, 02:29
Chetan Khatri Re: How to Keep Null values in Parquet Thu, 22 Nov, 02:48
Chris Olivier StackOverflowError for simple map Thu, 01 Nov, 20:12
Chris Olivier StackOverflowError for simple map (not to incubator mailing list) Thu, 01 Nov, 20:17
Chris Olivier Re: Is there any Spark source in Java Sat, 03 Nov, 19:09
Christopher Petrino Spark column combinations and combining multiple dataframes (pyspark) Mon, 26 Nov, 17:55
Christopher Petrino Re: Job hangs in blocked task in final parquet write stage Wed, 28 Nov, 15:33
Christopher Petrino Re: Job hangs in blocked task in final parquet write stage Thu, 29 Nov, 15:05
Colin Williams inferred schemas for spark streaming from a Kafka source Tue, 13 Nov, 20:32
Colin Williams Casting nested columns and updated nested struct fields. Thu, 22 Nov, 02:25
Colin Williams Re: Casting nested columns and updated nested struct fields. Fri, 23 Nov, 16:42
Colin Williams Re: Casting nested columns and updated nested struct fields. Fri, 23 Nov, 19:35
Conrad Lee Re: Job hangs in blocked task in final parquet write stage Tue, 27 Nov, 11:29
Conrad Lee Re: Job hangs in blocked task in final parquet write stage Wed, 28 Nov, 07:47
Conrad Lee Re: Job hangs in blocked task in final parquet write stage Thu, 29 Nov, 08:02
Daniel de Oliveira Mantovani Re: how to use cluster sparkSession like localSession Fri, 02 Nov, 03:57
David Hesson Spark event logging with s3a Thu, 08 Nov, 21:36
Dilip Biswal Re: Happy Diwali everyone!!! Wed, 07 Nov, 23:11
Dillon Dukek Re: Shuffle write explosion Mon, 05 Nov, 23:21
Dipl.-Inf. Rico Bergmann Spark DataSets and multiple write(.) calls Mon, 19 Nov, 08:03
Dipl.-Inf. Rico Bergmann Re: Spark DataSets and multiple write(.) calls Mon, 19 Nov, 12:51
Dipl.-Inf. Rico Bergmann Re: Spark DataSets and multiple write(.) calls Tue, 20 Nov, 08:14
Divya Narayan Read Avro Data using Spark Streaming Sat, 03 Nov, 03:33
Dongjoon Hyun Re: [ANNOUNCE] Announcing Apache Spark 2.4.0 Thu, 08 Nov, 19:31
Eike von Seggern Re: Pyspark create RDD of dictionary Fri, 02 Nov, 15:47
Eike von Seggern Re: Show function name in Logs for PythonUDFRunner Thu, 22 Nov, 12:24
Gabor Somogyi Re: PySpark Direct Streaming : SASL Security Compatibility Issue Wed, 28 Nov, 09:11
Gabriel Wang Re: how to use cluster sparkSession like localSession Fri, 02 Nov, 08:33
Georg Heiler Re: What is BDV in Spark Source Sat, 10 Nov, 13:53
Gourav Sengupta Re: Spark DataSets and multiple write(.) calls Tue, 20 Nov, 16:28
Holden Karau Re: Is there any Spark source in Java Sat, 03 Nov, 20:06
Holden Karau Re: [Spark Shell on AWS K8s Cluster]: Is there more documentation regarding how to run spark-shell on k8s cluster? Thu, 15 Nov, 14:48
Irving Duran Re: Do we need to kill a spark job every time we change and deploy it? Wed, 28 Nov, 20:33
JF Chen How to increase the parallelism of Spark Streaming application? Wed, 07 Nov, 07:27
JF Chen Re: How to increase the parallelism of Spark Streaming application? Thu, 08 Nov, 00:13
JF Chen Re: How to increase the parallelism of Spark Streaming application? Thu, 08 Nov, 00:14
JF Chen Re: How to increase the parallelism of Spark Streaming application? Thu, 08 Nov, 01:41
JF Chen [No Subject] Thu, 08 Nov, 08:19
JF Chen spark unsupported conversion to Stringtype error Wed, 28 Nov, 07:32
Jack Kolokasis StorageLevel: OffHeap Thu, 08 Nov, 12:35
Jack Kolokasis Measure Serialization / De-serialization Time Thu, 15 Nov, 13:54
James Starks Caused by: java.io.NotSerializableException: com.softwaremill.sttp.FollowRedirectsBackend Thu, 29 Nov, 14:45
James Starks Convert RDD[Iterrable[MyCaseClass]] to RDD[MyCaseClass] Fri, 30 Nov, 14:02
James Starks Re: Caused by: java.io.NotSerializableException: com.softwaremill.sttp.FollowRedirectsBackend Fri, 30 Nov, 14:24
Jean Georges Perrin Re: Is there any Spark source in Java Sat, 03 Nov, 18:54
Jeyhun Karimov Re: Is there any Spark source in Java Sat, 03 Nov, 18:30
Joe How does shuffle operation work in Spark? Wed, 07 Nov, 16:25
Joe Re: writing to local files on a worker Mon, 12 Nov, 04:37
Joe question about barrier execution mode in Spark 2.4.0 Mon, 12 Nov, 15:33
Jules Damji Re: [ANNOUNCE] Announcing Apache Spark 2.4.0 Thu, 08 Nov, 19:36
Jungtaek Lim Re: Equivalent of emptyDataFrame in StructuredStreaming Mon, 05 Nov, 23:34
Jungtaek Lim Re: [Spark Structued Streaming]: Read kafka offset from a timestamp Tue, 20 Nov, 00:37
Jungtaek Lim Re: Spark Streaming Tue, 27 Nov, 06:38
Koert Kuipers Re: Caused by: java.io.NotSerializableException: com.softwaremill.sttp.FollowRedirectsBackend Fri, 30 Nov, 05:08
Kuttaiah Robin How to use Dataset<Row> forEachPartion and groupByKey together Thu, 01 Nov, 06:15
Kuttaiah Robin Spark Listeners for getting dataset partition information in streaming application Fri, 02 Nov, 08:59
Lars Albertsson Re: Testing Apache Spark applications Thu, 15 Nov, 20:19
Message list1 · 2 · 3 · Next »Thread · Author · Date
Box list
Oct 201966
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137