spark-user mailing list archives: June 2018

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · Next »Thread · Author · Date
Dhruv Kumar [Spark Structured Streaming] Measure metrics from CsvSink for Rate source Fri, 22 Jun, 02:49
Dhruv Kumar Re: [Spark Structured Streaming] Measure metrics from CsvSink for Rate source Fri, 22 Jun, 05:35
Dhruv Kumar Re: [Spark Structured Streaming] Measure metrics from CsvSink for Rate source Fri, 22 Jun, 18:12
Dhruv Kumar Re: [Spark Structured Streaming] Measure metrics from CsvSink for Rate source Thu, 28 Jun, 22:15
Donni Khan the best tool to interact with Spark Tue, 26 Jun, 12:21
Elior Malul Re: spark partitionBy with partitioned column in json output Tue, 05 Jun, 06:54
Elior Malul Re: RepartitionByKey Behavior Fri, 22 Jun, 06:35
Eyal Zituny Re: [Help] Codegen Stage grows beyond 64 KB Sun, 17 Jun, 10:25
Farshid Zavareh [Spark Streaming] Spark Streaming with S3 vs Kinesis Mon, 25 Jun, 22:59
Farshid Zavareh Re: [Spark Streaming] Spark Streaming with S3 vs Kinesis Thu, 28 Jun, 07:00
Felix Cheung Re: Spark 2.3.1 not working on Java 10 Thu, 21 Jun, 14:41
Georg Heiler Re: best practices to implement library of custom transformations of Dataframe/Dataset Mon, 18 Jun, 19:49
Georg Heiler Re: Best way to process this dataset Tue, 19 Jun, 05:05
Gerard Maas Re: [Spark Streaming] Measure latency Tue, 26 Jun, 11:34
Gerard Maas Re: How to reduceByKeyAndWindow in Structured Streaming? Thu, 28 Jun, 09:24
Girish Subramanian Kafka streaming maxOffsetsPerTrigger Fri, 22 Jun, 08:06
Girish Subramanian Re: Recommendation of using StreamSinkProvider for a custom KairosDB Sink Mon, 25 Jun, 23:54
Holden Karau Re: Dataframe from 1.5G json (non JSONL) Tue, 05 Jun, 20:15
Holden Karau Spark ML online serving Thu, 07 Jun, 00:10
Holden Karau Re: Live Streamed Code Review today at 11am Pacific Fri, 08 Jun, 04:10
Holden Karau Re: Live Streamed Code Review today at 11am Pacific Thu, 14 Jun, 13:07
Holden Karau Re: Live Streamed Code Review today at 11am Pacific Wed, 27 Jun, 17:44
Hyukjin Kwon Re: Issue upgrading to Spark 2.3.1 (Maintenance Release) Fri, 15 Jun, 16:16
Hyukjin Kwon Re: Issue upgrading to Spark 2.3.1 (Maintenance Release) Fri, 15 Jun, 16:18
Irving Duran Re: If there is timestamp type data in DF, Spark 2.3 toPandas is much slower than spark 2.2. Fri, 08 Jun, 01:04
Irving Duran Re: [announce] BeakerX supports Scala+Spark in Jupyter Fri, 08 Jun, 01:09
Irving Duran Re: [Spark] Supporting python 3.5? Tue, 19 Jun, 13:44
Irving Duran Re: spark-shell doesn't start Tue, 19 Jun, 13:48
Jacek Laskowski Re: Spark 2.4 release date Mon, 18 Jun, 20:05
Javier Pareja Long and consistent wait between tasks in streaming job Thu, 07 Jun, 16:44
Javier Pareja Re: Long and consistent wait between tasks in streaming job Thu, 07 Jun, 21:59
Javier Pareja Re: Long and consistent wait between tasks in streaming job Thu, 07 Jun, 23:31
Jay Re: Append In-Place to S3 Sat, 02 Jun, 06:49
Jay Re: [PySpark] Releasing memory after a spark job is finished Tue, 05 Jun, 02:41
Jay Re: spark partitionBy with partitioned column in json output Tue, 05 Jun, 02:44
Jay Re: Dataframe from 1.5G json (non JSONL) Wed, 06 Jun, 14:28
Jay Re: Reg:- Py4JError in Windows 10 with Spark Wed, 06 Jun, 14:32
Jean Georges Perrin A code example of Catalyst optimization Mon, 04 Jun, 18:54
Jean Georges Perrin Re: submitting dependencies Wed, 27 Jun, 14:03
Jeff Zhang Re: Spark YARN Error - triggering spark-shell Fri, 08 Jun, 08:52
Jorge Machado Re: [Spark SQL] error in performing dataset union with complex data type (struct, list) Mon, 04 Jun, 09:01
Jorge Machado Re: [Spark SQL] error in performing dataset union with complex data type (struct, list) Mon, 04 Jun, 09:25
Jules Damji Re: Create an Empty dataframe Sat, 30 Jun, 15:38
Jungtaek Lim Re: RepartitionByKey Behavior Fri, 22 Jun, 00:07
Jungtaek Lim Re: [Spark Structured Streaming] Measure metrics from CsvSink for Rate source Fri, 22 Jun, 04:07
Kazuaki Ishizaki Re: Strange codegen error for SortMergeJoin in Spark 2.2.1 Thu, 07 Jun, 06:49
Kazuaki Ishizaki Re: [Help] Codegen Stage grows beyond 64 KB Wed, 20 Jun, 16:11
Kazuaki Ishizaki Re: [Help] Codegen Stage grows beyond 64 KB Wed, 20 Jun, 17:11
Keith Chapman Re: GC- Yarn vs Standalone K8 Tue, 12 Jun, 05:42
Koert Kuipers Re: Dataframe vs Dataset dilemma: either Row parsing or no filter push-down Mon, 18 Jun, 21:07
Kyunam Kim how to call database specific function when reading writing thru jdbc Fri, 08 Jun, 01:08
Lalwani, Jayesh Re: Spark structured streaming generate output path runtime Fri, 01 Jun, 13:39
Lalwani, Jayesh Re: spark partitionBy with partitioned column in json output Tue, 05 Jun, 02:41
Lalwani, Jayesh Re: Does Spark Structured Streaming have a JDBC sink or Do I need to use ForEachWriter? Thu, 21 Jun, 13:49
Lalwani, Jayesh Re: Spark 2.3.0 and Custom Sink Thu, 21 Jun, 20:36
Lalwani, Jayesh Can we get the partition Index in an UDF Mon, 25 Jun, 15:16
Lalwani, Jayesh Re: Increase no of tasks Tue, 26 Jun, 12:28
Lars Albertsson Re: testing frameworks Tue, 12 Jun, 15:51
Li Gao Spark 2.4 release date Mon, 18 Jun, 19:41
Li Liang One part of Spark MLlib Kmean Logic Performance problem Fri, 29 Jun, 08:32
Lian Jiang load hbase data using spark Mon, 18 Jun, 21:37
Luciano Resende [ANNOUNCE] Apache Bahir 2.1.2 Released Thu, 07 Jun, 08:53
Luciano Resende [ANNOUNCE] Apache Bahir 2.2.1 Released Wed, 27 Jun, 09:15
Mahender Sarangam Internal table stored NULL as \N. How to remove it Sat, 23 Jun, 10:50
Majid Azimi [Spark Streaming] Are SparkListener/StreamingListener callbacks called concurrently? Wed, 20 Jun, 07:56
Marcelo Vanzin Re: [SparkLauncher] stateChanged event not received in standalone cluster mode Wed, 06 Jun, 17:56
Marcelo Vanzin [ANNOUNCE] Announcing Apache Spark 2.3.1 Mon, 11 Jun, 19:47
Marcelo Vanzin Re: Spark user classpath setting Thu, 14 Jun, 20:37
Marcelo Vanzin Re: Issue upgrading to Spark 2.3.1 (Maintenance Release) Fri, 15 Jun, 16:11
Martin Peng How to work around NoOffsetForPartitionException when using Spark Streaming Fri, 01 Jun, 17:29
Matei Zaharia Re: how can I run spark job in my environment which is a single Ubuntu host with no hadoop installed Mon, 18 Jun, 01:21
Matteo Cossu Re: Help explaining explain() after DataFrame join reordering Tue, 05 Jun, 08:38
Matteo Cossu Re: Best way to process this dataset Tue, 19 Jun, 08:04
Mina Aslani Semi-Supervised self-training (e.g. partial fitting) Wed, 27 Jun, 15:28
Mohamed Nadjib MAMI Help explaining explain() after DataFrame join reordering Fri, 01 Jun, 16:31
Muthu Jayakumar Spark + CDB (Cockroach DB) support... Fri, 15 Jun, 21:38
Nathan Kronenfeld Re: Building SparkML vectors from long data Tue, 12 Jun, 19:59
Nathan Kronenfeld Re: RepartitionByKey Behavior Fri, 22 Jun, 14:29
Nicolas Paris Re: Dataframe from 1.5G json (non JSONL) Tue, 05 Jun, 20:37
Nicolas Paris Re: Dataframe from 1.5G json (non JSONL) Tue, 05 Jun, 20:55
Nicolas Paris Re: Best way to process this dataset Tue, 19 Jun, 20:36
Nikhil Goyal Zstd codec for writing dataframes Mon, 18 Jun, 19:31
Nirav Patel Spark sql creating managed table with location converts it to external table Fri, 22 Jun, 19:39
Patrick McCarthy Building SparkML vectors from long data Tue, 12 Jun, 18:24
Patrick McGloin How to handle java.sql.Date inside Maps with to_json / from_json Thu, 28 Jun, 09:53
Patrick McGloin Re: How to handle java.sql.Date inside Maps with to_json / from_json Thu, 28 Jun, 10:35
Peter Liu re: streaming - kafka partition transition time from (stage change logger) Mon, 11 Jun, 14:51
Peter Liu Re: spark 2.3.1 with kafka spark-streaming-kafka-0-10 (java.lang.AbstractMethodError) Thu, 28 Jun, 22:13
Phillip Henry Using checkpoint much, much faster than cache. Why? Tue, 05 Jun, 14:06
Pietro Gentile spark kudu issues Wed, 20 Jun, 15:31
Polisetti, Venkata Siva Rama Gopala Krishna Scala Partition Question Tue, 12 Jun, 12:02
Pranav Agrawal [Spark SQL] error in performing dataset union with complex data type (struct, list) Sat, 02 Jun, 17:44
Pranav Agrawal [Spark SQL] error in performing dataset union with complex data type (struct, list) Sat, 02 Jun, 17:48
Pranav Agrawal Re: [Spark SQL] error in performing dataset union with complex data type (struct, list) Mon, 04 Jun, 08:17
Pranav Agrawal Re: [Spark SQL] error in performing dataset union with complex data type (struct, list) Mon, 04 Jun, 09:09
Pranav Agrawal Re: [Spark SQL] error in performing dataset union with complex data type (struct, list) Mon, 04 Jun, 12:04
Prem Sure Re: [Spark Optimization] Why is one node getting all the pressure? Sat, 16 Jun, 11:22
Prem Sure Re: How to set spark.driver.memory? Tue, 19 Jun, 16:52
Rahul Agrawal Spark 2.3.1 not working on Java 10 Thu, 21 Jun, 14:22
Rahul Agrawal Re: Spark 2.3.1 not working on Java 10 Thu, 21 Jun, 15:27
Message list« Previous · 1 · 2 · 3 · 4 · Next »Thread · Author · Date
Box list
Jun 2019228
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137