spark-user mailing list archives: August 2018

Site index · List index
Message list1 · 2 · 3 · Next »Thread · Author · Date
周浥尘 Why repartitionAndSortWithinPartitions slower than MapReducer Mon, 20 Aug, 12:52
周浥尘 Re: Why repartitionAndSortWithinPartitions slower than MapReducer Mon, 20 Aug, 15:21
崔苗 Fw:multiple group by action Sat, 25 Aug, 02:55
崔苗 is spark TempView thread safe Fri, 31 Aug, 06:50
Guillermo Ortiz Fernández Refresh broadcast variable when it isn't the value. Sun, 19 Aug, 20:41
Guillermo Ortiz Fernández Spark Streaming - Kafka. java.lang.IllegalStateException: This consumer has already been closed. Wed, 29 Aug, 07:10
Guillermo Ortiz Fernández java.lang.OutOfMemoryError: Java heap space - Spark driver. Wed, 29 Aug, 08:38
Guillermo Ortiz Fernández Re: Spark Streaming - Kafka. java.lang.IllegalStateException: This consumer has already been closed. Wed, 29 Aug, 20:40
Maximiliano Patricio Méndez Dynamic Allocation not removing executors Wed, 15 Aug, 19:38
Maximiliano Patricio Méndez Re: Use Spark extension points to implement row-level security Fri, 17 Aug, 13:33
Happy每一天 Unsubscribe Tue, 21 Aug, 03:39
네이버 unsubscribe Thu, 09 Aug, 03:32
Jörn Franke Re: Broadcast variable size limit? Sun, 05 Aug, 15:31
Jörn Franke Re: Spark Sparser library Fri, 10 Aug, 07:06
Jörn Franke Re: How to parallelize zip file processing? Fri, 10 Aug, 21:30
Aakash Basu Accessing a dataframe from another Singleton class (Python) Mon, 13 Aug, 06:47
Aakash Basu How to convert Spark Streaming to Static Dataframe on the fly and pass it to a ML Model as batch Tue, 14 Aug, 07:31
Aakash Basu RDD Collect Issue Tue, 28 Aug, 12:08
Aakash Basu Which Py4J version goes with Spark 2.3.1? Wed, 29 Aug, 06:59
Akash Mishra Understanding spark.executor.memoryOverhead Thu, 09 Aug, 10:14
Alexander Chermenin Custom state store provider based on RocksDB Tue, 14 Aug, 12:40
Anton Puzanov How to make Yarn dynamically allocate resources for Spark Wed, 01 Aug, 07:30
Anton Puzanov How to make Yarn dynamically allocate resources for Spark Wed, 01 Aug, 08:27
Anton Puzanov Re: Split a row into multiple rows Java Wed, 01 Aug, 20:41
Apostolos N. Papadopoulos Re: Parallelism: behavioural difference in version 1.2 and 2.1!? Wed, 29 Aug, 14:06
Arun Mahadevan Re: Plans for Session Windows? Thu, 09 Aug, 17:12
Arun Mahadevan Re: Plans for Session Windows? Thu, 09 Aug, 20:29
Arun Mahadevan Re: Plans for Session Windows? Wed, 29 Aug, 18:18
Bang Xiao How to use 'insert overwrite [local] directory' correctly? Mon, 27 Aug, 07:33
Bang Xiao Re: How to use 'insert overwrite [local] directory' correctly? Mon, 27 Aug, 08:10
Bang Xiao Re: How to use 'insert overwrite [local] directory' correctly? Mon, 27 Aug, 09:46
Bathi CCDB Replacing groupBykey() with reduceByKey() Fri, 03 Aug, 22:05
Bathi CCDB Re: Replacing groupBykey() with reduceByKey() Mon, 06 Aug, 16:28
Biplob Biswas Re: Replacing groupBykey() with reduceByKey() Mon, 06 Aug, 08:20
Biplob Biswas Re: Replacing groupBykey() with reduceByKey() Wed, 08 Aug, 12:54
Brandon Geise from_json schema order Wed, 15 Aug, 22:36
Brandon Geise Re: CSV parser - how to parse column containing json data Fri, 31 Aug, 00:29
Bryan Jeffrey ConcurrentModificationExceptions with CachedKafkaConsumers Thu, 30 Aug, 17:27
Bryan Jeffrey Re: ConcurrentModificationExceptions with CachedKafkaConsumers Fri, 31 Aug, 13:55
Bryan Jeffrey Re: ConcurrentModificationExceptions with CachedKafkaConsumers Fri, 31 Aug, 16:56
Burak Yavuz Re: Spark Structured Streaming using S3 as data source Sun, 26 Aug, 22:11
Christiaan Ras Machine Learning with window data Fri, 03 Aug, 10:01
ClockSlave Error in java_gateway.py Thu, 09 Aug, 06:00
Cody Koeninger Re: Spark Streaming - Kafka. java.lang.IllegalStateException: This consumer has already been closed. Wed, 29 Aug, 20:28
Cody Koeninger Re: ConcurrentModificationExceptions with CachedKafkaConsumers Thu, 30 Aug, 18:56
Cody Koeninger Re: Spark Streaming - Kafka. java.lang.IllegalStateException: This consumer has already been closed. Thu, 30 Aug, 19:00
Cody Koeninger Re: ConcurrentModificationExceptions with CachedKafkaConsumers Fri, 31 Aug, 15:56
Daniel Zhang Intellij run Spark unit test Thu, 09 Aug, 00:35
David Rosenstrauch Re: How to add a new source to exsting struct streaming application, like a kafka source Wed, 01 Aug, 17:59
Deepak Sharma java.lang.IndexOutOfBoundsException: len is negative - when data size increases Thu, 16 Aug, 15:25
Eco Super unsubscribe Thu, 02 Aug, 06:24
Esa Heikkinen Spark CEP Tue, 14 Aug, 13:20
Evelyn Bayes Default Java Opts Standalone Thu, 30 Aug, 11:42
Fabio Wada Two different Hive instances running Fri, 17 Aug, 18:21
Fawze Abujaber Unable to see completed application in Spark 2 history web UI Wed, 08 Aug, 04:56
Fawze Abujaber Re: Unable to see completed application in Spark 2 history web UI Mon, 13 Aug, 08:53
Fawze Abujaber Re: Unable to see completed application in Spark 2 history web UI Wed, 15 Aug, 10:38
Fawze Abujaber Re: Unable to see completed application in Spark 2 history web UI Wed, 15 Aug, 14:25
Fawze Abujaber Re: Unable to see completed application in Spark 2 history web UI Thu, 16 Aug, 07:05
Fawze Abujaber Re: Unable to see completed application in Spark 2 history web UI Fri, 17 Aug, 09:02
Gerard Maas Re: How to convert Spark Streaming to Static Dataframe on the fly and pass it to a ML Model as batch Tue, 14 Aug, 09:51
Gerard Maas Re: About the question of Spark Structured Streaming window output Sun, 26 Aug, 21:00
Gerard Maas Re: Re: About the question of Spark Structured Streaming window output Mon, 27 Aug, 09:26
Girish Subramanian Re: [Structured Streaming] Two watermarks and StreamingQueryListener Sat, 11 Aug, 02:47
Gourav Sengupta Re: Newbie question on how to extract column value Tue, 07 Aug, 15:33
Gourav Sengupta Re: How to convert Spark Streaming to Static Dataframe on the fly and pass it to a ML Model as batch Tue, 14 Aug, 11:37
Gourav Sengupta Re: No space left on device Wed, 22 Aug, 06:36
Gourav Sengupta Re: No space left on device Wed, 22 Aug, 10:45
Gourav Sengupta Re: Which Py4J version goes with Spark 2.3.1? Wed, 29 Aug, 13:02
Great Info Handling Very Large volume(500TB) data using spark Sat, 25 Aug, 02:54
Guillermo Ortiz Caching small Rdd's take really long time and Spark seems frozen Thu, 23 Aug, 13:08
Guillermo Ortiz Re: Caching small Rdd's take really long time and Spark seems frozen Thu, 23 Aug, 20:43
Guillermo Ortiz Re: Caching small Rdd's take really long time and Spark seems frozen Fri, 24 Aug, 09:56
Guillermo Ortiz Local mode vs client mode with one executor Thu, 30 Aug, 21:00
HARSH TAKKAR Re: executing stored procedure through spark Mon, 13 Aug, 06:32
Hichame El Khalfi Kryoserializer with pyspark Thu, 09 Aug, 17:25
JF Chen How to deal with context dependent computing? Thu, 23 Aug, 02:52
JF Chen Re: How to deal with context dependent computing? Mon, 27 Aug, 01:38
Jacek Laskowski Re: Spark code to write to MySQL and Hive Wed, 29 Aug, 15:26
James Starks Newbie question on how to extract column value Tue, 07 Aug, 15:09
James Starks Re: Newbie question on how to extract column value Tue, 07 Aug, 16:12
James Starks Data source jdbc does not support streamed reading Wed, 08 Aug, 16:23
James Starks Pass config file through spark-submit Thu, 16 Aug, 14:29
James Starks Re: Pass config file through spark-submit Fri, 17 Aug, 09:05
Jayesh Lalwani Spark on Kubernetes: Kubernetes killing executors because of overallocation of memory Thu, 02 Aug, 19:34
Jayesh Lalwani Re: [External Sender] re: streaming, batch / spark 2.2.1 Thu, 02 Aug, 20:11
Jayesh Lalwani Does row_number over a window cause a shuffle? Fri, 03 Aug, 15:15
Jayesh Lalwani Re: [External Sender] Pitfalls of partitioning by host? Tue, 28 Aug, 17:22
Jean Georges Perrin Re: How to merge multiple rows Wed, 22 Aug, 20:12
Jeevan K. Srivatsa Re: java.nio.file.FileSystemException: /tmp/spark- .._cache : No space left on device Fri, 17 Aug, 15:13
Jeevan K. Srivatsa Re: Parallelism: behavioural difference in version 1.2 and 2.1!? Wed, 29 Aug, 14:40
Jeff Zhang Re: Run/install tensorframes on zeppelin pyspark Thu, 09 Aug, 00:52
John Zhuge Re: Handle BlockMissingException in pyspark Mon, 06 Aug, 19:49
Jungtaek Lim Re: groupBy and then coalesce impacts shuffle partitions in unintended way Thu, 09 Aug, 05:15
Jungtaek Lim Re: groupBy and then coalesce impacts shuffle partitions in unintended way Thu, 09 Aug, 07:10
Jungtaek Lim Re: Structured Streaming doesn't write checkpoint log when I use coalesce Thu, 09 Aug, 12:27
Jungtaek Lim Re: Re: About the question of Spark Structured Streaming window output Mon, 27 Aug, 03:01
Kazuaki Ishizaki Re: Slow Query Plan Generation Fri, 24 Aug, 17:11
Keith Chapman Pyspark error when converting string to timestamp in map function Fri, 17 Aug, 23:50
Koert Kuipers Re: Saving dataframes with partitionBy: append partitions, overwrite within each Wed, 01 Aug, 23:18
Message list1 · 2 · 3 · Next »Thread · Author · Date
Box list
May 2019222
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137