spark-user mailing list archives: August 2018

Site index · List index
Message list« Previous · 1 · 2 · 3 · Next »Thread · Author · Date
Koert Kuipers spark structured streaming with file based sources and sinks Mon, 06 Aug, 16:31
Koert Kuipers groupBy and then coalesce impacts shuffle partitions in unintended way Wed, 08 Aug, 19:39
Koert Kuipers Re: groupBy and then coalesce impacts shuffle partitions in unintended way Wed, 08 Aug, 19:47
Koert Kuipers Re: groupBy and then coalesce impacts shuffle partitions in unintended way Wed, 08 Aug, 20:22
Koert Kuipers Re: groupBy and then coalesce impacts shuffle partitions in unintended way Wed, 08 Aug, 20:54
Koert Kuipers Re: groupBy and then coalesce impacts shuffle partitions in unintended way Wed, 08 Aug, 20:55
Koert Kuipers Re: groupBy and then coalesce impacts shuffle partitions in unintended way Thu, 09 Aug, 05:38
Koert Kuipers Re: groupBy and then coalesce impacts shuffle partitions in unintended way Thu, 09 Aug, 06:07
Koert Kuipers Re: groupBy and then coalesce impacts shuffle partitions in unintended way Thu, 09 Aug, 14:47
Koert Kuipers something happened to MemoryStream after spark 2.3 Thu, 16 Aug, 20:52
Koert Kuipers Re: Why repartitionAndSortWithinPartitions slower than MapReducer Mon, 20 Aug, 15:29
Lehak Dharmani Can we deploy python script on a spark cluster Thu, 02 Aug, 12:46
Li Gao [K8S] Spark initContainer custom bootstrap support for Spark master Wed, 15 Aug, 16:11
Li Gao Re: [K8S] Spark initContainer custom bootstrap support for Spark master Thu, 16 Aug, 17:16
Lian Jiang spark structured streaming jobs working in HDP2.6 fail in HDP3.0 Thu, 30 Aug, 15:59
Lian Jiang Re: spark structured streaming jobs working in HDP2.6 fail in HDP3.0 Thu, 30 Aug, 17:18
Lian Jiang Re: spark structured streaming jobs working in HDP2.6 fail in HDP3.0 Thu, 30 Aug, 20:32
Luciano Resende [ANNOUNCE] Apache Toree 0.2.0-incubating Released Thu, 16 Aug, 03:17
Manu Zhang Re: Split a row into multiple rows Java Wed, 08 Aug, 06:16
Manu Zhang Re: Unable to see completed application in Spark 2 history web UI Wed, 15 Aug, 09:39
Manu Zhang Re: Unable to see completed application in Spark 2 history web UI Wed, 15 Aug, 14:11
Manu Zhang Re: Unable to see completed application in Spark 2 history web UI Thu, 16 Aug, 02:34
Manu Zhang Re: java.lang.UnsupportedOperationException: No Encoder found for Set[String] Thu, 16 Aug, 12:23
Manu Zhang Re: Unable to see completed application in Spark 2 history web UI Thu, 16 Aug, 23:01
Manu Zhang Re: java.lang.UnsupportedOperationException: No Encoder found for Set[String] Fri, 17 Aug, 01:49
Matt Cheah Re: Spark on Kubernetes: Kubernetes killing executors because of overallocation of memory Fri, 03 Aug, 00:36
Maxim Gekk Re: from_json function Wed, 15 Aug, 17:59
Michael Artz Re: Pitfalls of partitioning by host? Tue, 28 Aug, 02:11
Michael Styles Unsubscribe Thu, 30 Aug, 13:09
Mike Sukmanowsky Plans for Session Windows? Thu, 09 Aug, 15:02
Mike Sukmanowsky Re: Plans for Session Windows? Thu, 09 Aug, 19:23
Mike Sukmanowsky Re: Plans for Session Windows? Wed, 29 Aug, 17:38
Mina Aslani MultilayerPerceptronClassifier Fri, 10 Aug, 03:16
Mina Aslani How to get MultilayerPerceptronClassifier model parameters? Fri, 10 Aug, 14:37
N B DStream reduceByKeyAndWindow not using checkpointed data for inverse reducing old data Wed, 29 Aug, 23:16
Nicolas Paris Re: csv reader performance with multiline option Sat, 18 Aug, 18:08
Nikhil Goyal Driver OOM when using writing parquet Mon, 06 Aug, 23:59
Nikita Goyal Re: How do I generate current UTC timestamp in raw spark sql? Tue, 28 Aug, 09:53
Nikolay Skovpin Dynamic partitioning weird behavior Tue, 07 Aug, 14:47
Nirav Patel Re: Saving dataframes with partitionBy: append partitions, overwrite within each Wed, 01 Aug, 19:11
Nirav Patel Overwrite only specific partition with hive dynamic partitioning Wed, 01 Aug, 19:24
Nirav Patel Re: Saving dataframes with partitionBy: append partitions, overwrite within each Thu, 02 Aug, 18:37
Nirav Patel Re: Saving dataframes with partitionBy: append partitions, overwrite within each Thu, 02 Aug, 18:50
Nirav Patel Insert into dynamic partitioned hive/parquet table throws error - Partition spec contains non-partition columns Fri, 03 Aug, 00:01
Nirav Patel Re: Insert into dynamic partitioned hive/parquet table throws error - Partition spec contains non-partition columns Tue, 07 Aug, 18:01
Nirav Patel csv reader performance with multiline option Sat, 18 Aug, 16:07
Nirav Patel CSV parser - how to parse column containing json data Thu, 30 Aug, 23:19
Patrick Alwell Re: Two different Hive instances running Fri, 17 Aug, 20:29
Patrick McCarthy Re: How to merge multiple rows Wed, 22 Aug, 20:32
Patrick McCarthy Pitfalls of partitioning by host? Mon, 27 Aug, 17:22
Patrick McCarthy Re: Pitfalls of partitioning by host? Tue, 28 Aug, 14:28
Patrick McCarthy Re: Pitfalls of partitioning by host? Tue, 28 Aug, 14:29
Patrick McCarthy Re: Pitfalls of partitioning by host? Tue, 28 Aug, 14:31
Patrick McCarthy Re: Pitfalls of partitioning by host? Tue, 28 Aug, 14:56
Patrick McCarthy Re: Pitfalls of partitioning by host? Tue, 28 Aug, 14:59
Patrick McCarthy Re: Pitfalls of partitioning by host? Tue, 28 Aug, 14:59
Patrick McCarthy Re: Pitfalls of partitioning by host? Tue, 28 Aug, 18:06
Patrick McCarthy Re: Pitfalls of partitioning by host? Tue, 28 Aug, 18:09
Peter Liu re: streaming, batch / spark 2.2.1 Thu, 02 Aug, 18:42
Peter Liu Re: [External Sender] re: streaming, batch / spark 2.2.1 Thu, 02 Aug, 20:48
Polisetti, Venkata Siva Rama Gopala Krishna java.nio.file.FileSystemException: /tmp/spark- .._cache : No space left on device Fri, 17 Aug, 13:20
Polisetti, Venkata Siva Rama Gopala Krishna : Failed to create file system watcher service: User limit of inotify instances reached or too many open files Wed, 22 Aug, 08:24
Pranav Agrawal need workaround around HIVE-11625 / DISTRO-800 Tue, 07 Aug, 08:19
Pranav Agrawal Re: need workaround around HIVE-11625 / DISTRO-800 Wed, 08 Aug, 06:17
Ramaswamy, Muthuraman Structured Streaming : Custom Source and Sink Development and PySpark. Fri, 31 Aug, 01:23
Reynold Xin Re: Fw:multiple group by action Sat, 25 Aug, 03:15
Richard Siebeling Use Spark extension points to implement row-level security Fri, 17 Aug, 06:55
Richard Siebeling Re: Use Spark extension points to implement row-level security Sat, 18 Aug, 12:24
Ricky read snappy compressed files in spark Fri, 31 Aug, 10:35
Robb Greathouse Re: How to add a new source to exsting struct streaming application, like a kafka source Wed, 01 Aug, 16:36
Robb Greathouse Re: Machine Learning with window data Fri, 03 Aug, 14:15
Rosbrook, Andrew J Slow Query Plan Generation Fri, 24 Aug, 10:37
Rosbrook, Andrew J RE: Slow Query Plan Generation Tue, 28 Aug, 11:44
Russell Spitzer Re: Structured Streaming : Custom Source and Sink Development and PySpark. Fri, 31 Aug, 03:23
Ryan Adams unsubscribe Fri, 10 Aug, 14:23
Sam Lendle Why is the max iteration for svd not configurable in mllib? Fri, 10 Aug, 18:15
Sean Owen CVE-2018-11770: Apache Spark standalone master, Mesos REST APIs not controlled by authentication Mon, 13 Aug, 14:24
Serkan TAS Java API for statistics of spark job running on yarn Wed, 15 Aug, 12:00
Sherif Hamdy Re: Spark Structured Streaming using S3 as data source Mon, 27 Aug, 06:19
Shuporno Choudhury Clearing usercache on EMR [pyspark] Wed, 01 Aug, 07:19
Shuporno Choudhury Re: Clearing usercache on EMR [pyspark] Fri, 03 Aug, 06:43
Sonal Goyal Re: How to deal with context dependent computing? Thu, 23 Aug, 10:14
Sonal Goyal Re: Caching small Rdd's take really long time and Spark seems frozen Thu, 23 Aug, 13:28
Sonal Goyal Re: Caching small Rdd's take really long time and Spark seems frozen Fri, 24 Aug, 10:42
Sonal Goyal Re: Pitfalls of partitioning by host? Tue, 28 Aug, 17:03
Sonal Goyal Re: Spark code to write to MySQL and Hive Thu, 30 Aug, 06:25
Sonal Goyal Re: Default Java Opts Standalone Thu, 30 Aug, 17:23
Spico Florin Run/install tensorframes on zeppelin pyspark Wed, 08 Aug, 13:59
Spico Florin Re: Run/install tensorframes on zeppelin pyspark Fri, 10 Aug, 08:47
Steve Lewis No space left on device Mon, 20 Aug, 16:08
Subhash Sriram JdbcRDD - schema always resolved as nullable=true Thu, 16 Aug, 02:58
Swapnil Chougule Spark udf from external jar without enabling Hive Wed, 29 Aug, 10:42
Swapnil Chougule Type change support in spark parquet read-write Fri, 31 Aug, 09:32
Tathagata Das Re: [Structured Streaming] Two watermarks and StreamingQueryListener Fri, 10 Aug, 23:14
Uttam Data quality measurement for streaming data with apache spark Wed, 01 Aug, 10:11
V0lleyBallJunki3 java.lang.UnsupportedOperationException: No Encoder found for Set[String] Thu, 16 Aug, 01:59
Vadim Semenov Re: Broadcast variable size limit? Sun, 05 Aug, 21:38
Vadim Semenov Re: groupBy and then coalesce impacts shuffle partitions in unintended way Wed, 08 Aug, 20:13
Vadim Semenov Re: java.lang.IndexOutOfBoundsException: len is negative - when data size increases Thu, 16 Aug, 15:36
Vaibhav Kulkarni Shuffle uses Direct Memory Buffer even after setting "spark.shuffle.io.preferDirectBufs = false" Wed, 15 Aug, 15:30
Message list« Previous · 1 · 2 · 3 · Next »Thread · Author · Date
Box list
Sep 2021103
Aug 2021171
Jul 2021158
Jun 2021179
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137