spark-user mailing list archives: September 2017

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · 5 · Next »Thread · Author · Date
Patrick Duin [Spark Dataframe] How can I write a correct filter so the Hive table partitions are pruned correctly Wed, 13 Sep, 13:06
JG Perrin [Structured Streaming] Multiple sources best practice/recommendation Wed, 13 Sep, 14:56
Michael Armbrust   Re: [Structured Streaming] Multiple sources best practice/recommendation Thu, 14 Sep, 19:31
沈志宏 compile error: No classtag available while calling RDD.zip() Wed, 13 Sep, 15:07
Anastasios Zouzias   Re: compile error: No classtag available while calling RDD.zip() Wed, 13 Sep, 18:10
bluejoe     Re: compile error: No classtag available while calling RDD.zip() Thu, 14 Sep, 00:54
Yana Kadiyska HiveThriftserver does not seem to respect partitions Wed, 13 Sep, 15:17
Swapnil Shinde Minimum cost flow problem solving in Spark Wed, 13 Sep, 15:40
Michael Malak   Re: Minimum cost flow problem solving in Spark Wed, 13 Sep, 15:47
johan.grande....@orange.com RDD order preservation through transformations Wed, 13 Sep, 16:16
Suzen, Mehmet   Re: RDD order preservation through transformations Wed, 13 Sep, 16:24
Ankit Maloo     Re: RDD order preservation through transformations Wed, 13 Sep, 16:39
Suzen, Mehmet       Re: RDD order preservation through transformations Wed, 13 Sep, 17:46
lucas.g...@gmail.com         Re: RDD order preservation through transformations Wed, 13 Sep, 17:50
Suzen, Mehmet           Re: RDD order preservation through transformations Thu, 14 Sep, 01:36
johan.grande....@orange.com             RE: RDD order preservation through transformations Thu, 14 Sep, 08:01
johan.grande....@orange.com             RE: RDD order preservation through transformations Thu, 14 Sep, 08:24
johan.grande....@orange.com           RE: RDD order preservation through transformations Thu, 14 Sep, 08:42
Georg Heiler             Re: RDD order preservation through transformations Thu, 14 Sep, 09:05
Suzen, Mehmet             Re: RDD order preservation through transformations Thu, 14 Sep, 10:18
johan.grande....@orange.com               RE: RDD order preservation through transformations Fri, 15 Sep, 07:44
Suzen, Mehmet                 Re: RDD order preservation through transformations Fri, 15 Sep, 12:03
johan.grande....@orange.com                   RE: RDD order preservation through transformations Fri, 15 Sep, 13:07
kant kodali Should I use Dstream or Structured Stream to transfer data from source to sink and then back from sink to source? Wed, 13 Sep, 18:20
Tathagata Das   Re: Should I use Dstream or Structured Stream to transfer data from source to sink and then back from sink to source? Thu, 14 Sep, 07:23
kant kodali     Re: Should I use Dstream or Structured Stream to transfer data from source to sink and then back from sink to source? Fri, 15 Sep, 02:07
how sequence of chained jars in spark.(driver/executor).extraClassPath matters
Richard Xin   how sequence of chained jars in spark.(driver/executor).extraClassPath matters Wed, 13 Sep, 18:55
Mikhailau, Alex Re-sharded kinesis stream starts generating warnings after kinesis shard numbers were doubled Wed, 13 Sep, 20:16
Margus Roo spark.streaming.receiver.maxRate Thu, 14 Sep, 06:57
Margus Roo   Re: spark.streaming.receiver.maxRate Fri, 15 Sep, 18:50
Margus Roo     Re: spark.streaming.receiver.maxRate Fri, 15 Sep, 18:59
Akhil Das       Re: spark.streaming.receiver.maxRate Sat, 16 Sep, 15:02
KhajaAsmath Mohammed cannot cast to double from spark row Thu, 14 Sep, 19:20
Ram Sriharsha   Re: cannot cast to double from spark row Thu, 14 Sep, 19:33
Marco Mistroni PLs assist: trying to FlatMap a DataSet / partially OT Thu, 14 Sep, 21:21
Akhil Das   Re: PLs assist: trying to FlatMap a DataSet / partially OT Sat, 16 Sep, 15:53
Marco Mistroni     Re: PLs assist: trying to FlatMap a DataSet / partially OT Sat, 16 Sep, 21:05
Daniel O' Shaughnessy Nested RDD operation Fri, 15 Sep, 09:42
Jean Georges Perrin   Re: Nested RDD operation Fri, 15 Sep, 12:31
Daniel O' Shaughnessy     Re: Nested RDD operation Tue, 19 Sep, 11:20
ayan guha       Re: Nested RDD operation Tue, 19 Sep, 13:15
Jean Georges Perrin       Re: Nested RDD operation Tue, 19 Sep, 13:46
rpulluru Size exceeds Integer.MAX_VALUE issue with RandomForest Fri, 15 Sep, 12:55
Akhil Das   Re: Size exceeds Integer.MAX_VALUE issue with RandomForest Sat, 16 Sep, 15:24
Pulluru Ranjith     Re: Size exceeds Integer.MAX_VALUE issue with RandomForest Mon, 18 Sep, 08:57
Arun Khetarpal [SPARK-SQL] Does spark-sql have Authorization built in? Fri, 15 Sep, 15:13
Akhil Das   Re: [SPARK-SQL] Does spark-sql have Authorization built in? Sat, 16 Sep, 15:14
Jörn Franke   Re: [SPARK-SQL] Does spark-sql have Authorization built in? Sat, 16 Sep, 15:16
Arun Khetarpal   Re: [SPARK-SQL] Does spark-sql have Authorization built in? Mon, 18 Sep, 06:20
HARSH TAKKAR ConcurrentModificationException using Kafka Direct Stream Sun, 17 Sep, 14:48
kant kodali   Re: ConcurrentModificationException using Kafka Direct Stream Sun, 17 Sep, 18:58
HARSH TAKKAR     Re: ConcurrentModificationException using Kafka Direct Stream Mon, 18 Sep, 05:25
kant kodali       Re: ConcurrentModificationException using Kafka Direct Stream Mon, 18 Sep, 06:01
Anastasios Zouzias   Re: ConcurrentModificationException using Kafka Direct Stream Mon, 18 Sep, 06:08
pandees waran     Re: ConcurrentModificationException using Kafka Direct Stream Mon, 18 Sep, 06:13
HARSH TAKKAR       Re: ConcurrentModificationException using Kafka Direct Stream Mon, 18 Sep, 06:56
Cody Koeninger         Re: ConcurrentModificationException using Kafka Direct Stream Mon, 18 Sep, 15:17
HARSH TAKKAR           Re: ConcurrentModificationException using Kafka Direct Stream Tue, 19 Sep, 10:08
haibo wu Spark 2.1.1 Driver OOM when use interaction for large scale Sparse Vector Mon, 18 Sep, 01:57
haibo wu spark 2.1.1 ml.LogisticRegression with large feature set cause Kryo serialization failed: Buffer overflow Mon, 18 Sep, 01:59
Patrick Builder Pattern used by Spark source code architecture Mon, 18 Sep, 16:33
lucas.g...@gmail.com Question on partitionColumn for a JDBC read using a timestamp from MySql Mon, 18 Sep, 20:21
lucas.g...@gmail.com   Re: Question on partitionColumn for a JDBC read using a timestamp from MySql Tue, 19 Sep, 15:09
Jean Georges Perrin [Timer-0:WARN] Logging$class: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources Mon, 18 Sep, 21:24
Riccardo Ferrari   Re: [Timer-0:WARN] Logging$class: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources Mon, 18 Sep, 22:27
Hugo Reinwald Spark Executor - jaas.conf with useTicketCache=true Tue, 19 Sep, 08:18
Aakash Basu Help needed in Dividing open close dates column into multiple columns in dataframe Tue, 19 Sep, 09:02
Aakash Basu   Fwd: Help needed in Dividing open close dates column into multiple columns in dataframe Wed, 20 Sep, 19:35
Jacek Laskowski Uses of avg hash probe metric in HashAggregateExec? Tue, 19 Sep, 12:13
Alexander Ovcharenko SVD computation limit Tue, 19 Sep, 13:49
Mark Bittmann   Re: SVD computation limit Tue, 19 Sep, 19:40
Vadim Semenov     Re: SVD computation limit Tue, 19 Sep, 20:00
kant kodali How to read from multiple kafka topics using structured streaming (spark 2.2.0)? Tue, 19 Sep, 19:50
Jacek Laskowski   Re: How to read from multiple kafka topics using structured streaming (spark 2.2.0)? Tue, 19 Sep, 19:52
kant kodali     Re: How to read from multiple kafka topics using structured streaming (spark 2.2.0)? Tue, 19 Sep, 19:54
Cody Koeninger       Re: How to read from multiple kafka topics using structured streaming (spark 2.2.0)? Tue, 19 Sep, 20:34
Jacek Laskowski         Re: How to read from multiple kafka topics using structured streaming (spark 2.2.0)? Wed, 20 Sep, 05:14
kant kodali Structured streaming coding question Tue, 19 Sep, 20:54
kant kodali   Re: Structured streaming coding question Wed, 20 Sep, 05:09
kant kodali     Re: Structured streaming coding question Wed, 20 Sep, 05:11
Burak Yavuz       Re: Structured streaming coding question Wed, 20 Sep, 06:48
kant kodali         Re: Structured streaming coding question Wed, 20 Sep, 06:59
Burak Yavuz           Re: Structured streaming coding question Wed, 20 Sep, 07:07
kant kodali             Re: Structured streaming coding question Wed, 20 Sep, 07:51
kant kodali               Re: Structured streaming coding question Wed, 20 Sep, 09:26
kant kodali                 Re: Structured streaming coding question Wed, 20 Sep, 09:34
Jacek Laskowski     Re: Structured streaming coding question Wed, 20 Sep, 05:11
Jacek Laskowski   Re: Structured streaming coding question Wed, 20 Sep, 05:10
Gaurav1809 Cloudera - How to switch to the newly added Spark service (Spark2) from Spark 1.6 in CDH 5.12 Wed, 20 Sep, 03:46
Sudha KS   RE: Cloudera - How to switch to the newly added Spark service (Spark2) from Spark 1.6 in CDH 5.12 Wed, 20 Sep, 03:49
Gaurav1809     RE: Cloudera - How to switch to the newly added Spark service (Spark2) from Spark 1.6 in CDH 5.12 Wed, 20 Sep, 07:26
tole...@toletum.org Spark Streaming + Kafka + Hive: delayed Wed, 20 Sep, 11:08
Simon Dirmeier Pyspark define UDF for windows Wed, 20 Sep, 11:23
Weichen Xu   Re: Pyspark define UDF for windows Wed, 20 Sep, 13:28
Alexander Czech for loops in pyspark Wed, 20 Sep, 12:12
Weichen Xu   Re: for loops in pyspark Wed, 20 Sep, 12:22
Alexander Czech     Re: for loops in pyspark Thu, 21 Sep, 09:54
Jeff Zhang       Re: for loops in pyspark Thu, 21 Sep, 10:58
Femi Anthony         Re: for loops in pyspark Thu, 21 Sep, 13:07
kant kodali Is there a SparkILoop for Java? Wed, 20 Sep, 12:38
Weichen Xu   Re: Is there a SparkILoop for Java? Wed, 20 Sep, 12:45
kant kodali     Re: Is there a SparkILoop for Java? Wed, 20 Sep, 13:06
Jean Georges Perrin       Re: Is there a SparkILoop for Java? Wed, 20 Sep, 14:17
kant kodali         Re: Is there a SparkILoop for Java? Wed, 20 Sep, 20:08
Jeff Zhang           Re: Is there a SparkILoop for Java? Thu, 21 Sep, 10:52
Imran Rajjad graphframes on cluster Wed, 20 Sep, 12:47
Felix Cheung   Re: graphframes on cluster Wed, 20 Sep, 16:00
Imran Rajjad     Re: graphframes on cluster Fri, 22 Sep, 10:23
Kedarnath Dixit Spark code to get select firelds from ES Wed, 20 Sep, 14:22
Jean Georges Perrin   Re: Spark code to get select firelds from ES Wed, 20 Sep, 17:44
ayan guha     Re: Spark code to get select firelds from ES Wed, 20 Sep, 22:11
SRK How to get time slice or the batch time for which the current micro batch is running in Spark Streaming Wed, 20 Sep, 20:47
yohann jardin Re:Fwd: Help needed in Dividing open close dates column into multiple columns in dataframe Wed, 20 Sep, 21:23
Bandish Chheda [Structured Streaming] How to replay data and overwrite using FileSink Thu, 21 Sep, 01:20
Michael Armbrust   Re: [Structured Streaming] How to replay data and overwrite using FileSink Fri, 22 Sep, 19:51
张万新 How to use approx_count_distinct to count distinct numbers in a day but output the count of each hour? Thu, 21 Sep, 04:09
Chackravarthy Esakkimuthu How to pass sparkSession from driver to executor Thu, 21 Sep, 12:03
Riccardo Ferrari   Re: How to pass sparkSession from driver to executor Thu, 21 Sep, 13:18
Weichen Xu     Re: How to pass sparkSession from driver to executor Thu, 21 Sep, 13:31
lucas.g...@gmail.com       Re: How to pass sparkSession from driver to executor Thu, 21 Sep, 15:44
ayan guha         Re: How to pass sparkSession from driver to executor Thu, 21 Sep, 15:55
kant kodali How to know what are possible operations spark raw sql can support? Thu, 21 Sep, 12:46
Message list« Previous · 1 · 2 · 3 · 4 · 5 · Next »Thread · Author · Date
Box list
Sep 2019117
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137