spark-user mailing list archives: March 2015

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 14 · 15 · 16 · 17 · 18 · 19 · 20 · Next »Thread · Author · Date
Stuart Layton What are the best options for quickly filtering a DataFrame on a single column? Wed, 25 Mar, 14:41
Michael Armbrust   Re: What are the best options for quickly filtering a DataFrame on a single column? Wed, 25 Mar, 18:29
Stuart Layton     Re: What are the best options for quickly filtering a DataFrame on a single column? Wed, 25 Mar, 18:39
Michael Armbrust       Re: What are the best options for quickly filtering a DataFrame on a single column? Wed, 25 Mar, 19:25
RodrigoB Spark Streaming - Minimizing batch interval Wed, 25 Mar, 14:53
Sean Owen   Re: Spark Streaming - Minimizing batch interval Wed, 25 Mar, 15:38
Wang, Ningjun (LNG-NPV) Total size of serialized results is bigger than spark.driver.maxResultSize Wed, 25 Mar, 14:58
Denny Lee   Re: Total size of serialized results is bigger than spark.driver.maxResultSize Wed, 25 Mar, 17:45
Write Parquet File with spark-streaming with Spark 1.3
richiesgr   Write Parquet File with spark-streaming with Spark 1.3 Wed, 25 Mar, 15:53
Richard Grossman   Write Parquet File with spark-streaming with Spark 1.3 Thu, 26 Mar, 08:13
Cheng Lian     Re: Write Parquet File with spark-streaming with Spark 1.3 Thu, 26 Mar, 11:15
roni upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Wed, 25 Mar, 15:58
Nick Pentreath   Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Wed, 25 Mar, 16:26
roni     Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Wed, 25 Mar, 16:39
Nick Pentreath       Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Wed, 25 Mar, 16:45
Dean Wampler       Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Wed, 25 Mar, 16:50
roni         Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Wed, 25 Mar, 19:09
Dean Wampler           Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Wed, 25 Mar, 19:34
roni             Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Wed, 25 Mar, 20:13
Dean Wampler               Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Wed, 25 Mar, 20:18
roni                 Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Wed, 25 Mar, 21:54
Dean Wampler                   Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Thu, 26 Mar, 02:43
Steve Loughran                   Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems Thu, 26 Mar, 10:12
Ravi Reddy Recovered state for updateStateByKey and incremental streams processing Wed, 25 Mar, 17:09
Eduardo Cusa python : Out of memory: Kill process Wed, 25 Mar, 17:33
Davies Liu   Re: python : Out of memory: Kill process Wed, 25 Mar, 18:39
Eduardo Cusa     Re: python : Out of memory: Kill process Wed, 25 Mar, 18:49
Davies Liu       Re: python : Out of memory: Kill process Wed, 25 Mar, 19:00
Eduardo Cusa         Re: python : Out of memory: Kill process Thu, 26 Mar, 13:41
Eduardo Cusa           Re: python : Out of memory: Kill process Thu, 26 Mar, 17:02
Davies Liu             Re: python : Out of memory: Kill process Thu, 26 Mar, 17:29
Eduardo Cusa               Re: python : Out of memory: Kill process Thu, 26 Mar, 17:34
Davies Liu                 Re: python : Out of memory: Kill process Thu, 26 Mar, 18:16
Eduardo Cusa                   Re: python : Out of memory: Kill process Mon, 30 Mar, 14:18
ÐΞ...@Ҝ (๏̯͡๏) Unable to Hive program from Spark Programming Guide (OutOfMemoryError) Wed, 25 Mar, 17:48
ÐΞ...@Ҝ (๏̯͡๏)   Re: Unable to Hive program from Spark Programming Guide (OutOfMemoryError) Thu, 26 Mar, 02:24
ÐΞ...@Ҝ (๏̯͡๏)     Re: Unable to Hive program from Spark Programming Guide (OutOfMemoryError) Thu, 26 Mar, 08:10
Re: OOM for HiveFromSpark example
ÐΞ...@Ҝ (๏̯͡๏)   Re: OOM for HiveFromSpark example Wed, 25 Mar, 17:54
Zhan Zhang     Re: OOM for HiveFromSpark example Wed, 25 Mar, 18:06
ÐΞ...@Ҝ (๏̯͡๏)       Re: OOM for HiveFromSpark example Thu, 26 Mar, 02:25
Zhan Zhang         Re: OOM for HiveFromSpark example Thu, 26 Mar, 02:38
ÐΞ...@Ҝ (๏̯͡๏)           Re: OOM for HiveFromSpark example Thu, 26 Mar, 06:17
Akhil Das             Re: OOM for HiveFromSpark example Thu, 26 Mar, 06:45
ÐΞ...@Ҝ (๏̯͡๏)               Re: OOM for HiveFromSpark example Thu, 26 Mar, 08:08
Akhil Das                 Re: OOM for HiveFromSpark example Thu, 26 Mar, 08:11
ÐΞ...@Ҝ (๏̯͡๏)                   Re: OOM for HiveFromSpark example Thu, 26 Mar, 08:26
Akhil Das                     Re: OOM for HiveFromSpark example Thu, 26 Mar, 08:29
ÐΞ...@Ҝ (๏̯͡๏)                       Re: OOM for HiveFromSpark example Thu, 26 Mar, 08:36
Akhil Das                         Re: OOM for HiveFromSpark example Thu, 26 Mar, 08:37
Akhil Das                           Re: OOM for HiveFromSpark example Thu, 26 Mar, 08:40
Re: OutOfMemory : Java heap space error
ÐΞ...@Ҝ (๏̯͡๏)   Re: OutOfMemory : Java heap space error Wed, 25 Mar, 17:54
Stuart Layton Can a DataFrame be saved to s3 directly using Parquet? Wed, 25 Mar, 18:59
Michael Armbrust   Re: Can a DataFrame be saved to s3 directly using Parquet? Wed, 25 Mar, 19:15
Michael Armbrust     Re: Can a DataFrame be saved to s3 directly using Parquet? Wed, 25 Mar, 19:16
Khandeshi, Ami Spark shell never leaves ACCEPTED state in YARN CDH5 Wed, 25 Mar, 19:08
Marcelo Vanzin   Re: Spark shell never leaves ACCEPTED state in YARN CDH5 Wed, 25 Mar, 19:18
Dean Chen     Re: Spark shell never leaves ACCEPTED state in YARN CDH5 Thu, 26 Mar, 01:08
Tobias Pfeiffer   Re: Spark shell never leaves ACCEPTED state in YARN CDH5 Thu, 26 Mar, 01:08
Adrian Mocanu writing DStream RDDs to the same file Wed, 25 Mar, 19:49
Akhil Das   Re: writing DStream RDDs to the same file Thu, 26 Mar, 07:01
elliott cordo trouble with jdbc df in python Wed, 25 Mar, 21:19
Michael Armbrust   Re: trouble with jdbc df in python Wed, 25 Mar, 22:12
elliott cordo     Re: trouble with jdbc df in python Wed, 25 Mar, 23:04
Michael Armbrust       Re: trouble with jdbc df in python Wed, 25 Mar, 23:47
varvind Exception Failed to add a datanode. User may turn off this feature by setting dfs.client.block.write.replace-datanode-on-failure.policy in configuration Wed, 25 Mar, 21:31
Matt Cheah Cross-compatibility of YARN shuffle service Wed, 25 Mar, 22:44
Sandy Ryza   Re: Cross-compatibility of YARN shuffle service Fri, 27 Mar, 03:55
Manoj Samel How to specify the port for AM Actor ... Wed, 25 Mar, 22:49
Shixiong Zhu   Re: How to specify the port for AM Actor ... Wed, 25 Mar, 23:06
Manoj Samel     Re: How to specify the port for AM Actor ... Wed, 25 Mar, 23:13
Shixiong Zhu       Re: How to specify the port for AM Actor ... Wed, 25 Mar, 23:44
Manoj Samel         Re: How to specify the port for AM Actor ... Fri, 27 Mar, 23:14
Shixiong Zhu           Re: How to specify the port for AM Actor ... Mon, 30 Mar, 03:18
Haopu Wang [SparkSQL] How to calculate stddev on a DataFrame? Thu, 26 Mar, 02:28
Corey Nolet   Re: [SparkSQL] How to calculate stddev on a DataFrame? Thu, 26 Mar, 04:00
Denny Lee   Re: [SparkSQL] How to calculate stddev on a DataFrame? Thu, 26 Mar, 04:01
rkgurram The dreaded bradcast error Error: Failed to get broadcast_0_piece0 of broadcast_0 Thu, 26 Mar, 02:50
Pei-Lun Lee SparkSQL overwrite parquet file does not generate _common_metadata Thu, 26 Mar, 04:48
Cheng Lian   Re: SparkSQL overwrite parquet file does not generate _common_metadata Thu, 26 Mar, 11:26
Pei-Lun Lee     Re: SparkSQL overwrite parquet file does not generate _common_metadata Fri, 27 Mar, 03:33
Cheng Lian       Re: SparkSQL overwrite parquet file does not generate _common_metadata Fri, 27 Mar, 06:32
Pei-Lun Lee         Re: SparkSQL overwrite parquet file does not generate _common_metadata Fri, 27 Mar, 06:40
Cheng Lian           Re: SparkSQL overwrite parquet file does not generate _common_metadata Fri, 27 Mar, 11:03
Pei-Lun Lee             Re: SparkSQL overwrite parquet file does not generate _common_metadata Sat, 28 Mar, 02:52
Xi Shen How to troubleshoot server.TransportChannelHandler Exception Thu, 26 Mar, 05:28
Akhil Das   Re: How to troubleshoot server.TransportChannelHandler Exception Thu, 26 Mar, 06:48
Xi Shen     Re: How to troubleshoot server.TransportChannelHandler Exception Thu, 26 Mar, 07:22
Haopu Wang Can I call aggregate UDF in DataFrame? Thu, 26 Mar, 07:37
ÐΞ...@Ҝ (๏̯͡๏) Hive Table not from from Spark SQL Thu, 26 Mar, 07:56
ÐΞ...@Ҝ (๏̯͡๏)   Re: Hive Table not from from Spark SQL Thu, 26 Mar, 10:57
ÐΞ...@Ҝ (๏̯͡๏)     Re: Hive Table not from from Spark SQL Thu, 26 Mar, 11:02
Michael Armbrust       Re: Hive Table not from from Spark SQL Thu, 26 Mar, 15:05
ÐΞ...@Ҝ (๏̯͡๏)         Re: Hive Table not from from Spark SQL Thu, 26 Mar, 15:28
ÐΞ...@Ҝ (๏̯͡๏)           Re: Hive Table not from from Spark SQL Thu, 26 Mar, 15:31
ÐΞ...@Ҝ (๏̯͡๏)             Re: Hive Table not from from Spark SQL Fri, 27 Mar, 13:04
Denny Lee               Re: Hive Table not from from Spark SQL Fri, 27 Mar, 16:05
Cheng, Hao                 RE: Hive Table not from from Spark SQL Fri, 27 Mar, 17:23
李铖 Missing an output location for shuffle. : ( Thu, 26 Mar, 10:20
Michael Armbrust   Re: Missing an output location for shuffle. : ( Thu, 26 Mar, 15:01
李铖     Re: Missing an output location for shuffle. : ( Fri, 27 Mar, 01:50
李铖     Re: Missing an output location for shuffle. : ( Fri, 27 Mar, 01:52
kundan kumar Handling Big data for interactive BI tools Thu, 26 Mar, 10:26
Akhil Das   Re: Handling Big data for interactive BI tools Thu, 26 Mar, 11:17
Jörn Franke   Re: Handling Big data for interactive BI tools Thu, 26 Mar, 12:17
kundan kumar     Re: Handling Big data for interactive BI tools Thu, 26 Mar, 12:29
kundan kumar       Re: Handling Big data for interactive BI tools Thu, 26 Mar, 12:34
Jörn Franke         Re: Handling Big data for interactive BI tools Thu, 26 Mar, 15:54
Denny Lee           Re: Handling Big data for interactive BI tools Thu, 26 Mar, 17:29
Jon Chase Column not found in schema when querying partitioned table Thu, 26 Mar, 10:29
Jon Chase   Re: Column not found in schema when querying partitioned table Thu, 26 Mar, 10:46
ÐΞ...@Ҝ (๏̯͡๏)     Re: Column not found in schema when querying partitioned table Fri, 27 Mar, 09:54
Manish Gupta 8 Port configuration for BlockManagerId Thu, 26 Mar, 10:38
Manish Gupta 8   RE: Port configuration for BlockManagerId Mon, 30 Mar, 05:33
Masf Windowing and Analytics Functions in Spark SQL Thu, 26 Mar, 11:09
Arush Kharbanda   Re: Windowing and Analytics Functions in Spark SQL Thu, 26 Mar, 11:27
Cheng Lian     Re: Windowing and Analytics Functions in Spark SQL Thu, 26 Mar, 11:31
Masf       Re: Windowing and Analytics Functions in Spark SQL Thu, 26 Mar, 11:51
Arush Kharbanda         Re: Windowing and Analytics Functions in Spark SQL Thu, 26 Mar, 12:08
Xi Shen Why k-means cluster hang for a long time? Thu, 26 Mar, 12:09
Xi Shen   Re: Why k-means cluster hang for a long time? Thu, 26 Mar, 22:48
Xi Shen     Re: Why k-means cluster hang for a long time? Thu, 26 Mar, 23:02
Xi Shen       Re: Why k-means cluster hang for a long time? Thu, 26 Mar, 23:15
Xi Shen         Re: Why k-means cluster hang for a long time? Fri, 27 Mar, 00:01
Xi Shen           Re: Why k-means cluster hang for a long time? Fri, 27 Mar, 00:45
Xiangrui Meng             Re: Why k-means cluster hang for a long time? Mon, 30 Mar, 17:00
Xi Shen               Re: Why k-means cluster hang for a long time? Mon, 30 Mar, 22:55
Xiangrui Meng                 Re: Why k-means cluster hang for a long time? Mon, 30 Mar, 23:34
sergunok Why executor encourage OutOfMemoryException: Java heap space Thu, 26 Mar, 12:13
Stevo Slavić Spark-core and guava Thu, 26 Mar, 12:24
Sean Owen   Re: Spark-core and guava Thu, 26 Mar, 12:29
Stevo Slavić     Re: Spark-core and guava Thu, 26 Mar, 16:16
sergunok Which RDD operations preserve ordering? Thu, 26 Mar, 12:58
Ted Yu   Re: Which RDD operations preserve ordering? Thu, 26 Mar, 13:59
MEETHU MATHEW Spark-1.3.0 UI shows 0 cores in completed applications tab Thu, 26 Mar, 12:58
Sean Owen   Re: Spark-1.3.0 UI shows 0 cores in completed applications tab Thu, 26 Mar, 13:25
Bob DuCharme Populating a HashMap from a GraphX connectedComponents graph Thu, 26 Mar, 13:24
Stuart Layton RDD equivalent of HBase Scan Thu, 26 Mar, 13:46
Ted Yu   Re: RDD equivalent of HBase Scan Thu, 26 Mar, 13:54
Stuart Layton     Re: RDD equivalent of HBase Scan Thu, 26 Mar, 13:57
Sean Owen       Re: RDD equivalent of HBase Scan Thu, 26 Mar, 14:41
Adrian Mocanu Spark log shows only this line repeated: RecurringTimer - JobGenerator] DEBUG o.a.s.streaming.util.RecurringTimer - Callback for JobGenerator called at time X Thu, 26 Mar, 13:55
Ted Yu   Re: Spark log shows only this line repeated: RecurringTimer - JobGenerator] DEBUG o.a.s.streaming.util.RecurringTimer - Callback for JobGenerator called at time X Thu, 26 Mar, 14:50
Recreating the Mesos/Spark paper's experiments
Hans van den Bogert   Recreating the Mesos/Spark paper's experiments Thu, 26 Mar, 14:09
hbogert   Recreating the Mesos/Spark paper's experiments Thu, 26 Mar, 22:56
Kevin Conaway RDD Exception Handling Thu, 26 Mar, 14:15
Akhil Das   Re: RDD Exception Handling Fri, 27 Mar, 07:20
[Spark Streaming] Disk not being cleaned up during runtime after RDD being processed
NathanMarin   [Spark Streaming] Disk not being cleaned up during runtime after RDD being processed Thu, 26 Mar, 14:40
Nathan Marin   [Spark Streaming] Disk not being cleaned up during runtime after RDD being processed Thu, 26 Mar, 15:17
Nathan Marin   [Spark Streaming] Disk not being cleaned up during runtime after RDD being processed Sat, 28 Mar, 14:48
Akhil Das     Re: [Spark Streaming] Disk not being cleaned up during runtime after RDD being processed Sun, 29 Mar, 14:55
Ted Yu       Re: [Spark Streaming] Disk not being cleaned up during runtime after RDD being processed Sun, 29 Mar, 15:50
Nathan Marin         Re: [Spark Streaming] Disk not being cleaned up during runtime after RDD being processed Mon, 30 Mar, 10:14
Ravi Mody Implicit matrix factorization returning different results between spark 1.2.0 and 1.3.0 Thu, 26 Mar, 14:56
Xiangrui Meng   Re: Implicit matrix factorization returning different results between spark 1.2.0 and 1.3.0 Fri, 27 Mar, 18:27
Xiangrui Meng     Re: Implicit matrix factorization returning different results between spark 1.2.0 and 1.3.0 Mon, 30 Mar, 16:51
Sean Owen       Re: Implicit matrix factorization returning different results between spark 1.2.0 and 1.3.0 Tue, 31 Mar, 12:17
Xiangrui Meng         Re: Implicit matrix factorization returning different results between spark 1.2.0 and 1.3.0 Tue, 31 Mar, 21:41
Sean Owen           Re: Implicit matrix factorization returning different results between spark 1.2.0 and 1.3.0 Tue, 31 Mar, 21:50
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 14 · 15 · 16 · 17 · 18 · 19 · 20 · Next »Thread · Author · Date
Box list
Oct 2020151
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137