spark-user mailing list archives: August 2015

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 14 · 15 · 16 · 17 · Next »Thread · Author · Date
Oren Shpigel Spark with GCS Connector - Rate limit error Mon, 10 Aug, 11:09
Akhil Das   Re: Spark with GCS Connector - Rate limit error Tue, 11 Aug, 08:44
Zsombor Egyed How to connect to spark remotely from java Mon, 10 Aug, 11:44
Simon Elliston Ball   Re: How to connect to spark remotely from java Mon, 10 Aug, 12:10
Zsombor Egyed     Re: How to connect to spark remotely from java Mon, 10 Aug, 12:26
Yasemin Kaya EC2 cluster doesn't work saveAsTextFile Mon, 10 Aug, 12:08
Dean Wampler   Re: EC2 cluster doesn't work saveAsTextFile Mon, 10 Aug, 12:30
Yasemin Kaya     Re: EC2 cluster doesn't work saveAsTextFile Mon, 10 Aug, 12:58
Dean Wampler       Re: EC2 cluster doesn't work saveAsTextFile Mon, 10 Aug, 13:01
mark How to programmatically create, submit and report on Spark jobs? Mon, 10 Aug, 12:12
Ted Yu   Re: How to programmatically create, submit and report on Spark jobs? Mon, 10 Aug, 15:33
Ted Yu     Re: How to programmatically create, submit and report on Spark jobs? Tue, 11 Aug, 04:15
satish chandra j Spark Cassandra Connector issue Mon, 10 Aug, 12:44
Dean Wampler   Re: Spark Cassandra Connector issue Mon, 10 Aug, 12:49
satish chandra j     Re: Spark Cassandra Connector issue Mon, 10 Aug, 13:23
Dean Wampler       Re: Spark Cassandra Connector issue Mon, 10 Aug, 13:46
satish chandra j         Re: Spark Cassandra Connector issue Tue, 11 Aug, 03:53
satish chandra j           Re: Spark Cassandra Connector issue Tue, 11 Aug, 15:19
Mohit Durgapal spark-kafka directAPI vs receivers based API Mon, 10 Aug, 12:51
Cody Koeninger   Re: spark-kafka directAPI vs receivers based API Mon, 10 Aug, 13:14
spark vs flink low memory available
Pa Rö   spark vs flink low memory available Mon, 10 Aug, 13:59
Pa Rö   spark vs flink low memory available Mon, 10 Aug, 14:02
jun     Re:spark vs flink low memory available Tue, 11 Aug, 07:49
Ted Yu       Re: spark vs flink low memory available Tue, 11 Aug, 08:03
Pa Rö         Re: spark vs flink low memory available Tue, 11 Aug, 08:24
parö   spark vs flink low memory available Tue, 11 Aug, 07:04
Mario Pastorelli Spark Streaming dealing with broken files without dying Mon, 10 Aug, 14:14
Akhil Das   Re: Spark Streaming dealing with broken files without dying Tue, 11 Aug, 08:55
Hao Ren ClosureCleaner does not work for java code Mon, 10 Aug, 15:32
Sean Owen   Re: ClosureCleaner does not work for java code Mon, 10 Aug, 16:28
Dmitry Goldenberg How to fix OutOfMemoryError: GC overhead limit exceeded when using Spark Streaming checkpointing Mon, 10 Aug, 15:57
Cody Koeninger   Re: How to fix OutOfMemoryError: GC overhead limit exceeded when using Spark Streaming checkpointing Mon, 10 Aug, 16:10
Dmitry Goldenberg     Re: How to fix OutOfMemoryError: GC overhead limit exceeded when using Spark Streaming checkpointing Mon, 10 Aug, 16:34
Ted Yu       Re: How to fix OutOfMemoryError: GC overhead limit exceeded when using Spark Streaming checkpointing Mon, 10 Aug, 16:42
Dmitry Goldenberg         Re: How to fix OutOfMemoryError: GC overhead limit exceeded when using Spark Streaming checkpointing Mon, 10 Aug, 16:49
Cody Koeninger           Re: How to fix OutOfMemoryError: GC overhead limit exceeded when using Spark Streaming checkpointing Mon, 10 Aug, 17:07
Ted Yu             Re: How to fix OutOfMemoryError: GC overhead limit exceeded when using Spark Streaming checkpointing Mon, 10 Aug, 17:54
Dmitry Goldenberg             Re: How to fix OutOfMemoryError: GC overhead limit exceeded when using Spark Streaming checkpointing Mon, 10 Aug, 20:24
Cody Koeninger               Re: How to fix OutOfMemoryError: GC overhead limit exceeded when using Spark Streaming checkpointing Mon, 10 Aug, 20:27
Dmitry Goldenberg                 Re: How to fix OutOfMemoryError: GC overhead limit exceeded when using Spark Streaming checkpointing Mon, 10 Aug, 20:33
Cody Koeninger                   Re: How to fix OutOfMemoryError: GC overhead limit exceeded when using Spark Streaming checkpointing Mon, 10 Aug, 20:40
David Montague Problem with take vs. takeSample in PySpark Mon, 10 Aug, 16:49
Davies Liu   Re: Problem with take vs. takeSample in PySpark Mon, 10 Aug, 17:59
Mohit Anchlia Streaming of WordCount example Mon, 10 Aug, 17:29
Tathagata Das   Re: Streaming of WordCount example Mon, 10 Aug, 18:34
Mohit Anchlia     Re: Streaming of WordCount example Mon, 10 Aug, 18:43
Tathagata Das       Re: Streaming of WordCount example Mon, 10 Aug, 18:56
Mohit Anchlia         Re: Streaming of WordCount example Mon, 10 Aug, 19:43
Tathagata Das           Re: Streaming of WordCount example Mon, 10 Aug, 19:56
Mohit Anchlia             Re: Streaming of WordCount example Mon, 10 Aug, 23:15
Mohit Anchlia               Re: Streaming of WordCount example Mon, 10 Aug, 23:21
Tathagata Das                 Re: Streaming of WordCount example Mon, 10 Aug, 23:30
allonsy Kafka direct approach: blockInterval and topic partitions Mon, 10 Aug, 17:52
Cody Koeninger   Re: Kafka direct approach: blockInterval and topic partitions Mon, 10 Aug, 17:58
Luca     Re: Kafka direct approach: blockInterval and topic partitions Mon, 10 Aug, 18:12
unk1102 How to use custom Hadoop InputFormat in DataFrame? Mon, 10 Aug, 18:22
Michael Armbrust   Re: How to use custom Hadoop InputFormat in DataFrame? Mon, 10 Aug, 18:34
Umesh Kacha     Re: How to use custom Hadoop InputFormat in DataFrame? Mon, 10 Aug, 18:39
Re: Graceful shutdown for Spark Streaming
Michal Čizmazia   Re: Graceful shutdown for Spark Streaming Mon, 10 Aug, 19:12
Tathagata Das     Re: Graceful shutdown for Spark Streaming Mon, 10 Aug, 19:14
Fw: Your Application has been Received
Shing Hing Man   Fw: Your Application has been Received Mon, 10 Aug, 19:20
Ashish Soni Java Streaming Context - File Stream use Mon, 10 Aug, 19:40
Akhil Das   Re: Java Streaming Context - File Stream use Tue, 11 Aug, 09:08
Jerry Is there any external dependencies for lag() and lead() when using data frames? Mon, 10 Aug, 20:26
Michael Armbrust   Re: Is there any external dependencies for lag() and lead() when using data frames? Mon, 10 Aug, 21:03
Martin Senne     When will window .... Mon, 10 Aug, 21:15
Jerry     Re: Is there any external dependencies for lag() and lead() when using data frames? Mon, 10 Aug, 21:38
Jerry       Re: Is there any external dependencies for lag() and lead() when using data frames? Tue, 11 Aug, 02:55
Benjamin Ross         RE: Is there any external dependencies for lag() and lead() when using data frames? Tue, 11 Aug, 14:16
Benjamin Ross           RE: Is there any external dependencies for lag() and lead() when using data frames? Tue, 11 Aug, 14:21
Mike Trienis Optimal way to implement a small lookup table for identifiers in an RDD Mon, 10 Aug, 21:13
Shushant Arora avoid duplicate due to executor failure in spark stream Mon, 10 Aug, 21:32
Cody Koeninger   Re: avoid duplicate due to executor failure in spark stream Mon, 10 Aug, 21:45
Shushant Arora     Re: avoid duplicate due to executor failure in spark stream Tue, 11 Aug, 16:28
Cody Koeninger       Re: avoid duplicate due to executor failure in spark stream Wed, 12 Aug, 14:46
YaoPau collect() works, take() returns "ImportError: No module named iter" Mon, 10 Aug, 21:53
Davies Liu   Re: collect() works, take() returns "ImportError: No module named iter" Mon, 10 Aug, 22:24
Ruslan Dautkhanov   Re: collect() works, take() returns "ImportError: No module named iter" Mon, 10 Aug, 22:25
Jon Gregg     Re: collect() works, take() returns "ImportError: No module named iter" Tue, 11 Aug, 01:03
YaoPau   Re: collect() works, take() returns "ImportError: No module named iter" Wed, 12 Aug, 22:19
Re: Do I really need to build Spark for Hive/Thrift Server support?
roni   Re: Do I really need to build Spark for Hive/Thrift Server support? Mon, 10 Aug, 22:33
pkphlam Random Forest and StringIndexer in pyspark ML Pipeline Mon, 10 Aug, 22:56
Yanbo Liang   Re: Random Forest and StringIndexer in pyspark ML Pipeline Fri, 21 Aug, 10:35
Re: can't start master node on a standalone environment
pradyumnad   Re: can't start master node on a standalone environment Mon, 10 Aug, 23:13
Re: Json parsing library for Spark Streaming?
pradyumnad   Re: Json parsing library for Spark Streaming? Mon, 10 Aug, 23:38
Hyukjin Kwon Inquery about contributing codes Tue, 11 Aug, 03:02
Akhil Das   Re: Inquery about contributing codes Tue, 11 Aug, 09:13
李铖 Differents in loading data using spark datasource api and using jdbc Tue, 11 Aug, 03:23
satish chandra j   Re: Differents in loading data using spark datasource api and using jdbc Tue, 11 Aug, 04:01
sim Writing a DataFrame as compressed JSON Tue, 11 Aug, 04:12
Deepesh Maheshwari Error while output JavaDStream to disk and mongodb Tue, 11 Aug, 06:14
Jerrick Hoang Refresh table Tue, 11 Aug, 06:14
Cheng, Hao   RE: Refresh table Tue, 11 Aug, 06:38
Re: Wish for 1.4: upper bound on # tasks in Mesos
Haripriya Ayyalasomayajula   Re: Wish for 1.4: upper bound on # tasks in Mesos Tue, 11 Aug, 06:26
Rick Moritz     Re: Wish for 1.4: upper bound on # tasks in Mesos Tue, 11 Aug, 08:11
Re: Controlling number of executors on Mesos vs YARN
Haripriya Ayyalasomayajula   Re: Controlling number of executors on Mesos vs YARN Tue, 11 Aug, 06:38
Jerry Lam     Re: Controlling number of executors on Mesos vs YARN Tue, 11 Aug, 12:42
Haripriya Ayyalasomayajula       Re: Controlling number of executors on Mesos vs YARN Tue, 11 Aug, 13:21
Tim Chen         Re: Controlling number of executors on Mesos vs YARN Wed, 12 Aug, 08:18
Jerry Lam           Re: Controlling number of executors on Mesos vs YARN Wed, 12 Aug, 16:12
Ajay Singal           Re: Controlling number of executors on Mesos vs YARN Wed, 12 Aug, 18:48
Tim Chen             Re: Controlling number of executors on Mesos vs YARN Wed, 12 Aug, 21:51
Ajay Singal               Re: Controlling number of executors on Mesos vs YARN Thu, 13 Aug, 14:10
Fwd: How to minimize shuffling on Spark dataframe Join?
Abdullah Anwar   Fwd: How to minimize shuffling on Spark dataframe Join? Tue, 11 Aug, 08:44
Hemant Bhanawat     Re: How to minimize shuffling on Spark dataframe Join? Wed, 12 Aug, 05:02
Abdullah Anwar       Re: How to minimize shuffling on Spark dataframe Join? Wed, 12 Aug, 08:16
Romi Kuntsman         Re: How to minimize shuffling on Spark dataframe Join? Wed, 19 Aug, 20:05
Python3 Spark execution problems
Javier Domingo Cansino   Python3 Spark execution problems Tue, 11 Aug, 09:02
Javier Domingo Cansino   Python3 Spark execution problems Tue, 11 Aug, 09:33
AW: Spark GraphX memory requirements + java.lang.OutOfMemoryError: GC overhead limit exceeded
rene.pfitz...@nzz.ch   AW: Spark GraphX memory requirements + java.lang.OutOfMemoryError: GC overhead limit exceeded Tue, 11 Aug, 09:39
satish chandra j dse spark-submit multiple jars issue Tue, 11 Aug, 10:29
Javier Domingo Cansino   Re: dse spark-submit multiple jars issue Tue, 11 Aug, 10:38
satish chandra j     Re: dse spark-submit multiple jars issue Tue, 11 Aug, 12:44
Javier Domingo Cansino       Re: dse spark-submit multiple jars issue Tue, 11 Aug, 12:45
satish chandra j         Re: dse spark-submit multiple jars issue Tue, 11 Aug, 13:42
Javier Domingo Cansino           Re: dse spark-submit multiple jars issue Thu, 13 Aug, 10:22
Andrew Or             Re: dse spark-submit multiple jars issue Tue, 18 Aug, 21:22
Jyun-Fan Tsai How to specify column type when saving DataFrame as parquet file? Tue, 11 Aug, 10:58
Raghavendra Pandey   Re: How to specify column type when saving DataFrame as parquet file? Fri, 14 Aug, 14:29
Francis Lau     Re: How to specify column type when saving DataFrame as parquet file? Fri, 14 Aug, 16:03
JoneZhang Do you have any other method to get cpu elapsed time of an spark application Tue, 11 Aug, 12:34
Fabian Böhnlein mllib on (key, Iterable[Vector]) Tue, 11 Aug, 12:43
Feynman Liang   Re: mllib on (key, Iterable[Vector]) Tue, 11 Aug, 21:07
Maciej Szymkiewicz PySpark order-only window function issue Tue, 11 Aug, 13:41
Davies Liu   Re: PySpark order-only window function issue Thu, 13 Aug, 03:42
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 14 · 15 · 16 · 17 · Next »Thread · Author · Date
Box list
Dec 202113
Nov 2021153
Oct 202194
Sep 2021126
Aug 2021171
Jul 2021158
Jun 2021179
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137