spark-user mailing list archives: April 2018

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · Next »Thread · Author · Date
Vitaliy Pisarev Does joining table in Spark multiplies selected columns of smaller table? Sun, 08 Apr, 17:52
Jörn Franke   Re: Does joining table in Spark multiplies selected columns of smaller table? Sun, 08 Apr, 17:58
Vitaliy Pisarev     Re: Does joining table in Spark multiplies selected columns of smaller table? Sun, 08 Apr, 18:21
hantuzun [Mesos] How to Disable Blacklisting on Mesos? Sun, 08 Apr, 23:06
Susan X. Huynh   Re: [Mesos] How to Disable Blacklisting on Mesos? Tue, 10 Apr, 02:54
Junfeng Chen spark application running in yarn client mode is slower than in local mode. Mon, 09 Apr, 05:07
Jörn Franke   Re: spark application running in yarn client mode is slower than in local mode. Mon, 09 Apr, 06:12
Gopala Krishna Manchukonda     Re: spark application running in yarn client mode is slower than in local mode. Mon, 09 Apr, 06:20
Junfeng Chen       Re: spark application running in yarn client mode is slower than in local mode. Mon, 09 Apr, 06:47
Junfeng Chen     Re: spark application running in yarn client mode is slower than in local mode. Mon, 09 Apr, 06:44
Junfeng Chen     Re: spark application running in yarn client mode is slower than in local mode. Mon, 09 Apr, 07:20
Junfeng Chen     Re: spark application running in yarn client mode is slower than in local mode. Tue, 10 Apr, 02:05
Saisai Shao       Re: spark application running in yarn client mode is slower than in local mode. Tue, 10 Apr, 02:19
Junfeng Chen         Re: spark application running in yarn client mode is slower than in local mode. Tue, 10 Apr, 03:31
Junfeng Chen           Re: spark application running in yarn client mode is slower than in local mode. Tue, 10 Apr, 06:51
Shiyuan A bug triggered by a particular sequence of "select", "groupby" and "join" in Spark 2.3.0 Mon, 09 Apr, 17:50
Gourav Sengupta   Re: A bug triggered by a particular sequence of "select", "groupby" and "join" in Spark 2.3.0 Mon, 09 Apr, 22:21
Shiyuan     Re: A bug triggered by a particular sequence of "select", "groupby" and "join" in Spark 2.3.0 Tue, 10 Apr, 20:37
Shiyuan       Re: A bug triggered by a particular sequence of "select", "groupby" and "join" in Spark 2.3.0 Tue, 10 Apr, 21:42
Gourav Sengupta         Re: A bug triggered by a particular sequence of "select", "groupby" and "join" in Spark 2.3.0 Wed, 11 Apr, 04:03
Shiyuan           Re: A bug triggered by a particular sequence of "select", "groupby" and "join" in Spark 2.3.0 Wed, 11 Apr, 17:47
Alessandro Solimando             Re: A bug triggered by a particular sequence of "select", "groupby" and "join" in Spark 2.3.0 Wed, 11 Apr, 17:55
Shiyuan               Re: A bug triggered by a particular sequence of "select", "groupby" and "join" in Spark 2.3.0 Wed, 11 Apr, 20:53
Niu Zhaojie pyspark.daemon exhaust a lot of memory Tue, 10 Apr, 04:35
Aakash Basu Is DLib available for Spark? Tue, 10 Apr, 07:52
Dmitry Spark on Kubernetes (minikube) 2.3 fails with class not found exception Tue, 10 Apr, 08:34
Yinan Li   Re: Spark on Kubernetes (minikube) 2.3 fails with class not found exception Tue, 10 Apr, 17:17
Dmitry     Re: Spark on Kubernetes (minikube) 2.3 fails with class not found exception Tue, 10 Apr, 18:02
Marcelo Vanzin       Re: Spark on Kubernetes (minikube) 2.3 fails with class not found exception Tue, 10 Apr, 18:06
Shahab Yunus StringIndexer with high cardinality huge data Tue, 10 Apr, 13:01
Filipp Zhinkin   Re: StringIndexer with high cardinality huge data Tue, 10 Apr, 14:00
Nick Pentreath     Re: StringIndexer with high cardinality huge data Tue, 10 Apr, 14:05
Shahab Yunus       Re: StringIndexer with high cardinality huge data Tue, 10 Apr, 14:59
Guillermo Ortiz Testing spark streaming action Tue, 10 Apr, 15:35
Jörn Franke   Re: Testing spark streaming action Tue, 10 Apr, 16:32
kant kodali Does spark.driver.memory, spark.executor.memory still in use when we have spark.memory.fraction? Tue, 10 Apr, 16:59
kant kodali   Re: Does spark.driver.memory, spark.executor.memory still in use when we have spark.memory.fraction? Wed, 11 Apr, 09:33
José Raúl Pérez Rodríguez cache OS memory and spark usage of it Tue, 10 Apr, 17:27
yncxcw   Re: cache OS memory and spark usage of it Tue, 10 Apr, 23:36
Jose Raul Perez Rodriguez     Re: cache OS memory and spark usage of it Wed, 11 Apr, 06:57
yncxcw       Re: cache OS memory and spark usage of it Wed, 11 Apr, 16:10
Deepansh Goyal package reload in dapply SparkR Tue, 10 Apr, 17:42
王晨伊 [Structured Streaming] why events size is 0 when use mapGroupsWithState Tue, 10 Apr, 18:37
Re: Apache Kafka / Spark Integration - Exception - The server disconnected before a response was received.
M Singh   Re: Apache Kafka / Spark Integration - Exception - The server disconnected before a response was received. Tue, 10 Apr, 22:31
Colin Williams Specifying a custom Partitioner on RDD creation in Spark 2 Wed, 11 Apr, 00:47
@Nan...@ Not able to access Pyspark into Jupyter notebook Wed, 11 Apr, 03:36
Dylan Guedes   Re: Not able to access Pyspark into Jupyter notebook Wed, 11 Apr, 18:18
How to use disk instead of just InMemoryRelation when use JDBC datasource in SPARKSQL?
Louis Hust   How to use disk instead of just InMemoryRelation when use JDBC datasource in SPARKSQL? Wed, 11 Apr, 03:41
Louis Hust   How to use disk instead of just InMemoryRelation when use JDBC datasource in SPARKSQL? Thu, 12 Apr, 04:12
Takeshi Yamamuro     Re: How to use disk instead of just InMemoryRelation when use JDBC datasource in SPARKSQL? Thu, 12 Apr, 06:29
SRK Does structured streaming support Spark Kafka Direct? Wed, 11 Apr, 06:03
Tathagata Das   Re: Does structured streaming support Spark Kafka Direct? Thu, 12 Apr, 07:03
SRK How do I implement forEachWriter in structured streaming so that the connection is created once per partition? Wed, 11 Apr, 06:15
@Nan...@ Issue with map function in Spark 2.2.0 Wed, 11 Apr, 07:11
ayan guha   Re: Issue with map function in Spark 2.2.0 Wed, 11 Apr, 07:30
how to use the sql join in java please
1427357...@qq.com   how to use the sql join in java please Wed, 11 Apr, 07:14
Yu, Yucai   Re: how to use the sql join in java please Wed, 11 Apr, 08:01
1427357...@qq.com   Re: Re: how to use the sql join in java please Wed, 11 Apr, 08:11
杜斌 How to submit some code segment to existing SparkContext Wed, 11 Apr, 07:46
Saisai Shao   Re: How to submit some code segment to existing SparkContext Wed, 11 Apr, 08:02
1427357...@qq.com Hot to filter the datatime in dataset with java code please? Wed, 11 Apr, 08:19
宋源栋 Spark is only using one worker machine when more are available Wed, 11 Apr, 09:10
Jhon Anderson Cardenas Diaz   Re: Spark is only using one worker machine when more are available Wed, 11 Apr, 14:42
宋源栋     回复:Spark is only using one worker machine when more are available Thu, 12 Apr, 02:39
Jhon Anderson Cardenas Diaz       Re: Spark is only using one worker machine when more are available Thu, 12 Apr, 14:19
Marius         Spark Kubernetes Volumes Thu, 12 Apr, 14:50
Yinan Li           Re: Spark Kubernetes Volumes Thu, 12 Apr, 17:55
Anirudh Ramanathan             Re: Spark Kubernetes Volumes Thu, 12 Apr, 18:27
Gourav Sengupta         Re: Spark is only using one worker machine when more are available Thu, 12 Apr, 17:48
feng wang Structured Streaming output a lot pieces of files with Append Mode Wed, 11 Apr, 10:33
Broadcasting huge array or persisting on HDFS to read on executors - both not working
surender kumar   Broadcasting huge array or persisting on HDFS to read on executors - both not working Wed, 11 Apr, 20:34
Matteo Cossu     Re: Broadcasting huge array or persisting on HDFS to read on executors - both not working Wed, 11 Apr, 20:40
surender kumar       Re: Broadcasting huge array or persisting on HDFS to read on executors - both not working Wed, 11 Apr, 21:01
Matteo Cossu         Re: Broadcasting huge array or persisting on HDFS to read on executors - both not working Thu, 12 Apr, 07:43
surender kumar           Re: Broadcasting huge array or persisting on HDFS to read on executors - both not working Thu, 12 Apr, 10:25
Gourav Sengupta             Re: Broadcasting huge array or persisting on HDFS to read on executors - both not working Thu, 12 Apr, 12:19
surender kumar               Re: Broadcasting huge array or persisting on HDFS to read on executors - both not working Thu, 12 Apr, 15:50
Junfeng Chen Nullpointerexception error when in repartition Thu, 12 Apr, 01:58
Tathagata Das   Re: Nullpointerexception error when in repartition Thu, 12 Apr, 07:08
Junfeng Chen     Re: Nullpointerexception error when in repartition Thu, 12 Apr, 10:02
Tathagata Das       Re: Nullpointerexception error when in repartition Thu, 12 Apr, 23:09
Junfeng Chen         Re: Nullpointerexception error when in repartition Fri, 13 Apr, 04:25
Problem running Kubernetes example v2.2.0-kubernetes-0.5.0
Rico Bergmann   Problem running Kubernetes example v2.2.0-kubernetes-0.5.0 Thu, 12 Apr, 06:02
Felix Cheung   Re: Problem running Kubernetes example v2.2.0-kubernetes-0.5.0 Sun, 22 Apr, 20:00
Fwd: pyspark:APID iS coming as null
nirav nishith   Fwd: pyspark:APID iS coming as null Thu, 12 Apr, 06:44
Gerard Maas [Structured Streaming] File source, Parquet format: use of the mergeSchema option. Thu, 12 Apr, 08:08
igor.berman Driver aborts on Mesos when unable to connect to one of external shuffle services Thu, 12 Apr, 08:48
Szuromi Tamás   Re: Driver aborts on Mesos when unable to connect to one of external shuffle services Thu, 12 Apr, 10:27
igor.berman     Re: Driver aborts on Mesos when unable to connect to one of external shuffle services Mon, 16 Apr, 19:41
Holden Karau Live Stream Code Reviews :) Thu, 12 Apr, 19:23
Matteo Olivi   Re: Live Stream Code Reviews :) Thu, 12 Apr, 19:47
Marco Mistroni     Re: Live Stream Code Reviews :) Thu, 12 Apr, 20:01
Holden Karau       Re: Live Stream Code Reviews :) Thu, 12 Apr, 20:03
Gourav Sengupta   Re: Live Stream Code Reviews :) Thu, 12 Apr, 21:08
Holden Karau     Re: Live Stream Code Reviews :) Fri, 13 Apr, 17:42
jb44 Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 01:32
Haoyuan Li   Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 03:41
jb44     Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 11:28
Geoff Von Allmen       Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 15:09
Jason Boorn         Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 15:18
Geoff Von Allmen           Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 15:33
Jason Boorn             Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 15:38
yohann jardin               Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 16:47
Jason Boorn                 Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 16:52
Marcelo Vanzin   Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 17:21
Jason Boorn     Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 17:47
Gene Pang       Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 18:26
Jason Boorn         Re: Spark LOCAL mode and external jar (extraClassPath) Fri, 13 Apr, 18:32
Gene Pang           Re: Spark LOCAL mode and external jar (extraClassPath) Sat, 14 Apr, 16:08
Jason Boorn             Re: Spark LOCAL mode and external jar (extraClassPath) Sat, 14 Apr, 17:02
Gourav Sengupta               Re: Spark LOCAL mode and external jar (extraClassPath) Sun, 15 Apr, 02:27
kant kodali Does partition by and order by works only in stateful case? Fri, 13 Apr, 02:34
Tathagata Das   Re: Does partition by and order by works only in stateful case? Fri, 13 Apr, 02:53
kant kodali     Re: Does partition by and order by works only in stateful case? Sat, 14 Apr, 23:10
Gourav Sengupta     Re: Does partition by and order by works only in stateful case? Sun, 15 Apr, 02:18
Krishna Kalyan Structured Streaming on Kubernetes Fri, 13 Apr, 07:27
Tathagata Das   Re: Structured Streaming on Kubernetes Fri, 13 Apr, 08:27
Matt Cheah     Re: Structured Streaming on Kubernetes Fri, 13 Apr, 18:55
Anirudh Ramanathan       Re: Structured Streaming on Kubernetes Fri, 13 Apr, 19:08
Krishna Kalyan         Re: Structured Streaming on Kubernetes Mon, 16 Apr, 08:51
Junfeng Chen Transforming json string in structured streaming problem Fri, 13 Apr, 07:52
Sbf xyz Passing Hive Context to FPGrowth. Fri, 13 Apr, 07:56
Srikanth Task failure to read input files Fri, 13 Apr, 13:06
SNEHASISH DUTTA Shuffling Data After Union and Write Fri, 13 Apr, 16:26
Rahul Nandi   Re: Shuffling Data After Union and Write Fri, 13 Apr, 17:29
lsn24 Spark parse fixed length file [Java] Fri, 13 Apr, 17:02
Georg Heiler   Re: Spark parse fixed length file [Java] Fri, 13 Apr, 18:57
lsn24     Re: Spark parse fixed length file [Java] Thu, 19 Apr, 16:57
Lalwani, Jayesh   Re: Spark parse fixed length file [Java] Mon, 16 Apr, 02:50
lsn24     Re: Spark parse fixed length file [Java] Thu, 19 Apr, 17:00
Message list« Previous · 1 · 2 · 3 · 4 · Next »Thread · Author · Date
Box list
Dec 20211
Nov 2021153
Oct 202194
Sep 2021126
Aug 2021171
Jul 2021158
Jun 2021179
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137