spark-user mailing list archives: October 2018

Site index · List index
Message list1 · 2 · 3 · Next »Thread · Author · Date
Girish Vasmatkar Use SparkContext in Web Application Mon, 01 Oct, 06:48
Girish Vasmatkar   Re: Use SparkContext in Web Application Thu, 04 Oct, 04:55
Girish Vasmatkar     Re: Use SparkContext in Web Application Thu, 04 Oct, 04:56
Jörn Franke     Re: Use SparkContext in Web Application Thu, 04 Oct, 06:25
vincent gromakowski       Re: Use SparkContext in Web Application Thu, 04 Oct, 07:00
Girish Vasmatkar         Re: Use SparkContext in Web Application Thu, 04 Oct, 08:16
Mahender Sarangam Unable to read multiple JSON.Gz File. Mon, 01 Oct, 08:59
Jyoti Ranjan Mahapatra   RE: Unable to read multiple JSON.Gz File. Tue, 02 Oct, 20:48
Mahender Sarangam     Re: Unable to read multiple JSON.Gz File. Thu, 18 Oct, 08:30
Re: Time-Series Forecasting
Mina Aslani   Re: Time-Series Forecasting Mon, 01 Oct, 14:16
Re: Pyspark Partitioning
Gourav Sengupta   Re: Pyspark Partitioning Mon, 01 Oct, 16:22
dimitris plakas   Pyspark Partitioning Thu, 04 Oct, 20:27
Vitaliy Pisarev     Re: Pyspark Partitioning Thu, 04 Oct, 20:36
Re: BroadcastJoin failed on partitioned parquet table
Wenchen Fan   Re: BroadcastJoin failed on partitioned parquet table Tue, 02 Oct, 02:37
zakhavan How to do sliding window operation on RDDs in Pyspark? Tue, 02 Oct, 16:30
Taylor Cox   RE: How to do sliding window operation on RDDs in Pyspark? Tue, 02 Oct, 17:37
zakhavan     RE: How to do sliding window operation on RDDs in Pyspark? Tue, 02 Oct, 20:15
Taylor Cox       RE: How to do sliding window operation on RDDs in Pyspark? Tue, 02 Oct, 21:59
zakhavan         RE: How to do sliding window operation on RDDs in Pyspark? Thu, 04 Oct, 16:06
Re: CSV parser - how to parse column containing json data
Nirav Patel   Re: CSV parser - how to parse column containing json data Tue, 02 Oct, 20:59
Brandon Geise     Re: CSV parser - how to parse column containing json data Tue, 02 Oct, 21:45
Aakash Basu How to read remote HDFS from Spark using username? Wed, 03 Oct, 07:02
Aakash Basu   Re: How to read remote HDFS from Spark using username? Wed, 03 Oct, 07:34
Jörn Franke     Re: How to read remote HDFS from Spark using username? Wed, 03 Oct, 07:44
Aakash Basu       Re: How to read remote HDFS from Spark using username? Wed, 03 Oct, 09:19
jcgarciam Restarting a failed Spark streaming job running on top of a yarn cluster Wed, 03 Oct, 12:21
Olivier Girardot Back to SQL Wed, 03 Oct, 17:41
Reynold Xin   Re: Back to SQL Wed, 03 Oct, 20:03
kant kodali How to do a broadcast join using raw Spark SQL 2.3.1 or 2.3.2? Wed, 03 Oct, 23:37
kathleen li   Re: How to do a broadcast join using raw Spark SQL 2.3.1 or 2.3.2? Thu, 04 Oct, 01:22
Jianshi Huang Specifying different version of pyspark.zip and py4j files on worker nodes with Spark pre-installed Thu, 04 Oct, 09:19
Apostolos N. Papadopoulos   Re: Specifying different version of pyspark.zip and py4j files on worker nodes with Spark pre-installed Thu, 04 Oct, 09:51
Marcelo Vanzin   Re: Specifying different version of pyspark.zip and py4j files on worker nodes with Spark pre-installed Thu, 04 Oct, 16:10
Jianshi Huang     Re: Specifying different version of pyspark.zip and py4j files on worker nodes with Spark pre-installed Thu, 04 Oct, 17:22
Marcelo Vanzin       Re: Specifying different version of pyspark.zip and py4j files on worker nodes with Spark pre-installed Thu, 04 Oct, 17:25
Gourav Sengupta         Re: Specifying different version of pyspark.zip and py4j files on worker nodes with Spark pre-installed Thu, 04 Oct, 19:34
Jianshi Huang           Re: Specifying different version of pyspark.zip and py4j files on worker nodes with Spark pre-installed Fri, 05 Oct, 04:46
Jianshi Huang         Re: Specifying different version of pyspark.zip and py4j files on worker nodes with Spark pre-installed Fri, 05 Oct, 04:47
Jianshi Huang         Re: Specifying different version of pyspark.zip and py4j files on worker nodes with Spark pre-installed Fri, 05 Oct, 04:53
Marcelo Vanzin           Re: Specifying different version of pyspark.zip and py4j files on worker nodes with Spark pre-installed Fri, 05 Oct, 15:59
Re: java.lang.IllegalArgumentException: requirement failed: BLAS.dot(x: Vector, y:Vector) was given Vectors with non-matching sizes
hager   Re: java.lang.IllegalArgumentException: requirement failed: BLAS.dot(x: Vector, y:Vector) was given Vectors with non-matching sizes Thu, 04 Oct, 14:53
Sushil Chaudhary subscribe Thu, 04 Oct, 18:48
sushil.chaudhary Spark 2.3.1 leaves _temporary dir back on s3 even after write to s3 is done. Thu, 04 Oct, 20:01
mmuru PySpark structured streaming job throws socket exception Thu, 04 Oct, 20:59
Shixiong(Ryan) Zhu   Re: PySpark structured streaming job throws socket exception Thu, 04 Oct, 21:26
mmuru     Re: PySpark structured streaming job throws socket exception Thu, 04 Oct, 21:40
Jean Georges Perrin Where is the DAG stored before catalyst gets it? Thu, 04 Oct, 22:36
Jacek Laskowski   Re: Where is the DAG stored before catalyst gets it? Sat, 06 Oct, 17:23
Kuttaiah Robin Recreate Dataset<Row> from list of Row in spark streaming application. Fri, 05 Oct, 13:22
Shixiong(Ryan) Zhu   Re: Recreate Dataset<Row> from list of Row in spark streaming application. Fri, 05 Oct, 17:42
Kuttaiah Robin     Re: Recreate Dataset<Row> from list of Row in spark streaming application. Sat, 06 Oct, 15:13
Donni Khan Unsubscribe Fri, 05 Oct, 14:36
Buckler, Christine [PySpark join] Resolved attribute(s) missing from... Attribute(s) with the same name appear in the operation Fri, 05 Oct, 19:59
Re: error in job
Muthu Jayakumar   Re: error in job Sat, 06 Oct, 12:17
阎志涛 Executor hang Sun, 07 Oct, 12:24
kathleen li   Re: Executor hang Sun, 07 Oct, 17:03
阎志涛     答复: Executor hang Sun, 07 Oct, 22:21
kathleen li       Re: 答复: Executor hang Mon, 08 Oct, 04:15
阎志涛         答复: 答复: Executor hang Tue, 09 Oct, 01:16
Shubham Chaurasia Target java version not set when building spark with tags/v2.4.0-rc2 Mon, 08 Oct, 05:45
Deepak Sharma Error while upserting ElasticSearch from Spark 2.2 Mon, 08 Oct, 09:13
冯 远森 td Mon, 08 Oct, 15:10
Nirav Patel CSV parser - is there a way to find malformed csv record Mon, 08 Oct, 18:57
Shuporno Choudhury   Re: CSV parser - is there a way to find malformed csv record Tue, 09 Oct, 14:30
Taylor Cox   RE: CSV parser - is there a way to find malformed csv record Tue, 09 Oct, 20:40
Nirav Patel     Re: CSV parser - is there a way to find malformed csv record Tue, 09 Oct, 22:09
ayan guha SparkR issue Tue, 09 Oct, 06:20
Felix Cheung   Re: SparkR issue Sun, 14 Oct, 22:03
Shubham Chaurasia DataSourceV2 APIs creating multiple instances of DataSourceReader and hence not preserving the state Tue, 09 Oct, 06:31
Mendelson, Assaf   RE: DataSourceV2 APIs creating multiple instances of DataSourceReader and hence not preserving the state Tue, 09 Oct, 10:29
Shubham Chaurasia     Re: DataSourceV2 APIs creating multiple instances of DataSourceReader and hence not preserving the state Tue, 09 Oct, 11:01
Mendelson, Assaf       RE: DataSourceV2 APIs creating multiple instances of DataSourceReader and hence not preserving the state Tue, 09 Oct, 11:12
Shubham Chaurasia         Re: DataSourceV2 APIs creating multiple instances of DataSourceReader and hence not preserving the state Tue, 09 Oct, 13:39
Hyukjin Kwon           Re: DataSourceV2 APIs creating multiple instances of DataSourceReader and hence not preserving the state Wed, 10 Oct, 03:30
Ryan Blue             Re: DataSourceV2 APIs creating multiple instances of DataSourceReader and hence not preserving the state Fri, 19 Oct, 20:57
Jörn Franke           Re: DataSourceV2 APIs creating multiple instances of DataSourceReader and hence not preserving the state Wed, 10 Oct, 05:32
Internal Spark class is not registered by Kryo
Lijun Cao   Internal Spark class is not registered by Kryo Tue, 09 Oct, 11:22
Lijun Cao   Internal Spark class is not registered by Kryo Tue, 09 Oct, 11:29
BOT   Internal Spark class is not registered by Kryo Tue, 09 Oct, 11:47
BOT   Internal Spark class is not registered by Kryo Tue, 09 Oct, 13:34
曹礼俊   Internal Spark class is not registered by Kryo Tue, 09 Oct, 14:18
曹礼俊   Internal Spark class is not registered by Kryo Tue, 09 Oct, 15:00
Lijun Cao Spark internal class is not registered by Kryo Tue, 09 Oct, 11:25
V0lleyBallJunki3 Any way to see the size of the broadcast variable? Tue, 09 Oct, 15:44
Gourav Sengupta   Re: Any way to see the size of the broadcast variable? Tue, 09 Oct, 16:12
V0lleyBallJunki3     Re: Any way to see the size of the broadcast variable? Tue, 09 Oct, 17:48
kant kodali Spark on YARN not utilizing all the YARN containers available Tue, 09 Oct, 17:20
Dillon Dukek   Re: Spark on YARN not utilizing all the YARN containers available Tue, 09 Oct, 19:52
Gourav Sengupta     Re: Spark on YARN not utilizing all the YARN containers available Tue, 09 Oct, 19:57
Dillon Dukek       Re: Spark on YARN not utilizing all the YARN containers available Tue, 09 Oct, 20:04
Gourav Sengupta         Re: Spark on YARN not utilizing all the YARN containers available Tue, 09 Oct, 21:54
Dillon Dukek           Re: Spark on YARN not utilizing all the YARN containers available Wed, 10 Oct, 05:38
Gourav Sengupta             Re: Spark on YARN not utilizing all the YARN containers available Wed, 10 Oct, 09:58
kant kodali Does spark.streaming.concurrentJobs still exist? Tue, 09 Oct, 18:50
zakhavan Exception: It appears that you are attempting to reference SparkContext from a broadcast variable, action, or transformation. SparkContext can only be used on the driver, not in code that it run on workers. For more information, see SPARK-5063. Tue, 09 Oct, 19:46
Li Gao [K8S] Option to keep the executor pods after job finishes Tue, 09 Oct, 21:17
Yinan Li   Re: [K8S] Option to keep the executor pods after job finishes Tue, 09 Oct, 22:00
Ramaswamy, Muthuraman PySpark Streaming : Accessing the Remote Secured Kafka Wed, 10 Oct, 01:31
sparksql exception when using regexp_replace
付涛   sparksql exception when using regexp_replace Wed, 10 Oct, 08:57
付涛   sparksql exception when using regexp_replace Wed, 10 Oct, 08:57
Jean Georges Perrin Triangle Apache Spark Meetup Wed, 10 Oct, 09:54
Theyaa Matti Bad Message 413 Request Entity too large - Spark History UI through Knox Wed, 10 Oct, 13:43
☼ R Nair getBytes : save as pdf Wed, 10 Oct, 15:30
Joel D   Re: getBytes : save as pdf Thu, 11 Oct, 02:35
Message list1 · 2 · 3 · Next »Thread · Author · Date
Box list
Dec 201953
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137