spark-user mailing list archives: August 2016

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · Next »Thread · Author · Date
pseudo oduesp decribe function limit of columns Tue, 02 Aug, 13:39
janardhan shetty   Re: decribe function limit of columns Tue, 02 Aug, 20:34
dueckm Are join/groupBy operations with wide Java Beans using Dataset API much slower than using RDD API? [*] Tue, 02 Aug, 13:50
Mich Talebzadeh Extracting key word from a textual column Tue, 02 Aug, 14:00
Yong Zhang   Re: Extracting key word from a textual column Tue, 02 Aug, 15:26
Mich Talebzadeh     Re: Extracting key word from a textual column Tue, 02 Aug, 16:25
Sonal Goyal       Re: Extracting key word from a textual column Tue, 02 Aug, 16:56
Mich Talebzadeh         Re: Extracting key word from a textual column Tue, 02 Aug, 20:34
Jörn Franke           Re: Extracting key word from a textual column Tue, 02 Aug, 21:29
Ted Yu             Re: Extracting key word from a textual column Tue, 02 Aug, 22:10
Mich Talebzadeh               Re: Extracting key word from a textual column Tue, 02 Aug, 22:51
ayan guha                 Re: Extracting key word from a textual column Tue, 02 Aug, 23:20
Jörn Franke                   Re: Extracting key word from a textual column Wed, 03 Aug, 05:32
Mich Talebzadeh                     Re: Extracting key word from a textual column Wed, 03 Aug, 10:26
Jörn Franke                 Re: Extracting key word from a textual column Wed, 03 Aug, 05:32
Satyanarayan Patel Saving input schema along with PipelineModel Tue, 02 Aug, 15:21
Satya Narayan1 Fwd: Saving input schema along with PipelineModel Tue, 02 Aug, 15:32
Re: spark 1.6.0 read s3 files error.
freedafeng   Re: spark 1.6.0 read s3 files error. Tue, 02 Aug, 16:26
Andy Davidson     Re: spark 1.6.0 read s3 files error. Tue, 02 Aug, 16:54
freedafeng     Re: spark 1.6.0 read s3 files error. Tue, 02 Aug, 17:06
FW: [jupyter] newbie. apache spark python3 'Jupyter' data frame problem with auto completion and accessing documentation
Andy Davidson   FW: [jupyter] newbie. apache spark python3 'Jupyter' data frame problem with auto completion and accessing documentation Tue, 02 Aug, 16:57
Liangzhao Zeng Job can not terminated in Spark 2.0 on Yarn Tue, 02 Aug, 17:06
Ted Yu   Re: Job can not terminated in Spark 2.0 on Yarn Tue, 02 Aug, 18:01
dhruve ashar     Re: Job can not terminated in Spark 2.0 on Yarn Tue, 02 Aug, 18:24
Liangzhao Zeng       Re: Job can not terminated in Spark 2.0 on Yarn Tue, 02 Aug, 19:06
Liangzhao Zeng     Re: Job can not terminated in Spark 2.0 on Yarn Tue, 02 Aug, 19:13
dhruve ashar       Re: Job can not terminated in Spark 2.0 on Yarn Tue, 02 Aug, 19:46
Liangzhao Zeng         Re: Job can not terminated in Spark 2.0 on Yarn Tue, 02 Aug, 20:02
Cesar saving data frame to optimize joins at a later time Tue, 02 Aug, 19:09
pgb In 2.0.0, is it possible to fetch a query from an external database (rather than grab the whole table)? Tue, 02 Aug, 20:25
Jacek Laskowski   Re: In 2.0.0, is it possible to fetch a query from an external database (rather than grab the whole table)? Tue, 02 Aug, 22:38
Mich Talebzadeh     Re: In 2.0.0, is it possible to fetch a query from an external database (rather than grab the whole table)? Tue, 02 Aug, 23:41
Dragisa Krsmanovic [2.0.0] mapPartitions on DataFrame unable to find encoder Tue, 02 Aug, 20:55
Ted Yu   Re: [2.0.0] mapPartitions on DataFrame unable to find encoder Tue, 02 Aug, 23:50
Dragisa Krsmanovic     Re: [2.0.0] mapPartitions on DataFrame unable to find encoder Tue, 02 Aug, 23:59
Sun Rui   Re: [2.0.0] mapPartitions on DataFrame unable to find encoder Wed, 03 Aug, 01:58
Dragisa Krsmanovic     Re: [2.0.0] mapPartitions on DataFrame unable to find encoder Wed, 03 Aug, 16:35
Re: calling dataset.show on a custom object - displays toString() value as first column and blank for rest
Jacek Laskowski   Re: calling dataset.show on a custom object - displays toString() value as first column and blank for rest Tue, 02 Aug, 22:44
Utkarsh Sengar Spark 2.0 error: Wrong FS: file://spark-warehouse, expected: file:/// Tue, 02 Aug, 23:47
Sean Owen   Re: Spark 2.0 error: Wrong FS: file://spark-warehouse, expected: file:/// Tue, 02 Aug, 23:48
Utkarsh Sengar     Re: Spark 2.0 error: Wrong FS: file://spark-warehouse, expected: file:/// Wed, 03 Aug, 00:02
Utkarsh Sengar       Re: Spark 2.0 error: Wrong FS: file://spark-warehouse, expected: file:/// Wed, 03 Aug, 00:53
Ulanov, Alexander     RE: Spark 2.0 error: Wrong FS: file://spark-warehouse, expected: file:/// Wed, 03 Aug, 12:28
Stop Spark Streaming Jobs
Pradeep   Stop Spark Streaming Jobs Wed, 03 Aug, 01:47
Park Kyeong Hee     RE: Stop Spark Streaming Jobs Wed, 03 Aug, 02:24
Park Kyeong Hee     FW: Stop Spark Streaming Jobs Wed, 03 Aug, 02:25
Pradeep       Re: Stop Spark Streaming Jobs Wed, 03 Aug, 03:23
Tony Lane         Re: Stop Spark Streaming Jobs Wed, 03 Aug, 15:44
Sandeep Nemuri           Re: Stop Spark Streaming Jobs Thu, 04 Aug, 07:01
Sandeep Nemuri             Re: Stop Spark Streaming Jobs Thu, 04 Aug, 07:03
satyajit vegesna Spark on yarn, only 1 or 2 vcores getting allocated to the containers getting created. Wed, 03 Aug, 04:11
Saisai Shao   Re: Spark on yarn, only 1 or 2 vcores getting allocated to the containers getting created. Wed, 03 Aug, 07:53
Mungeol Heo     Re: Spark on yarn, only 1 or 2 vcores getting allocated to the containers getting created. Thu, 04 Aug, 01:16
Mungeol Heo     Re: Spark on yarn, only 1 or 2 vcores getting allocated to the containers getting created. Thu, 04 Aug, 01:17
Abhishek Ranjan Relative path in absolute URI Wed, 03 Aug, 06:54
Sean Owen   Re: Relative path in absolute URI Wed, 03 Aug, 10:19
luohui20...@sina.com How to get recommand result for users in a kafka SparkStreaming Application Wed, 03 Aug, 07:01
Cody Koeninger   Re: How to get recommand result for users in a kafka SparkStreaming Application Wed, 03 Aug, 13:30
luohui20...@sina.com 回复:How to get recommand result for users in a kafka SparkStreaming Application Wed, 03 Aug, 07:27
colin "object cannot be cast to Double" using pipline with pyspark Wed, 03 Aug, 08:15
lev spark 2.0.0 - how to build an uber-jar? Wed, 03 Aug, 08:43
Mich Talebzadeh   Re: spark 2.0.0 - how to build an uber-jar? Wed, 03 Aug, 08:51
Saisai Shao     Re: spark 2.0.0 - how to build an uber-jar? Wed, 03 Aug, 08:54
Mich Talebzadeh     Re: spark 2.0.0 - how to build an uber-jar? Wed, 03 Aug, 10:01
Re: How to partition a SparkDataFrame using all distinct column values in sparkR
Sun Rui   Re: How to partition a SparkDataFrame using all distinct column values in sparkR Wed, 03 Aug, 10:04
eviekas Spark 2.0 empty result in some tpc-h queries Wed, 03 Aug, 10:55
Bhupendra Mishra Spark steaming with Flume jobs failing Wed, 03 Aug, 11:00
Carlo.Allocca converting a Dataset<Row> into JavaRDD<LabeledPoint> Wed, 03 Aug, 11:14
Carlo.Allocca   Re: converting a Dataset<Row> into JavaRDD<LabeledPoint> Wed, 03 Aug, 11:32
Yana Kadiyska [Thriftserver2] Controlling number of tasks Wed, 03 Aug, 12:03
ayan guha   Re: [Thriftserver2] Controlling number of tasks Wed, 03 Aug, 14:42
Chanh Le   Re: [Thriftserver2] Controlling number of tasks Wed, 03 Aug, 16:13
Takeshi Yamamuro     Re: [Thriftserver2] Controlling number of tasks Wed, 03 Aug, 16:30
Rychnovsky, Dusan Managed memory leak detected + OutOfMemoryError: Unable to acquire X bytes of memory, got 0 Wed, 03 Aug, 12:03
Ted Yu   Re: Managed memory leak detected + OutOfMemoryError: Unable to acquire X bytes of memory, got 0 Wed, 03 Aug, 13:52
Rychnovsky, Dusan   Re: Managed memory leak detected + OutOfMemoryError: Unable to acquire X bytes of memory, got 0 Wed, 03 Aug, 13:58
Rychnovsky, Dusan     Re: Managed memory leak detected + OutOfMemoryError: Unable to acquire X bytes of memory, got 0 Wed, 03 Aug, 16:05
Ted Yu       Re: Managed memory leak detected + OutOfMemoryError: Unable to acquire X bytes of memory, got 0 Wed, 03 Aug, 16:10
Rychnovsky, Dusan         Re: Managed memory leak detected + OutOfMemoryError: Unable to acquire X bytes of memory, got 0 Wed, 03 Aug, 16:11
Ted Yu           Re: Managed memory leak detected + OutOfMemoryError: Unable to acquire X bytes of memory, got 0 Wed, 03 Aug, 16:20
Aseem Bansal Spark 2.0 - Case sensitive column names while reading csv Wed, 03 Aug, 12:04
ravi.gawai Sparkstreaming not consistently picking files from streaming directory Wed, 03 Aug, 12:55
Ben Teeuwen OOM with StringIndexer, 800m rows & 56m distinct value column Wed, 03 Aug, 14:00
Ben Teeuwen   Re: OOM with StringIndexer, 800m rows & 56m distinct value column Thu, 04 Aug, 08:07
Nick Pentreath     Re: OOM with StringIndexer, 800m rows & 56m distinct value column Thu, 04 Aug, 10:14
Ben Teeuwen       Re: OOM with StringIndexer, 800m rows & 56m distinct value column Thu, 04 Aug, 13:25
Nick Pentreath         Re: OOM with StringIndexer, 800m rows & 56m distinct value column Thu, 04 Aug, 13:44
Ben Teeuwen           Re: OOM with StringIndexer, 800m rows & 56m distinct value column Thu, 11 Aug, 20:14
Nick Pentreath             Re: OOM with StringIndexer, 800m rows & 56m distinct value column Thu, 11 Aug, 20:24
Ben Teeuwen               Re: OOM with StringIndexer, 800m rows & 56m distinct value column Fri, 19 Aug, 15:19
Davies Liu                 Re: OOM with StringIndexer, 800m rows & 56m distinct value column Fri, 19 Aug, 21:33
Davies Liu                   Re: OOM with StringIndexer, 800m rows & 56m distinct value column Fri, 19 Aug, 21:34
Ben Teeuwen                     Re: OOM with StringIndexer, 800m rows & 56m distinct value column Mon, 22 Aug, 13:05
[SQL] Reading from hive table is listing all files in S3
Mehdi Meziane   [SQL] Reading from hive table is listing all files in S3 Wed, 03 Aug, 14:03
Mich Talebzadeh     Re: [SQL] Reading from hive table is listing all files in S3 Wed, 03 Aug, 14:47
Mehdi Meziane   Re: [SQL] Reading from hive table is listing all files in S3 Wed, 03 Aug, 16:08
Gourav Sengupta     Re: [SQL] Reading from hive table is listing all files in S3 Wed, 03 Aug, 16:24
Mich Talebzadeh     Re: [SQL] Reading from hive table is listing all files in S3 Wed, 03 Aug, 16:25
Paolo Patierno Python : StreamingContext isn't created properly Wed, 03 Aug, 14:16
Paolo Patierno   RE: Python : StreamingContext isn't created properly Wed, 03 Aug, 14:51
Selvam Raman SparkSession for RDBMS Wed, 03 Aug, 14:19
Takeshi Yamamuro   Re: SparkSession for RDBMS Wed, 03 Aug, 16:43
Tony Lane Error in building spark core on windows - any suggestions please Wed, 03 Aug, 14:30
Tony Lane   Re: Error in building spark core on windows - any suggestions please Wed, 03 Aug, 15:05
Sean Owen   Re: Error in building spark core on windows - any suggestions please Wed, 03 Aug, 15:06
Soumitra Johri How does MapWithStateRDD distribute the data Wed, 03 Aug, 14:42
Ben Teeuwen   Re: How does MapWithStateRDD distribute the data Wed, 03 Aug, 16:32
Cody Koeninger   Re: How does MapWithStateRDD distribute the data Wed, 03 Aug, 16:34
Kazuaki Ishizaki Change nullable property in Dataset schema Wed, 03 Aug, 14:45
Kazuaki Ishizaki   Re: Change nullable property in Dataset schema Wed, 10 Aug, 07:04
Jacek Laskowski     Re: Change nullable property in Dataset schema Sun, 14 Aug, 19:55
Kazuaki Ishizaki       Re: Change nullable property in Dataset schema Wed, 17 Aug, 06:47
Koert Kuipers   Re: Change nullable property in Dataset schema Mon, 15 Aug, 19:34
Kazuaki Ishizaki     Re: Change nullable property in Dataset schema Wed, 17 Aug, 06:18
Rohit Chaddha Calling KmeansModel predict method Wed, 03 Aug, 16:24
Tony Lane   Re: Calling KmeansModel predict method Wed, 03 Aug, 17:41
Re: How to generate a sequential key in rdd across executors
yeshwanth kumar   Re: How to generate a sequential key in rdd across executors Wed, 03 Aug, 16:35
Carlo.Allocca Dataset<Row> and JavaRDD<Row>: how to eliminate the header. Wed, 03 Aug, 16:44
Aseem Bansal   Re: Dataset<Row> and JavaRDD<Row>: how to eliminate the header. Wed, 03 Aug, 17:13
Carlo.Allocca     Re: Dataset<Row> and JavaRDD<Row>: how to eliminate the header. Wed, 03 Aug, 17:32
Mich Talebzadeh       Re: Dataset<Row> and JavaRDD<Row>: how to eliminate the header. Wed, 03 Aug, 17:45
Carlo.Allocca         Re: Dataset<Row> and JavaRDD<Row>: how to eliminate the header. Wed, 03 Aug, 17:58
Carlo.Allocca           Re: Dataset<Row> and JavaRDD<Row>: how to eliminate the header. Wed, 03 Aug, 19:01
Mich Talebzadeh             Re: Dataset<Row> and JavaRDD<Row>: how to eliminate the header. Wed, 03 Aug, 20:43
Carlo.Allocca               Re: Dataset<Row> and JavaRDD<Row>: how to eliminate the header. Wed, 03 Aug, 20:49
Mich Talebzadeh                 Re: Dataset<Row> and JavaRDD<Row>: how to eliminate the header. Wed, 03 Aug, 21:01
Carlo.Allocca                   Re: Dataset<Row> and JavaRDD<Row>: how to eliminate the header. Wed, 03 Aug, 21:14
Utkarsh Sengar Spark 2.0: Task never completes Wed, 03 Aug, 17:04
Utkarsh Sengar   Re: Spark 2.0: Task never completes Thu, 04 Aug, 01:35
Tony Lane Using sparse vector leads to array out of bounds exception Wed, 03 Aug, 17:43
Sean Owen   Re: Using sparse vector leads to array out of bounds exception Wed, 03 Aug, 17:52
Tony Lane     Re: Using sparse vector leads to array out of bounds exception Wed, 03 Aug, 18:52
Sean Owen       Re: Using sparse vector leads to array out of bounds exception Wed, 03 Aug, 19:09
Tony Lane         Re: Using sparse vector leads to array out of bounds exception Wed, 03 Aug, 19:45
Sean Owen           Re: Using sparse vector leads to array out of bounds exception Wed, 03 Aug, 19:51
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · Next »Thread · Author · Date
Box list
Nov 2020134
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137