spark-user mailing list archives: April 2019

Site index · List index
Message list1 · 2 · 3 · Next »Thread · Author · Date
Re: Spark SQL API taking longer time than DF API.
neeraj bhadani   Re: Spark SQL API taking longer time than DF API. Mon, 01 Apr, 08:44
neeraj bhadani     Re: Spark SQL API taking longer time than DF API. Mon, 08 Apr, 08:21
ch...@cmartinit.co.uk       Re: Spark SQL API taking longer time than DF API. Mon, 08 Apr, 08:31
Yeikel         Re: Spark SQL API taking longer time than DF API. Wed, 17 Apr, 18:19
Re: [spark sql performance] Only 1 executor to write output?
Mike Chan   Re: [spark sql performance] Only 1 executor to write output? Mon, 01 Apr, 09:12
Re: Spark Kafka Batch Write guarantees
Shixiong(Ryan) Zhu   Re: Spark Kafka Batch Write guarantees Mon, 01 Apr, 16:13
hemant singh     Re: Spark Kafka Batch Write guarantees Mon, 01 Apr, 18:32
Re: Understanding State Store storage behavior for the Stream Deduplication function
Shixiong(Ryan) Zhu   Re: Understanding State Store storage behavior for the Stream Deduplication function Mon, 01 Apr, 16:20
Gerard Maas     Re: Understanding State Store storage behavior for the Stream Deduplication function Mon, 01 Apr, 16:47
Arun Mahadevan       Re: Understanding State Store storage behavior for the Stream Deduplication function Mon, 01 Apr, 17:51
Re: Spark streaming error - Query terminated with exception: assertion failed: Invalid batch: a#660,b#661L,c#662,d#663,,… 26 more fields != b#1291L
Shixiong(Ryan) Zhu   Re: Spark streaming error - Query terminated with exception: assertion failed: Invalid batch: a#660,b#661L,c#662,d#663,,… 26 more fields != b#1291L Mon, 01 Apr, 16:26
Steve Pruitt [Spark ML] [Pyspark] [Scenario Beginner] [Level Beginner] Mon, 01 Apr, 16:39
Steve Pruitt   [Spark ML] [Pyspark] [Scenario Beginner] [Level Beginner] Tue, 02 Apr, 15:13
Alok Bhandari MLLIB , Does Spark support Canopy Clustering ? Tue, 02 Apr, 12:57
Jack Kolokasis Load Time from HDFS Tue, 02 Apr, 14:06
yeikel valdes   Re:Load Time from HDFS Wed, 10 Apr, 17:55
Mich Talebzadeh     Re: Load Time from HDFS Wed, 10 Apr, 19:32
Dmitry Goldenberg Issues with Spark Streaming checkpointing of Kafka topic content Tue, 02 Apr, 15:39
Dmitry Goldenberg   Re: Issues with Spark Streaming checkpointing of Kafka topic content Tue, 02 Apr, 15:48
Re: How to extract data in parallel from RDBMS tables
Surendra , Manchikanti   Re: How to extract data in parallel from RDBMS tables Tue, 02 Apr, 18:07
Jason Nerothin     Re: How to extract data in parallel from RDBMS tables Tue, 02 Apr, 19:18
Magnus Nilsson Logging DataFrame API pipelines Tue, 02 Apr, 22:43
Re: Upcoming talks on BigDL and Analytics Zoo this week
Jason Dai   Re: Upcoming talks on BigDL and Analytics Zoo this week Wed, 03 Apr, 13:21
VHPC 19 CfP VHPC19: HPC Virtualization-Containers: Paper due May 1, 2019 (extended) Wed, 03 Apr, 16:38
Arthur Li Question about relationship between number of files and initial tasks(partitions) Thu, 04 Apr, 01:37
Jason Nerothin   Re: Question about relationship between number of files and initial tasks(partitions) Thu, 04 Apr, 13:52
yeikel valdes     Re: Question about relationship between number of files and initial tasks(partitions) Wed, 10 Apr, 17:51
Sagar Grover       Re: Question about relationship between number of files and initial tasks(partitions) Thu, 11 Apr, 12:22
em...@yeikel.com         RE: Question about relationship between number of files and initial tasks(partitions) Sat, 13 Apr, 18:35
Chetan Khatri dropDuplicate on timestamp based column unexpected output Thu, 04 Apr, 04:51
Abdeali Kothari   Re: dropDuplicate on timestamp based column unexpected output Thu, 04 Apr, 05:11
Chetan Khatri     Re: dropDuplicate on timestamp based column unexpected output Thu, 04 Apr, 06:15
Abdeali Kothari       Re: dropDuplicate on timestamp based column unexpected output Thu, 04 Apr, 06:40
Chetan Khatri         Re: dropDuplicate on timestamp based column unexpected output Thu, 04 Apr, 07:38
Abdeali Kothari           Re: dropDuplicate on timestamp based column unexpected output Thu, 04 Apr, 11:32
Chetan Khatri             Re: dropDuplicate on timestamp based column unexpected output Thu, 04 Apr, 12:49
Jason Nerothin               Re: dropDuplicate on timestamp based column unexpected output Thu, 04 Apr, 13:46
Jason Nerothin                 Re: dropDuplicate on timestamp based column unexpected output Thu, 04 Apr, 17:13
Chetan Khatri                   Re: dropDuplicate on timestamp based column unexpected output Thu, 04 Apr, 18:24
Doaa Medhat Why "spark-streaming-kafka-0-10" is still experimental? Thu, 04 Apr, 07:52
Adaryl Wakefield pickling a udf Thu, 04 Apr, 10:11
Abdeali Kothari   Re: pickling a udf Thu, 04 Apr, 11:34
Adaryl Wakefield     RE: pickling a udf Thu, 04 Apr, 17:36
Jeff Evans Why does this spark-shell invocation get suspended due to tty output? Thu, 04 Apr, 16:21
Prasad Bhalerao reporting use case Thu, 04 Apr, 18:48
Jason Nerothin   Re: reporting use case Thu, 04 Apr, 19:05
Prasad Bhalerao     Re: reporting use case Thu, 04 Apr, 19:23
Teemu Heikkilä       Re: reporting use case Thu, 04 Apr, 19:27
Hall, Steven         Re: <External>Re: reporting use case Thu, 04 Apr, 20:38
Prasad Bhalerao         Re: reporting use case Fri, 05 Apr, 01:19
Serena S Yuan Qn about decision tree apache spark java Thu, 04 Apr, 21:36
Abdeali Kothari   Re: Qn about decision tree apache spark java Thu, 04 Apr, 22:32
Re: How shall I configure the Spark executor memory size and the Alluxio worker memory size on a machine?
Bin Fan   Re: How shall I configure the Spark executor memory size and the Alluxio worker memory size on a machine? Fri, 05 Apr, 04:29
Bin Fan     Re: How shall I configure the Spark executor memory size and the Alluxio worker memory size on a machine? Fri, 05 Apr, 05:27
DB Tsai [ANNOUNCE] Announcing Apache Spark 2.4.1 Fri, 05 Apr, 05:59
Madabhattula Rajesh Kumar combineByKey Fri, 05 Apr, 07:11
Jason Nerothin   Re: combineByKey Fri, 05 Apr, 16:28
Madabhattula Rajesh Kumar     Re: combineByKey Fri, 05 Apr, 17:25
Jason Nerothin       Re: combineByKey Fri, 05 Apr, 17:30
Shyam P Is there any spark API function to handle a group of companies at once in this scenario? Fri, 05 Apr, 09:50
Basavaraj   Checking if cascading graph computation is possible in Spark Fri, 05 Apr, 11:35
Jason Nerothin     Re: Checking if cascading graph computation is possible in Spark Fri, 05 Apr, 16:43
Basavaraj       Re: Checking if cascading graph computation is possible in Spark Fri, 05 Apr, 18:15
Jason Nerothin         Re: Checking if cascading graph computation is possible in Spark Fri, 05 Apr, 18:48
Basavaraj   Checking if cascading graph computation is possible in Spark Fri, 05 Apr, 11:36
Mich Talebzadeh   Re: Is there any spark API function to handle a group of companies at once in this scenario? Sun, 07 Apr, 08:34
Shyam P     Re: Is there any spark API function to handle a group of companies at once in this scenario? Mon, 08 Apr, 07:18
Mich Talebzadeh       Re: Is there any spark API function to handle a group of companies at once in this scenario? Tue, 09 Apr, 16:38
Shyam P         Re: Is there any spark API function to handle a group of companies at once in this scenario? Sat, 13 Apr, 00:54
Mich Talebzadeh How to retrieve multiple columns values (in one row) to variables in Spark Scala method Fri, 05 Apr, 19:28
ayan guha   Re: How to retrieve multiple columns values (in one row) to variables in Spark Scala method Fri, 05 Apr, 22:56
Mich Talebzadeh     Re: How to retrieve multiple columns values (in one row) to variables in Spark Scala method Sat, 06 Apr, 15:45
Lian Jiang writing into oracle database is very slow Sat, 06 Apr, 14:59
Yeikel   Re: writing into oracle database is very slow Sat, 13 Apr, 18:44
Jörn Franke   Re: writing into oracle database is very slow Thu, 18 Apr, 15:34
spark receiver     Re: writing into oracle database is very slow Fri, 19 Apr, 08:46
Khare, Ankit       Re: writing into oracle database is very slow Fri, 19 Apr, 13:58
Lian Jiang         Re: writing into oracle database is very slow Fri, 19 Apr, 18:42
Mich Talebzadeh           Re: writing into oracle database is very slow Fri, 19 Apr, 21:36
M Bilal Observing DAGScheduler Log Messages Sun, 07 Apr, 16:04
Jacek Laskowski   Re: Observing DAGScheduler Log Messages Sun, 07 Apr, 19:21
M Bilal     Re: Observing DAGScheduler Log Messages Mon, 08 Apr, 05:26
Manu Zhang Spark driver crashed with internal error Mon, 08 Apr, 03:00
Paul.Baurie...@telekom.de Parallelize Join Problem Mon, 08 Apr, 15:41
Yeikel   Re: Parallelize Join Problem Wed, 17 Apr, 18:12
asma zgolli     Re: Parallelize Join Problem Wed, 17 Apr, 18:18
Re: spark-sklearn
Sudhir Babu Pothineni   Re: spark-sklearn Mon, 08 Apr, 18:43
Stephen Boesch     Re: spark-sklearn Mon, 08 Apr, 18:52
Sudhir Babu Pothineni       Re: spark-sklearn Mon, 08 Apr, 20:22
Abdeali Kothari         Re: spark-sklearn Tue, 09 Apr, 04:17
Siddharth Reddy [No Subject] Mon, 08 Apr, 18:53
Subash Prabakar Spark2: Deciphering saving text file name Tue, 09 Apr, 00:54
Jason Nerothin   Re: Spark2: Deciphering saving text file name Tue, 09 Apr, 17:05
Akila Wajirasena Structured streaming flatMapGroupWithState results out of order messages when reading from Kafka Tue, 09 Apr, 09:37
Jason Nerothin   Re: Structured streaming flatMapGroupWithState results out of order messages when reading from Kafka Tue, 09 Apr, 15:54
Akila Wajirasena     Re: Structured streaming flatMapGroupWithState results out of order messages when reading from Kafka Wed, 10 Apr, 07:30
Tomasz Krol Refresh parquet metadata on Spark Thrift Server Tue, 09 Apr, 16:05
V0lleyBallJunki3 Unable to broadcast a very large variable Wed, 10 Apr, 09:06
Ashic Mahtab   Re: Unable to broadcast a very large variable Wed, 10 Apr, 09:10
V0lleyBallJunki3     Re: Unable to broadcast a very large variable Wed, 10 Apr, 16:40
Dillon Dukek       Re: Unable to broadcast a very large variable Wed, 10 Apr, 17:00
Siddharth Reddy         Re: Unable to broadcast a very large variable Wed, 10 Apr, 17:45
V0lleyBallJunki3         Re: Unable to broadcast a very large variable Thu, 11 Apr, 23:52
Dillon Dukek           Re: Unable to broadcast a very large variable Fri, 12 Apr, 16:17
Chetan Khatri How to print DataFrame.show(100) to text file at HDFS Sat, 13 Apr, 13:10
Nuthan Reddy   Re: How to print DataFrame.show(100) to text file at HDFS Sun, 14 Apr, 05:01
Chetan Khatri     Re: How to print DataFrame.show(100) to text file at HDFS Sun, 14 Apr, 09:09
Brandon Geise       Re: How to print DataFrame.show(100) to text file at HDFS Sun, 14 Apr, 13:53
em...@yeikel.com         RE: How to print DataFrame.show(100) to text file at HDFS Sun, 14 Apr, 17:58
Jungtaek Lim Offline state manipulation tool for structured streaming query Sat, 13 Apr, 14:13
Felix Cheung ApacheCon NA 2019 Call For Proposal and help promoting Spark project Sat, 13 Apr, 16:50
Felix Cheung   Re: ApacheCon NA 2019 Call For Proposal and help promoting Spark project Sun, 14 Apr, 19:48
Debabrata Ghosh Best Practice for Writing data into a Hive table Sat, 13 Apr, 16:59
Yeikel   Re: Best Practice for Writing data into a Hive table Sat, 13 Apr, 18:40
--jars vs --spark.executor.extraClassPath vs --spark.driver.extraClassPath
rajat kumar   --jars vs --spark.executor.extraClassPath vs --spark.driver.extraClassPath Mon, 15 Apr, 04:01
rajat kumar     Re: --jars vs --spark.executor.extraClassPath vs --spark.driver.extraClassPath Sat, 20 Apr, 08:04
Subash Prabakar       Re: --jars vs --spark.executor.extraClassPath vs --spark.driver.extraClassPath Sat, 20 Apr, 09:34
Jason Nerothin         Re: --jars vs --spark.executor.extraClassPath vs --spark.driver.extraClassPath Sat, 20 Apr, 13:55
Message list1 · 2 · 3 · Next »Thread · Author · Date
Box list
May 2019207
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137