spark-user mailing list archives: March 2018

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · 5 · Next »Thread · Author · Date
Christopher Piggott Custom metrics sink Fri, 16 Mar, 20:09
Felix Cheung   Re: Custom metrics sink Sat, 17 Mar, 00:17
Silvio Fiorito   Re: Custom metrics sink Sat, 17 Mar, 01:39
Koert Kuipers change spark default for a setting without overriding user Fri, 16 Mar, 22:43
Bahubali Jain Dataframe size using RDDStorageInfo objects Sat, 17 Mar, 06:05
Serega Sheypak Append more files to existing partitioned data Sat, 17 Mar, 12:18
Denis Bolshakov   Re: Append more files to existing partitioned data Sat, 17 Mar, 13:15
Serega Sheypak     Re: Append more files to existing partitioned data Sat, 17 Mar, 17:46
Denis Bolshakov       Re: Append more files to existing partitioned data Sun, 18 Mar, 08:30
Serega Sheypak         Re: Append more files to existing partitioned data Sun, 18 Mar, 15:42
Vitaliy Pisarev Accessing a file that was passed via --files to spark submit Sun, 18 Mar, 08:06
Marcelo Vanzin   Re: Accessing a file that was passed via --files to spark submit Mon, 19 Mar, 17:23
Mahender Sarangam Dynamic Key JSON Parsing Sun, 18 Mar, 10:12
Mahender Sarangam Scala - Spark for beginners Sun, 18 Mar, 10:15
Gerard Maas   Re: Scala - Spark for beginners Sun, 18 Mar, 15:30
CPC parquet late column materialization Sun, 18 Mar, 17:02
nguyen duc Tuan   Re: parquet late column materialization Sun, 18 Mar, 17:09
CPC     Re: parquet late column materialization Sun, 18 Mar, 17:52
nguyen duc Tuan       Re: parquet late column materialization Sun, 18 Mar, 23:56
Serega Sheypak Run spark 2.2 on yarn as usual java application Sun, 18 Mar, 23:19
Jacek Laskowski   Re: Run spark 2.2 on yarn as usual java application Mon, 19 Mar, 06:16
Serega Sheypak     Re: Run spark 2.2 on yarn as usual java application Mon, 19 Mar, 11:58
Jörn Franke       Re: Run spark 2.2 on yarn as usual java application Mon, 19 Mar, 12:41
Serega Sheypak         Re: Run spark 2.2 on yarn as usual java application Mon, 19 Mar, 13:02
Gurusamy Thirupathy Hive to Oracle using Spark - Type(Date) conversion issue Mon, 19 Mar, 02:34
Deepak Sharma   Re: Hive to Oracle using Spark - Type(Date) conversion issue Mon, 19 Mar, 02:36
ayan guha     Re: Hive to Oracle using Spark - Type(Date) conversion issue Mon, 19 Mar, 03:04
Gurusamy Thirupathy     Re: Hive to Oracle using Spark - Type(Date) conversion issue Tue, 20 Mar, 02:45
Jörn Franke       Re: Hive to Oracle using Spark - Type(Date) conversion issue Tue, 20 Mar, 07:17
Gurusamy Thirupathy         Re: Hive to Oracle using Spark - Type(Date) conversion issue Wed, 21 Mar, 03:08
Debabrata Ghosh Calling Pyspark functions in parallel Mon, 19 Mar, 05:54
Jules Damji   Re: Calling Pyspark functions in parallel Mon, 19 Mar, 13:46
Debabrata Ghosh     Re: Calling Pyspark functions in parallel Mon, 19 Mar, 16:43
Artem Moskvin [Spark Structured Streaming, Spark 2.3.0] Calling current_timestamp() function within a streaming dataframe results in dataType error Mon, 19 Mar, 14:08
Shahab Yunus Warnings on data insert into Hive Table using PySpark Mon, 19 Mar, 15:41
Michael Shtelma Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 19 Mar, 16:59
Keith Chapman   Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 19 Mar, 17:07
Michael Shtelma     Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 19 Mar, 17:29
Keith Chapman       Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 19 Mar, 17:38
Michael Shtelma         Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 19 Mar, 20:05
Keith Chapman           Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 26 Mar, 19:15
Michael Shtelma             Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 26 Mar, 19:28
Gourav Sengupta               Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Wed, 28 Mar, 05:37
Michael Shtelma                 Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Wed, 28 Mar, 09:00
Gourav Sengupta                   Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Wed, 28 Mar, 09:22
Geoff Von Allmen Structured Streaming: distinct (Spark 2.2) Mon, 19 Mar, 19:04
Burak Yavuz   Re: Structured Streaming: distinct (Spark 2.2) Mon, 19 Mar, 19:18
kant kodali select count * doesnt seem to respect update mode in Kafka Structured Streaming? Mon, 19 Mar, 20:35
Geoff Von Allmen   Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Mon, 19 Mar, 20:57
kant kodali     Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Tue, 20 Mar, 00:27
Geoff Von Allmen       Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Tue, 20 Mar, 13:09
kant kodali         Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Tue, 20 Mar, 21:58
Michael Armbrust           Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Wed, 21 Mar, 00:00
kant kodali             Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Wed, 21 Mar, 02:28
1427357...@qq.com the meaining of "samplePointsPerPartitionHint" in RangePartitioner Tue, 20 Mar, 07:59
SNEHASISH DUTTA Access Table with Spark Dataframe Tue, 20 Mar, 13:17
hemant singh   Re: Access Table with Spark Dataframe Tue, 20 Mar, 15:09
Re: [Structured Streaming] Commit protocol to move temp files to dest path only when complete, with code
dcam   Re: [Structured Streaming] Commit protocol to move temp files to dest path only when complete, with code Tue, 20 Mar, 14:09
lucas-vsco [Structured Streaming] Query Metrics to MetricsSink Tue, 20 Mar, 14:36
lucas-vsco   Re: [Structured Streaming] Query Metrics to MetricsSink Tue, 20 Mar, 15:11
purna pradeep Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 02:47
Yinan Li   Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 03:50
purna pradeep     Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 12:01
Gourav Sengupta   Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 14:32
lucas.g...@gmail.com     Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 14:37
Josh Goldsborough       Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 23:09
Gourav Sengupta       Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Thu, 22 Mar, 00:27
Shiyuan strange behavior of joining dataframes Wed, 21 Mar, 04:58
Shiyuan   Re: strange behavior of joining dataframes Sat, 24 Mar, 00:29
Aakash Basu Wait for 30 seconds before terminating Spark Streaming Wed, 21 Mar, 10:41
Jorge Machado HadoopDelegationTokenProvider Wed, 21 Mar, 14:32
Marcelo Vanzin   Re: HadoopDelegationTokenProvider Wed, 21 Mar, 16:28
Priyank Shrivastava [Structured Streaming] Application Updates in Production Wed, 21 Mar, 18:56
Tathagata Das   Re: [Structured Streaming] Application Updates in Production Thu, 22 Mar, 00:27
Tathagata Das     Re: [Structured Streaming] Application Updates in Production Fri, 23 Mar, 03:00
kant kodali Is there a mutable dataframe spark structured streaming 2.3.0? Thu, 22 Mar, 02:20
Aakash Basu   Re: Is there a mutable dataframe spark structured streaming 2.3.0? Thu, 22 Mar, 09:07
Jorge Machado     Re: Is there a mutable dataframe spark structured streaming 2.3.0? Thu, 22 Mar, 09:08
kant kodali       Re: Is there a mutable dataframe spark structured streaming 2.3.0? Thu, 22 Mar, 14:48
kant kodali         Re: Is there a mutable dataframe spark structured streaming 2.3.0? Sat, 24 Mar, 03:38
Rohit Karlupia Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 03:36
☼ R Nair (रविशंकर नायर)   Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 04:33
Holden Karau     Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 04:38
Fawze Abujaber       Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 04:48
Rohit Karlupia         Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 14:22
Shmuel Blitz           Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 15:05
Shmuel Blitz             Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 22:43
Fawze Abujaber               Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Fri, 23 Mar, 04:33
Fawze Abujaber                 Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Fri, 23 Mar, 16:24
Shmuel Blitz                   Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Sun, 25 Mar, 07:34
Fawze Abujaber                     Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Sun, 25 Mar, 07:40
Shmuel Blitz                       Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Sun, 25 Mar, 07:55
Rohit Karlupia                         Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Sun, 25 Mar, 09:46
Shmuel Blitz                           Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Sun, 25 Mar, 20:08
Rohit Karlupia                             Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Mon, 26 Mar, 10:35
Fawze Abujaber                               Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Tue, 27 Mar, 02:23
Rohit Karlupia                                 Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Tue, 27 Mar, 03:45
Fawze Abujaber                                   Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Tue, 27 Mar, 03:50
Rohit Karlupia                                     Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Tue, 27 Mar, 06:04
nayan sharma Spark Druid Ingestion Thu, 22 Mar, 07:21
Jorge Machado   Re: Spark Druid Ingestion Thu, 22 Mar, 07:24
nayan sharma     Re: Spark Druid Ingestion Thu, 22 Mar, 07:37
Kamalanathan Venkatesan Need config params while doing rdd.foreach or map Thu, 22 Mar, 09:18
ayan guha   Re: Need config params while doing rdd.foreach or map Thu, 22 Mar, 11:45
KhajaAsmath Mohammed Transaction Examplefor spark streaming in Spark2.2 Fri, 23 Mar, 01:59
Yong Zhang java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to Case class Fri, 23 Mar, 02:08
Yong Zhang   Re: java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to Case class Fri, 23 Mar, 17:34
Apache Spark Structured Streaming - Kafka Consumer cannot fetch records for offset exception
M Singh   Apache Spark Structured Streaming - Kafka Consumer cannot fetch records for offset exception Fri, 23 Mar, 03:06
Tathagata Das     Re: Apache Spark Structured Streaming - Kafka Consumer cannot fetch records for offset exception Fri, 23 Mar, 03:15
Apache Spark Structured Streaming - Kafka Streaming - Option to ignore checkpoint
M Singh   Apache Spark Structured Streaming - Kafka Streaming - Option to ignore checkpoint Fri, 23 Mar, 04:11
Aakash Basu Structured Streaming Spark 2.3 Query Fri, 23 Mar, 05:45
Bowden, Chris   Re: Structured Streaming Spark 2.3 Query Fri, 23 Mar, 06:10
Jorge Machado Spark and Accumulo Delegation tokens Fri, 23 Mar, 06:29
Saisai Shao   Re: Spark and Accumulo Delegation tokens Fri, 23 Mar, 06:38
Jorge Machado     Re: Spark and Accumulo Delegation tokens Fri, 23 Mar, 07:10
Saisai Shao       Re: Spark and Accumulo Delegation tokens Fri, 23 Mar, 11:24
Donni Khan Calculate co-occurring terms Fri, 23 Mar, 07:57
Donni Khan   Re: Calculate co-occurring terms Tue, 27 Mar, 07:50
Michael Shtelma Using CBO on Spark 2.3 with analyzed hive tables Fri, 23 Mar, 09:20
Takeshi Yamamuro   Re: Using CBO on Spark 2.3 with analyzed hive tables Fri, 23 Mar, 12:51
Michael Shtelma     Re: Using CBO on Spark 2.3 with analyzed hive tables Fri, 23 Mar, 16:23
Takeshi Yamamuro       Re: Using CBO on Spark 2.3 with analyzed hive tables Fri, 23 Mar, 23:39
Michael Shtelma         Re: Using CBO on Spark 2.3 with analyzed hive tables Tue, 27 Mar, 13:57
Stefano Pettini [Spark Core] Getting the number of stages a job is made of Fri, 23 Mar, 11:20
崔苗 how to use lit() in spark-java Fri, 23 Mar, 12:32
Anil Langote   Re: how to use lit() in spark-java Fri, 23 Mar, 13:33
Anthony, Olufemi   Re: how to use lit() in spark-java Fri, 23 Mar, 13:47
Stefano Pettini [Spark Core] details of persisting RDDs Fri, 23 Mar, 15:53
Message list« Previous · 1 · 2 · 3 · 4 · 5 · Next »Thread · Author · Date
Box list
Dec 202128
Nov 2021153
Oct 202194
Sep 2021126
Aug 2021171
Jul 2021158
Jun 2021179
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137