spark-user mailing list archives: March 2018

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · 5 · Next »Thread · Author · Date
Jacek Laskowski NPE in Subexpression Elimination optimization Fri, 16 Mar, 14:56
klrmowse Spark 2.x Core: .setMaster(local[*]) output is different from spark-submit Fri, 16 Mar, 15:12
Aakash Basu Re: Multiple Kafka Spark Streaming Dataframe Join query Fri, 16 Mar, 16:12
Trevor Grant GOTO Chicago Talk / Discount Fri, 16 Mar, 16:51
Cody Koeninger Re: is it possible to use Spark 2.3.0 along with Kafka 0.9.0.1? Fri, 16 Mar, 19:08
Christopher Piggott Custom metrics sink Fri, 16 Mar, 20:09
Koert Kuipers change spark default for a setting without overriding user Fri, 16 Mar, 22:43
Felix Cheung Re: Custom metrics sink Sat, 17 Mar, 00:17
Silvio Fiorito Re: Custom metrics sink Sat, 17 Mar, 01:39
Bahubali Jain Dataframe size using RDDStorageInfo objects Sat, 17 Mar, 06:05
Serega Sheypak Re: how "hour" function in Spark SQL is supposed to work? Sat, 17 Mar, 12:09
Serega Sheypak Append more files to existing partitioned data Sat, 17 Mar, 12:18
Denis Bolshakov Re: Append more files to existing partitioned data Sat, 17 Mar, 13:15
Serega Sheypak Re: Append more files to existing partitioned data Sat, 17 Mar, 17:46
klrmowse Re: Spark 2.x Core: .setMaster(local[*]) output is different from spark-submit Sat, 17 Mar, 18:48
Vitaliy Pisarev Accessing a file that was passed via --files to spark submit Sun, 18 Mar, 08:06
Denis Bolshakov Re: Append more files to existing partitioned data Sun, 18 Mar, 08:30
Mahender Sarangam Dynamic Key JSON Parsing Sun, 18 Mar, 10:12
Mahender Sarangam Scala - Spark for beginners Sun, 18 Mar, 10:15
Gerard Maas Re: Scala - Spark for beginners Sun, 18 Mar, 15:30
Serega Sheypak Re: Append more files to existing partitioned data Sun, 18 Mar, 15:42
CPC parquet late column materialization Sun, 18 Mar, 17:02
nguyen duc Tuan Re: parquet late column materialization Sun, 18 Mar, 17:09
Jacek Laskowski Re: NPE in Subexpression Elimination optimization Sun, 18 Mar, 17:21
CPC Re: parquet late column materialization Sun, 18 Mar, 17:52
Serega Sheypak Run spark 2.2 on yarn as usual java application Sun, 18 Mar, 23:19
nguyen duc Tuan Re: parquet late column materialization Sun, 18 Mar, 23:56
Gurusamy Thirupathy Hive to Oracle using Spark - Type(Date) conversion issue Mon, 19 Mar, 02:34
Deepak Sharma Re: Hive to Oracle using Spark - Type(Date) conversion issue Mon, 19 Mar, 02:36
ayan guha Re: Hive to Oracle using Spark - Type(Date) conversion issue Mon, 19 Mar, 03:04
Hyukjin Kwon Re: [PySpark SQL] sql function to_date and to_timestamp return the same data type Mon, 19 Mar, 05:47
Debabrata Ghosh Calling Pyspark functions in parallel Mon, 19 Mar, 05:54
Jacek Laskowski Re: Run spark 2.2 on yarn as usual java application Mon, 19 Mar, 06:16
Serega Sheypak Re: Run spark 2.2 on yarn as usual java application Mon, 19 Mar, 11:58
Jörn Franke Re: Run spark 2.2 on yarn as usual java application Mon, 19 Mar, 12:41
Serega Sheypak Re: Run spark 2.2 on yarn as usual java application Mon, 19 Mar, 13:02
Jules Damji Re: Calling Pyspark functions in parallel Mon, 19 Mar, 13:46
Artem Moskvin [Spark Structured Streaming, Spark 2.3.0] Calling current_timestamp() function within a streaming dataframe results in dataType error Mon, 19 Mar, 14:08
Shahab Yunus Warnings on data insert into Hive Table using PySpark Mon, 19 Mar, 15:41
Debabrata Ghosh Re: Calling Pyspark functions in parallel Mon, 19 Mar, 16:43
Michael Shtelma Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 19 Mar, 16:59
Keith Chapman Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 19 Mar, 17:07
Marcelo Vanzin Re: Accessing a file that was passed via --files to spark submit Mon, 19 Mar, 17:23
Michael Shtelma Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 19 Mar, 17:29
Keith Chapman Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 19 Mar, 17:38
Geoff Von Allmen Structured Streaming: distinct (Spark 2.2) Mon, 19 Mar, 19:04
Burak Yavuz Re: Structured Streaming: distinct (Spark 2.2) Mon, 19 Mar, 19:18
Michael Shtelma Re: Running out of space on /tmp file system while running spark job on yarn because of size of blockmgr folder Mon, 19 Mar, 20:05
kant kodali select count * doesnt seem to respect update mode in Kafka Structured Streaming? Mon, 19 Mar, 20:35
Geoff Von Allmen Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Mon, 19 Mar, 20:57
kant kodali Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Tue, 20 Mar, 00:27
Gurusamy Thirupathy Re: Hive to Oracle using Spark - Type(Date) conversion issue Tue, 20 Mar, 02:45
Jörn Franke Re: Hive to Oracle using Spark - Type(Date) conversion issue Tue, 20 Mar, 07:17
1427357...@qq.com the meaining of "samplePointsPerPartitionHint" in RangePartitioner Tue, 20 Mar, 07:59
Geoff Von Allmen Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Tue, 20 Mar, 13:09
SNEHASISH DUTTA Access Table with Spark Dataframe Tue, 20 Mar, 13:17
dcam Re: [Structured Streaming] Commit protocol to move temp files to dest path only when complete, with code Tue, 20 Mar, 14:09
lucas-vsco [Structured Streaming] Query Metrics to MetricsSink Tue, 20 Mar, 14:36
hemant singh Re: Access Table with Spark Dataframe Tue, 20 Mar, 15:09
lucas-vsco Re: [Structured Streaming] Query Metrics to MetricsSink Tue, 20 Mar, 15:11
Serega Sheypak Re: how "hour" function in Spark SQL is supposed to work? Tue, 20 Mar, 21:43
kant kodali Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Tue, 20 Mar, 21:58
Serega Sheypak Re: how "hour" function in Spark SQL is supposed to work? Tue, 20 Mar, 22:14
Michael Armbrust Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Wed, 21 Mar, 00:00
kant kodali Re: select count * doesnt seem to respect update mode in Kafka Structured Streaming? Wed, 21 Mar, 02:28
purna pradeep Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 02:47
Gurusamy Thirupathy Re: Hive to Oracle using Spark - Type(Date) conversion issue Wed, 21 Mar, 03:08
Yinan Li Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 03:50
Shiyuan strange behavior of joining dataframes Wed, 21 Mar, 04:58
Aakash Basu Re: Multiple Kafka Spark Streaming Dataframe Join query Wed, 21 Mar, 08:38
Aakash Basu Wait for 30 seconds before terminating Spark Streaming Wed, 21 Mar, 10:41
purna pradeep Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 12:01
Gourav Sengupta Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 14:32
Jorge Machado HadoopDelegationTokenProvider Wed, 21 Mar, 14:32
lucas.g...@gmail.com Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 14:37
Marcelo Vanzin Re: HadoopDelegationTokenProvider Wed, 21 Mar, 16:28
Priyank Shrivastava [Structured Streaming] Application Updates in Production Wed, 21 Mar, 18:56
Josh Goldsborough Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Wed, 21 Mar, 23:09
Tathagata Das Re: [Structured Streaming] Application Updates in Production Thu, 22 Mar, 00:27
Gourav Sengupta Re: Rest API for Spark2.3 submit on kubernetes(version 1.8.*) cluster Thu, 22 Mar, 00:27
kant kodali Is there a mutable dataframe spark structured streaming 2.3.0? Thu, 22 Mar, 02:20
Rohit Karlupia Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 03:36
☼ R Nair (रविशंकर नायर) Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 04:33
Holden Karau Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 04:38
Fawze Abujaber Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 04:48
nayan sharma Spark Druid Ingestion Thu, 22 Mar, 07:21
Jorge Machado Re: Spark Druid Ingestion Thu, 22 Mar, 07:24
nayan sharma Re: Spark Druid Ingestion Thu, 22 Mar, 07:37
Aakash Basu Re: Is there a mutable dataframe spark structured streaming 2.3.0? Thu, 22 Mar, 09:07
Jorge Machado Re: Is there a mutable dataframe spark structured streaming 2.3.0? Thu, 22 Mar, 09:08
Kamalanathan Venkatesan Need config params while doing rdd.foreach or map Thu, 22 Mar, 09:18
ayan guha Re: Need config params while doing rdd.foreach or map Thu, 22 Mar, 11:45
Rohit Karlupia Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 14:22
kant kodali Re: Is there a mutable dataframe spark structured streaming 2.3.0? Thu, 22 Mar, 14:48
Shmuel Blitz Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 15:05
Shmuel Blitz Re: Open sourcing Sparklens: Qubole's Spark Tuning Tool Thu, 22 Mar, 22:43
KhajaAsmath Mohammed Transaction Examplefor spark streaming in Spark2.2 Fri, 23 Mar, 01:59
Yong Zhang java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to Case class Fri, 23 Mar, 02:08
Tathagata Das Re: [Structured Streaming] Application Updates in Production Fri, 23 Mar, 03:00
M Singh Apache Spark Structured Streaming - Kafka Consumer cannot fetch records for offset exception Fri, 23 Mar, 03:06
Message list« Previous · 1 · 2 · 3 · 4 · 5 · Next »Thread · Author · Date
Box list
Jan 2020101
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137