spark-user mailing list archives: July 2018

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · Next »Thread · Author · Date
Dhaval Modi How to stop streaming jobs Sun, 15 Jul, 09:54
Re: Stopping StreamingContext
Dhaval Modi   Re: Stopping StreamingContext Sun, 15 Jul, 09:59
Re: Properly stop applications or jobs within the application
Dhaval Modi   Re: Properly stop applications or jobs within the application Sun, 15 Jul, 10:01
Re: Stopping a Spark Streaming Context gracefully
Dhaval Modi   Re: Stopping a Spark Streaming Context gracefully Sun, 15 Jul, 10:03
Maximiliano Patricio Méndez Security in pyspark using extensions Sun, 15 Jul, 23:16
Bryan Jeffrey Heap Memory in Spark 2.3.0 Mon, 16 Jul, 18:43
Imran Rashid   Re: Heap Memory in Spark 2.3.0 Tue, 17 Jul, 19:59
Gautam Singaraju Running Production ML Pipelines Mon, 16 Jul, 21:11
Shmuel Blitz   Re: Running Production ML Pipelines Tue, 17 Jul, 09:04
Mich Talebzadeh Performance considerations, Using microservices for ZooKeeper & Kafka in Spark Streaming Mon, 16 Jul, 22:46
Aakash Basu Query on Profiling Spark Code Tue, 17 Jul, 07:10
Patil, Prashasth   RE: Query on Profiling Spark Code Tue, 31 Jul, 07:36
Aakash Basu     Re: Query on Profiling Spark Code Tue, 31 Jul, 08:53
sathich joining streams from multiple kafka clusters Tue, 17 Jul, 19:08
Nirav Patel Dataframe from partitioned parquet table missing partition columns from schema Tue, 17 Jul, 22:48
Nirav Patel   Re: Dataframe from partitioned parquet table missing partition columns from schema Tue, 17 Jul, 22:55
Chethan Spark (Scala) Streaming [Convert rdd [org.bson.document] - > dataframe] Wed, 18 Jul, 18:23
vincent gromakowski   Re: Spark (Scala) Streaming [Convert rdd [org.bson.document] - > dataframe] Wed, 18 Jul, 18:58
Re: [STRUCTURED STREAM] Join static dataset in state function (flatMapGroupsWithState)
Gerard Maas   Re: [STRUCTURED STREAM] Join static dataset in state function (flatMapGroupsWithState) Thu, 19 Jul, 13:19
Christiaan Ras     Re: [STRUCTURED STREAM] Join static dataset in state function (flatMapGroupsWithState) Thu, 19 Jul, 19:23
Abhishek Tripathi [Spark Structured Streaming on K8S]: Debug - File handles/descriptor (unix pipe) leaking Thu, 19 Jul, 14:02
Abhishek Tripathi   Re: [Spark Structured Streaming on K8S]: Debug - File handles/descriptor (unix pipe) leaking Mon, 23 Jul, 14:10
Yuval.Itzchakov     Re: [Spark Structured Streaming on K8S]: Debug - File handles/descriptor (unix pipe) leaking Wed, 25 Jul, 13:59
Patrick McCarthy Arrow type issue with Pandas UDF Thu, 19 Jul, 14:07
Bryan Cutler   Re: Arrow type issue with Pandas UDF Fri, 20 Jul, 04:45
Patrick McCarthy     Re: Arrow type issue with Pandas UDF Tue, 24 Jul, 18:11
Gourav Sengupta       Re: Arrow type issue with Pandas UDF Tue, 24 Jul, 18:20
Deepu Raj Compute /Storage Calculation Thu, 19 Jul, 17:39
Nirav Patel Mulitple joins with same Dataframe throws Ambiguous reference error Thu, 19 Jul, 21:11
Nirav Patel Re: Mulitple joins with same Dataframe throws AnalysisException: resolved attribute(s) Thu, 19 Jul, 21:13
kanth...@gmail.com   Re: Mulitple joins with same Dataframe throws AnalysisException: resolved attribute(s) Fri, 20 Jul, 01:49
Joel D     Re: Mulitple joins with same Dataframe throws AnalysisException: resolved attribute(s) Fri, 20 Jul, 01:52
Parquet
amin mohebbi   Parquet Fri, 20 Jul, 01:34
Muthu Jayakumar     Re: Parquet Fri, 20 Jul, 14:16
Garlapati, Suryanarayana (Nokia - IN/Bangalore) Query on Spark Hive with kerberos Enabled on Kubernetes Fri, 20 Jul, 15:06
Garlapati, Suryanarayana (Nokia - IN/Bangalore)   RE: Query on Spark Hive with kerberos Enabled on Kubernetes Sat, 21 Jul, 13:20
Garlapati, Suryanarayana (Nokia - IN/Bangalore)     RE: Query on Spark Hive with kerberos Enabled on Kubernetes Tue, 24 Jul, 02:36
Daniel Mateus Pires [SPARK-SQL] Reading JSON column as a DataFrame and keeping partitioning information Fri, 20 Jul, 16:56
fmilano Filtering DataType column with Timestamp Fri, 20 Jul, 17:56
Bryan Jeffrey Spark-Streaming-Kafka10_2.11 on Spark 2.3 Fri, 20 Jul, 18:55
Bryan Jeffrey   Re: Spark-Streaming-Kafka10_2.11 on Spark 2.3 Fri, 20 Jul, 19:45
Cody Koeninger     Re: Spark-Streaming-Kafka10_2.11 on Spark 2.3 Fri, 20 Jul, 21:29
Bryan Jeffrey     Re: Spark-Streaming-Kafka10_2.11 on Spark 2.3 Fri, 20 Jul, 21:34
Uğur Sopaoğlu Apache Spark Cluster Mon, 23 Jul, 12:21
John, Vishal (Agoda) Where can I read the Kafka offsets in SparkSQL application Tue, 24 Jul, 11:30
Gourav Sengupta   Re: Where can I read the Kafka offsets in SparkSQL application Tue, 24 Jul, 11:36
dddaaa How to read json data from kafka and store to hdfs with spark structued streaming? Tue, 24 Jul, 22:38
Tathagata Das   Re: How to read json data from kafka and store to hdfs with spark structued streaming? Fri, 27 Jul, 00:38
dddaaa     Re: How to read json data from kafka and store to hdfs with spark structued streaming? Fri, 27 Jul, 08:40
Arbab Khalil       Re: How to read json data from kafka and store to hdfs with spark structued streaming? Fri, 27 Jul, 10:33
dddaaa         Re: How to read json data from kafka and store to hdfs with spark structued streaming? Fri, 27 Jul, 10:39
Arbab Khalil           Re: How to read json data from kafka and store to hdfs with spark structued streaming? Fri, 27 Jul, 23:59
Holden Karau Live Code Reviews, Coding, and Dev Tools Wed, 25 Jul, 03:54
康逸之 ***UNCHECKED*** How dose spark streaming program (written with scala)call python file Wed, 25 Jul, 07:34
康逸之   How dose spark streaming program call python file Wed, 25 Jul, 07:38
sridhararao mutluri ***UNCHECKED*** UNSUBSCRIBE Wed, 25 Jul, 07:40
Elior Malul Bug in Window Function Wed, 25 Jul, 08:04
Jacek Laskowski   Re: Bug in Window Function Wed, 25 Jul, 10:47
Biplob Biswas Backpressure initial rate not working Wed, 25 Jul, 14:23
Biplob Biswas   Re: Backpressure initial rate not working Thu, 26 Jul, 11:21
Todd Nist     Re: Backpressure initial rate not working Thu, 26 Jul, 12:33
Biplob Biswas       Re: Backpressure initial rate not working Thu, 26 Jul, 12:45
Todd Nist         Re: Backpressure initial rate not working Thu, 26 Jul, 13:03
Biplob Biswas           Re: Backpressure initial rate not working Thu, 26 Jul, 16:19
Hichame El Khalfi Use Arrow instead of Pickle without pandas_udf Wed, 25 Jul, 20:27
Holden Karau   Re: Use Arrow instead of Pickle without pandas_udf Wed, 25 Jul, 20:41
Hichame El Khalfi     Re: Use Arrow instead of Pickle without pandas_udf Wed, 25 Jul, 22:36
Bryan Cutler       Re: Use Arrow instead of Pickle without pandas_udf Mon, 30 Jul, 22:40
Hichame El Khalfi       Re: Use Arrow instead of Pickle without pandas_udf Tue, 31 Jul, 11:42
nookala Split a row into multiple rows Java Thu, 26 Jul, 02:03
Patil, Prashasth   RE: Split a row into multiple rows Java Tue, 31 Jul, 07:10
Jonathan Apple Exceptions with simplest Structured Streaming example Thu, 26 Jul, 12:41
Tathagata Das   Re: Exceptions with simplest Structured Streaming example Thu, 26 Jul, 22:49
Jonathan Apple     Re: Exceptions with simplest Structured Streaming example Fri, 27 Jul, 00:38
Vitaliy Pisarev Optimizing a join with bucketing Thu, 26 Jul, 15:36
utkarsh rathor Question of spark streaming Fri, 27 Jul, 12:14
Arun Mahadevan   Re: Question of spark streaming Fri, 27 Jul, 17:58
dineshdharme Iterative rdd union + reduceByKey operations on small dataset leads to "No space left on device" error on account of lot of shuffle spill. Fri, 27 Jul, 12:52
Vadim Semenov   Re: Iterative rdd union + reduceByKey operations on small dataset leads to "No space left on device" error on account of lot of shuffle spill. Fri, 27 Jul, 18:24
Dinesh Dharme     Re: Iterative rdd union + reduceByKey operations on small dataset leads to "No space left on device" error on account of lot of shuffle spill. Fri, 27 Jul, 19:46
kant kodali How to Create one DB connection per executor and close it after the job is done? Sat, 28 Jul, 06:33
Patrick McGloin   Re: How to Create one DB connection per executor and close it after the job is done? Mon, 30 Jul, 14:37
kant kodali     Re: How to Create one DB connection per executor and close it after the job is done? Mon, 30 Jul, 17:31
Vadim Semenov       Re: How to Create one DB connection per executor and close it after the job is done? Mon, 30 Jul, 17:57
karan alang modeling timestamp in Avro messages (read using Spark Structured Streaming) Sun, 29 Jul, 16:53
bsikander [SparkContext] will application immediately stop after sc.stop()? Sun, 29 Jul, 18:56
Message list« Previous · 1 · 2 · 3 · 4 · Next »Thread · Author · Date
Box list
Dec 202113
Nov 2021153
Oct 202194
Sep 2021126
Aug 2021171
Jul 2021158
Jun 2021179
May 2021187
Apr 2021267
Mar 2021346
Feb 2021166
Jan 2021242
Dec 2020203
Nov 2020147
Oct 2020236
Sep 2020136
Aug 2020166
Jul 2020248
Jun 2020263
May 2020282
Apr 2020335
Mar 2020232
Feb 2020136
Jan 2020141
Dec 2019138
Nov 2019125
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137