spark-user mailing list archives: September 2018

Site index · List index
Message list1 · 2 · 3 · Next »Thread · Author · Date
Uğur Sopaoğlu Apache Spark and Airflow connection Mon, 24 Sep, 12:26
Guillermo Ortiz Fernández deploy-mode cluster. FileNotFoundException Wed, 05 Sep, 09:11
Guillermo Ortiz Fernández Re: deploy-mode cluster. FileNotFoundException Wed, 05 Sep, 09:17
Guillermo Ortiz Fernández Trying to improve performance of the driver. Thu, 13 Sep, 15:48
阎志涛 How to retreive data from nested json use dataframe Sat, 08 Sep, 14:38
Jörn Franke Re: Drawing Big Data tech diagrams using Pen Tablets Wed, 12 Sep, 18:38
Jörn Franke Re: Time-Series Forecasting Wed, 19 Sep, 16:55
Jörn Franke Re: Use Shared Variable in PySpark Executors Sat, 22 Sep, 15:39
Jörn Franke Re: How to access line fileName in loading file using the textFile method Mon, 24 Sep, 12:56
Aakash Basu XGBoost Not distributing on cluster having more than 1 worker Thu, 06 Sep, 10:05
Aakash Basu Re: XGBoost Not distributing on cluster having more than 1 worker Thu, 06 Sep, 11:16
Aakash Basu [Help] Set nThread in Spark cluster Wed, 12 Sep, 08:40
Aakash Basu Local vs Cluster Fri, 14 Sep, 08:21
Aakash Basu Re: Should python-2 be supported in Spark 3.0? Mon, 17 Sep, 01:31
Aakash Basu Re: Time-Series Forecasting Thu, 20 Sep, 04:22
Akash Mishra Re: Time-Series Forecasting Thu, 20 Sep, 08:35
Alchemist How to repartition Spark DStream Kafka ConsumerRecord RDD. Fri, 28 Sep, 09:13
Aleksander Eskilson [ML] Setting Non-Transform Params for a Pipeline & PipelineModel Wed, 05 Sep, 14:16
Alessandro Solimando Re: Need to convert Dataset<Row> to HashMap<String, String> Fri, 28 Sep, 07:22
Alessandro Solimando Re: Need to convert Dataset<Row> to HashMap<String, String> Fri, 28 Sep, 07:48
Ambi, Aniket Spark Use Case Analysis Fri, 21 Sep, 09:00
Anu B Nair Unsubscribe Thu, 06 Sep, 07:03
Apostolos N. Papadopoulos Re: Error in show() Fri, 07 Sep, 06:52
Apostolos N. Papadopoulos Re: Spark job's driver programe consums too much memory Fri, 07 Sep, 14:15
Apostolos N. Papadopoulos Re: Spark job's driver programe consums too much memory Fri, 07 Sep, 14:46
Apostolos N. Papadopoulos Re: Local vs Cluster Fri, 14 Sep, 08:38
Arko Provo Mukherjee Encoder for JValue Wed, 19 Sep, 01:05
Arko Provo Mukherjee Re: Encoder for JValue Wed, 19 Sep, 18:13
Ashic Mahtab spark and STS tokens (Federation Tokens) Wed, 26 Sep, 12:48
AssafMendelson Data source V2 in spark 2.4.0 Thu, 27 Sep, 13:10
Ben_W Run Spark on Java 10 Sat, 29 Sep, 02:57
Bibudh Lahiri Unsubscribe Fri, 07 Sep, 10:04
Chetan Khatri How to do efficient self join with Spark-SQL and Scala Fri, 21 Sep, 19:37
Christopher Carney Fwd: Spark 2.3.1: k8s driver pods stuck in Initializing state Wed, 26 Sep, 18:11
Christopher Carney Spark 2.3.1: k8s driver pods stuck in Initializing state Wed, 26 Sep, 21:14
Dan Osipov Required fields in Parquet Tue, 18 Sep, 21:27
Daniele Foroni [SparkSQL] Count Distinct issue Fri, 14 Sep, 18:54
David Hesson Seeing a framework registration loop with Spark 2.3.1 on DCOS 1.10.0 Wed, 05 Sep, 00:01
Debajyoti Roy Given events with start and end times, how to count the number of simultaneous events using Spark? Wed, 26 Sep, 16:32
Deepak Sharma Re: getting error: value toDF is not a member of Seq[columns] Wed, 05 Sep, 09:09
Dillon Dukek Re: DirectFileOutputCommitter in Spark 2.3.1 Wed, 19 Sep, 18:17
Dillon Dukek Re: [Spark SQL]: Java Spark Classes With Attributes of Type Set In Datasets Tue, 25 Sep, 22:00
Dongjoon Hyun Re: Metastore problem on Spark2.3 with Hive3.0 Tue, 18 Sep, 03:13
Erik Erlandson Should python-2 be supported in Spark 3.0? Sat, 15 Sep, 18:09
Erik Erlandson Re: Should python-2 be supported in Spark 3.0? Sat, 15 Sep, 18:46
Erik Erlandson Re: Should python-2 be supported in Spark 3.0? Mon, 17 Sep, 17:57
Erik Erlandson Re: Should python-2 be supported in Spark 3.0? Mon, 17 Sep, 20:11
Erik Erlandson Re: Should python-2 be supported in Spark 3.0? Tue, 18 Sep, 01:53
Felix Cheung Re: Should python-2 be supported in Spark 3.0? Mon, 17 Sep, 01:18
Felix Cheung Re: spark.lapply Wed, 26 Sep, 12:57
Felix Cheung Re: can Spark 2.4 work on JDK 11? Sat, 29 Sep, 22:52
Femi Anthony Re: [External Sender] Re: How to make pyspark use custom python? Fri, 07 Sep, 02:54
Femi Anthony Re: [External Sender] How to debug Spark job Fri, 07 Sep, 10:32
Fiona Chow Unsubscribe Fri, 07 Sep, 10:05
Garlapati, Suryanarayana (Nokia - IN/Bangalore) RE: [K8S] Driver and Executor Logging Sat, 08 Sep, 16:03
Garlapati, Suryanarayana (Nokia - IN/Bangalore) Support STS to run in k8s deployment with spark deployment mode as cluster Sun, 16 Sep, 04:45
Garlapati, Suryanarayana (Nokia - IN/Bangalore) RE: Support STS to run in k8s deployment with spark deployment mode as cluster Sun, 16 Sep, 05:58
Garlapati, Suryanarayana (Nokia - IN/Bangalore) Python kubernetes spark 2.4 branch Tue, 25 Sep, 17:23
Garlapati, Suryanarayana (Nokia - IN/Bangalore) RE: Python kubernetes spark 2.4 branch Wed, 26 Sep, 03:43
Garlapati, Suryanarayana (Nokia - IN/Bangalore) RE: Python kubernetes spark 2.4 branch Wed, 26 Sep, 17:43
Gokula Krishnan D [Spark SQL] why spark sql hash() are returns the same hash value though the keys/expr are not same Wed, 26 Sep, 01:57
Gokula Krishnan D Re: [Spark SQL] why spark sql hash() are returns the same hash value though the keys/expr are not same Fri, 28 Sep, 15:39
Gourav Sengupta Re: Drawing Big Data tech diagrams using Pen Tablets Wed, 12 Sep, 17:34
Gourav Sengupta Re: Drawing Big Data tech diagrams using Pen Tablets Wed, 12 Sep, 20:46
Gourav Sengupta Re: Time-Series Forecasting Thu, 20 Sep, 09:12
Gourav Sengupta Re: Live Streamed Code Review today at 11am Pacific Fri, 21 Sep, 07:53
Guillermo Ortiz DAGScheduler in SparkStreaming Fri, 14 Sep, 10:35
Harel Gliksman Driver OutOfMemoryError in MapOutputTracker$.serializeMapStatuses for 40 TB shuffle. Fri, 07 Sep, 14:34
Harel Gliksman Re: Driver OutOfMemoryError in MapOutputTracker$.serializeMapStatuses for 40 TB shuffle. Fri, 07 Sep, 15:40
Hasunie Adikari Unsubscribe Thu, 27 Sep, 10:47
Holden Karau Re: Live Streamed Code Review today at 11am Pacific Fri, 21 Sep, 06:40
Hugh Hyndman DataSourceReader and SupportPushDownFilters for Short types Sat, 08 Sep, 23:17
Hugh Hyndman PushDown Filter Not Reset Sun, 09 Sep, 10:56
Hyukjin Kwon Re: How to make pyspark use custom python? Thu, 06 Sep, 06:58
Hyukjin Kwon Re: Should python-2 be supported in Spark 3.0? Mon, 17 Sep, 02:00
Iryna Kharaborkina Unsubscribe Wed, 26 Sep, 08:48
James Starks How to debug Spark job Fri, 07 Sep, 09:47
James Starks Re: [External Sender] How to debug Spark job Fri, 07 Sep, 13:05
James Starks Spark job's driver programe consums too much memory Fri, 07 Sep, 14:04
James Starks Re: Spark job's driver programe consums too much memory Fri, 07 Sep, 14:39
James Starks Re: Spark job's driver programe consums too much memory Fri, 07 Sep, 15:09
Jan Brabec (janbrabe) Re: Question about Spark cluster memory usage monitoring Mon, 24 Sep, 06:57
Jatin Puri Lightweight pipeline execution for single eow Fri, 21 Sep, 16:58
Jatin Puri Re: Lightweight pipeline execution for single eow Wed, 26 Sep, 08:44
Joel D Text from pdf spark Fri, 28 Sep, 17:10
Joel D Re: Text from pdf spark Fri, 28 Sep, 17:35
Jonas Shomorony Python Dependencies Issue on EMR Fri, 14 Sep, 02:08
Jonas Shomorony Re: Python Dependencies Issue on EMR Fri, 21 Sep, 00:21
Jungtaek Lim Re: getting error: value toDF is not a member of Seq[columns] Wed, 05 Sep, 08:17
Jungtaek Lim Re: getting error: value toDF is not a member of Seq[columns] Wed, 05 Sep, 08:30
Jungtaek Lim Re: getting error: value toDF is not a member of Seq[columns] Wed, 05 Sep, 08:32
Jungtaek Lim Re: getting error: value toDF is not a member of Seq[columns] Wed, 05 Sep, 08:41
Jungtaek Lim Re: getting error: value toDF is not a member of Seq[columns] Thu, 06 Sep, 09:10
Jungtaek Lim Looking for some feedbacks on proposal - native support of session window Thu, 27 Sep, 23:45
Jungtaek Lim Re: Watermarking without aggregation with Structured Streaming Sun, 30 Sep, 08:41
Junior Alvarez spark.lapply Wed, 26 Sep, 11:31
Junior Alvarez RE: spark.lapply Thu, 27 Sep, 09:29
Kuttaiah Robin Spark FlatMapGroupsWithStateFunction throws cannot resolve 'named_struct()' due to data type mismatch 'SerializeFromObject" Tue, 18 Sep, 04:12
Kuttaiah Robin Creating spark Row from database values Wed, 26 Sep, 12:01
Kuttaiah Robin Re: Creating spark Row from database values Wed, 26 Sep, 13:59
Message list1 · 2 · 3 · Next »Thread · Author · Date
Box list
May 2019280
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137