spark-user mailing list archives: November 2016

Site index · List index
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · Next »Thread · Author · Date
Dave Jaffe Re: Running stress tests on spark cluster to avoid wild-goose chase later Tue, 15 Nov, 17:28
David Lauzon Tracking opened files by Spark application Fri, 25 Nov, 18:28
David Robison creating a javaRDD using newAPIHadoopFile and FixedLengthInputFormat Tue, 15 Nov, 13:44
David Robison Problem submitting a spark job using yarn-client as master Tue, 15 Nov, 21:45
David Robison RE: Problem submitting a spark job using yarn-client as master Wed, 16 Nov, 14:03
David Robison RE: submitting a spark job using yarn-client and getting NoClassDefFoundError: org/apache/spark/Logging Wed, 16 Nov, 20:05
David Robison newAPIHadoopFile throws a JsonMappingException: Infinite recursion (StackOverflowError) error Thu, 17 Nov, 15:11
Davies Liu Re: UDF with column value comparison fails with PySpark Thu, 10 Nov, 19:19
Debasish Ghosh outlier detection using StreamingKMeans Thu, 17 Nov, 14:03
Debasish Ghosh Re: using StreamingKMeans Sat, 19 Nov, 19:24
Debasish Ghosh Re: using StreamingKMeans Sat, 19 Nov, 23:29
Debasish Ghosh Re: using StreamingKMeans Sun, 20 Nov, 01:11
Deepak Sharma Re: Optimized way to use spark as db to hdfs etl Sat, 05 Nov, 16:12
Deepak Sharma Re: Possible DR solution Fri, 11 Nov, 17:11
Deepak Sharma Re: Possible DR solution Fri, 11 Nov, 17:14
Deepak Sharma Re: what is the optimized way to combine multiple dataframes into one dataframe ? Wed, 16 Nov, 08:01
Denis Bolshakov Re: Pasting into spark-shell doesn't work for Databricks example Tue, 22 Nov, 14:35
Denis Bolshakov Re: Pasting into spark-shell doesn't work for Databricks example Wed, 23 Nov, 06:58
Denny Lee Re: GraphFrame BFS Tue, 01 Nov, 15:50
Denny Lee Re: How do I convert a data frame to broadcast variable? Thu, 03 Nov, 15:59
Denny Lee Re: Newbie question - Best way to bootstrap with Spark Mon, 07 Nov, 07:46
Denny Lee Re: hope someone can recommend some books for me,a spark beginner Mon, 07 Nov, 07:53
Denny Lee Re: Spark app write too many small parquet files Mon, 28 Nov, 06:08
Devi P.V what is the optimized way to combine multiple dataframes into one dataframe ? Wed, 16 Nov, 07:05
Didac Gil [No Subject] Mon, 28 Nov, 21:55
Dirceu Semighini Filho Re: Writing parquet table using spark Wed, 16 Nov, 12:17
Dirceu Semighini Filho Re: Spark Streaming Data loss on failure to write BlockAdditionEvent failure to WAL Thu, 17 Nov, 13:50
Dirceu Semighini Filho Re: Spark Streaming Data loss on failure to write BlockAdditionEvent failure to WAL Thu, 17 Nov, 17:18
Divya Gehlot Re: Best practice for preprocessing feature with DataFrame Thu, 17 Nov, 04:23
Dmitry Dzhus subtractByKey modifes values in the source RDD Wed, 23 Nov, 16:15
Don Drake Re: sbt shenanigans for a Spark-based project Sun, 13 Nov, 22:52
Don Drake Re: sbt shenanigans for a Spark-based project Mon, 14 Nov, 23:13
Donald Matthews incomplete aggregation in a GROUP BY Thu, 03 Nov, 15:05
Drooghaag, Benoit (Nokia - BE) RE: CSV to parquet preserving partitioning Wed, 16 Nov, 13:53
Edden Burrow Any with S3 experience with Spark? Having ListBucket issues Wed, 16 Nov, 22:34
Elf Of Lothlorein Save a spark RDD to disk Tue, 08 Nov, 22:08
Elkhan Dadashov SparkLauncer 2.0.1 version working incosistently in yarn-client mode Sat, 05 Nov, 09:54
Elkhan Dadashov Re: SparkLauncer 2.0.1 version working incosistently in yarn-client mode Thu, 10 Nov, 08:31
Elkhan Dadashov appHandle.kill(), SparkSubmit Process, JVM questions related to SparkLauncher design and Spark Driver Fri, 11 Nov, 22:49
Elkhan Dadashov SparkDriver memory calculation mismatch Sat, 12 Nov, 02:18
Elkhan Dadashov Exception not failing Python applications (in yarn client mode) - SparkLauncher says app succeeded, where app actually has failed Sat, 12 Nov, 03:32
Elkhan Dadashov Re: SparkDriver memory calculation mismatch Sat, 12 Nov, 09:13
Elkhan Dadashov Re: SparkDriver memory calculation mismatch Sat, 12 Nov, 09:59
Elkhan Dadashov Re: Correct SparkLauncher usage Sat, 12 Nov, 10:26
Elkhan Dadashov Re: Does the delegator map task of SparkLauncher need to stay alive until Spark job finishes ? Wed, 16 Nov, 01:57
Elkhan Dadashov Re: Does the delegator map task of SparkLauncher need to stay alive until Spark job finishes ? Wed, 16 Nov, 02:33
Erwan ALLAIN Application config management Wed, 09 Nov, 11:06
Erwan ALLAIN How to use logback Mon, 28 Nov, 14:00
Esa Heikkinen Simple "state machine" functionality using Scala or Python Tue, 15 Nov, 09:43
Fanjin Zeng How to avoid unnecessary spark starkups on every request? Wed, 02 Nov, 07:34
Felix Cheung Re: Issue Running sparkR on YARN Thu, 10 Nov, 00:54
Felix Cheung Re: Strongly Connected Components Fri, 11 Nov, 03:49
Felix Cheung Re: How to propagate R_LIBS to sparkr executors Fri, 18 Nov, 02:15
Felix Cheung Re: PySpark to remote cluster Wed, 30 Nov, 23:43
Femi Anthony Reading csv files with quoted fields containing embedded commas Sat, 05 Nov, 21:58
Femi Anthony Re: Reading csv files with quoted fields containing embedded commas Mon, 07 Nov, 02:28
Ganesh VectorUDT and ml.Vector Mon, 07 Nov, 13:25
Georg Heiler Fill na with last value Thu, 17 Nov, 16:36
Georg Heiler Re: build models in parallel Tue, 29 Nov, 16:59
Gerard Casey GraphX and Public Transport Shortest Paths Tue, 08 Nov, 20:12
Gerard Casey RDD to HDFS - Kerberos - authentication error - RetryInvocationHandler Fri, 11 Nov, 17:48
Gerard Maas [StackOverflow] Size exceeds Integer.MAX_VALUE When Joining 2 Large DFs Fri, 25 Nov, 12:05
Gmail Re: Third party library Sun, 27 Nov, 02:44
Gourav Sengupta Re: Very long pause/hang at end of execution Sun, 06 Nov, 19:34
Gourav Sengupta SPARK 2.0 CSV exports (https://issues.apache.org/jira/browse/SPARK-16893) Wed, 30 Nov, 18:19
Gourav Sengupta Re: Can't read tables written in Spark 2.1 in Spark 2.0 (and earlier) Wed, 30 Nov, 18:30
Haig Didizian how to write a substring search efficiently? Tue, 08 Nov, 13:36
Han-Cheol Cho null values returned by max() over a window function Tue, 29 Nov, 03:57
Hao Ren [Spark Streaming] map and window operation on DStream only process one batch Tue, 22 Nov, 13:48
Haopu Wang expected behavior of Kafka dynamic topic subscription Fri, 04 Nov, 02:43
Haopu Wang InvalidClassException when load KafkaDirectStream from checkpoint (Spark 2.0.0) Fri, 04 Nov, 09:23
Haopu Wang RE: expected behavior of Kafka dynamic topic subscription Mon, 07 Nov, 01:31
Haopu Wang RE: InvalidClassException when load KafkaDirectStream from checkpoint (Spark 2.0.0) Tue, 08 Nov, 08:06
Haopu Wang Kafka stream offset management question Tue, 08 Nov, 08:21
Hitesh Goyal if conditions Mon, 28 Nov, 04:45
Hitesh Goyal RE: if conditions Mon, 28 Nov, 06:12
Hitesh Goyal time to run Spark SQL query Mon, 28 Nov, 12:41
Hoang Bao Thien Re: Kafka segmentation Thu, 17 Nov, 09:18
Hoang Bao Thien Re: Kafka segmentation Thu, 17 Nov, 18:48
Hoang Bao Thien Re: Kafka segmentation Thu, 17 Nov, 18:53
Holden Karau Re: java.lang.ClassNotFoundException: org.apache.spark.sql.SparkSession$ . Please Help!!!!!!! Fri, 04 Nov, 21:05
Holden Karau Re: Spark-packages Mon, 07 Nov, 04:31
Holden Karau Re: SparkILoop doesn't run Thu, 17 Nov, 16:53
Holden Karau Re: PySpark TaskContext Thu, 24 Nov, 09:48
Holden Karau Re: Yarn resource utilization with Spark pipe() Thu, 24 Nov, 09:59
Holden Karau Re: PySpark TaskContext Thu, 24 Nov, 10:05
Holden Karau Re: PySpark TaskContext Thu, 24 Nov, 10:10
Holden Karau Re: PySpark TaskContext Thu, 24 Nov, 11:23
Holden Karau Re: Yarn resource utilization with Spark pipe() Thu, 24 Nov, 21:30
Hster Geguri kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Thu, 17 Nov, 23:58
Hster Geguri Mac vs cluster Re: kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Sat, 19 Nov, 17:12
Hster Geguri Re: Mac vs cluster Re: kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic Sat, 19 Nov, 17:56
Hyukjin Kwon Re: Spark XML ignore namespaces Fri, 04 Nov, 05:01
Hyukjin Kwon Re: Error creating SparkSession, in IntelliJ Fri, 04 Nov, 05:03
Hyukjin Kwon Re: Reading csv files with quoted fields containing embedded commas Sun, 06 Nov, 11:59
Hyukjin Kwon Re: pyspark: accept unicode column names in DataFrame.corr and cov Sat, 12 Nov, 11:43
Hyukjin Kwon Re: Spark SQL shell hangs Mon, 14 Nov, 01:18
Hyukjin Kwon Re: How to read a Multi Line json object via Spark Tue, 15 Nov, 08:11
Hyukjin Kwon Re: Spark-xml - OutOfMemoryError: Requested array size exceeds VM limit Wed, 16 Nov, 00:52
Hyukjin Kwon Re: How do I convert json_encoded_blob_column into a data frame? (This may be a feature request) Wed, 16 Nov, 10:49
Message list« Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · Next »Thread · Author · Date
Box list
Nov 201973
Oct 2019124
Sep 2019160
Aug 2019187
Jul 2019193
Jun 2019265
May 2019317
Apr 2019263
Mar 2019248
Feb 2019186
Jan 2019244
Dec 2018202
Nov 2018235
Oct 2018275
Sep 2018235
Aug 2018262
Jul 2018309
Jun 2018377
May 2018386
Apr 2018410
Mar 2018444
Feb 2018383
Jan 2018332
Dec 2017350
Nov 2017267
Oct 2017410
Sep 2017452
Aug 2017525
Jul 2017520
Jun 2017645
May 2017549
Apr 2017564
Mar 2017621
Feb 2017744
Jan 2017889
Dec 2016865
Nov 20161118
Oct 20161115
Sep 20161402
Aug 20161564
Jul 20161684
Jun 20161457
May 20161496
Apr 20161411
Mar 20162044
Feb 20161799
Jan 20161740
Dec 20151870
Nov 20151541
Oct 20152041
Sep 20152125
Aug 20151978
Jul 20152343
Jun 20152366
May 20151864
Apr 20152314
Mar 20152577
Feb 20152187
Jan 20152152
Dec 20141937
Nov 20142024
Oct 20142244
Sep 20142094
Aug 20141949
Jul 20142389
Jun 20141773
May 20141397
Apr 20141459
Mar 20141286
Feb 20141029
Jan 2014925
Dec 2013611
Nov 2013558
Oct 2013505
Sep 2013235
Aug 201397
Jul 20137