spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yin Huai <yh...@databricks.com>
Subject Re: Spark DataFrame 1.4 write to parquet/saveAsTable tasks fail
Date Wed, 17 Jun 2015 17:59:04 GMT
So, the second attemp of those tasks failed with NPE can complete and the
job eventually finished?

On Mon, Jun 15, 2015 at 10:37 PM, Night Wolf <nightwolfzor@gmail.com> wrote:

> Hey Yin,
>
> Thanks for the link to the JIRA. I'll add details to it. But I'm able to
> reproduce it, at least in the same shell session, every time I do a write I
> get a random number of tasks failing on the first run with the NPE.
>
> Using dynamic allocation of executors in YARN mode. No speculative
> execution is enabled.
>
> On Tue, Jun 16, 2015 at 3:11 PM, Yin Huai <yhuai@databricks.com> wrote:
>
>> I saw it once but I was not clear how to reproduce it. The jira I created
>> is https://issues.apache.org/jira/browse/SPARK-7837.
>>
>> More information will be very helpful. Were those errors from speculative
>> tasks or regular tasks (the first attempt of the task)? Is this error
>> deterministic (can you reproduce every time you run this command)?
>>
>> Thanks,
>>
>> Yin
>>
>> On Mon, Jun 15, 2015 at 8:59 PM, Night Wolf <nightwolfzor@gmail.com>
>> wrote:
>>
>>> Looking at the logs of the executor, looks like it fails to find the
>>> file; e.g. for task 10323.0
>>>
>>>
>>> 15/06/16 13:43:13 ERROR output.FileOutputCommitter: Hit IOException
>>> trying to rename
>>> maprfs:///user/hive/warehouse/is_20150617_test2/_temporary/_attempt_201506161340_0000_m_010181_0/part-r-353626.gz.parquet
>>> to maprfs:/user/hive/warehouse/is_20150617_test2/part-r-353626.gz.parquet
>>> java.io.IOException: Invalid source or target
>>> at com.mapr.fs.MapRFileSystem.rename(MapRFileSystem.java:952)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:201)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:225)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:167)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.performCommit$1(SparkHadoopMapRedUtil.scala:100)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.commitTask(SparkHadoopMapRedUtil.scala:137)
>>> at
>>> org.apache.spark.sql.sources.BaseWriterContainer.commitTask(commands.scala:357)
>>> at
>>> org.apache.spark.sql.sources.DefaultWriterContainer.commitTask(commands.scala:394)
>>> at org.apache.spark.sql.sources.InsertIntoHadoopFsRelation.org
>>> $apache$spark$sql$sources$InsertIntoHadoopFsRelation$$writeRows$1(commands.scala:157)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> 15/06/16 13:43:13 ERROR mapred.SparkHadoopMapRedUtil: Error committing
>>> the output of task: attempt_201506161340_0000_m_010181_0
>>> java.io.IOException: Invalid source or target
>>> at com.mapr.fs.MapRFileSystem.rename(MapRFileSystem.java:952)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:201)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:225)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:167)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.performCommit$1(SparkHadoopMapRedUtil.scala:100)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.commitTask(SparkHadoopMapRedUtil.scala:137)
>>> at
>>> org.apache.spark.sql.sources.BaseWriterContainer.commitTask(commands.scala:357)
>>> at
>>> org.apache.spark.sql.sources.DefaultWriterContainer.commitTask(commands.scala:394)
>>> at org.apache.spark.sql.sources.InsertIntoHadoopFsRelation.org
>>> $apache$spark$sql$sources$InsertIntoHadoopFsRelation$$writeRows$1(commands.scala:157)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> 15/06/16 13:43:16 ERROR output.FileOutputCommitter: Hit IOException
>>> trying to rename
>>> maprfs:///user/hive/warehouse/is_20150617_test2/_temporary/_attempt_201506161341_0000_m_010323_0/part-r-353768.gz.parquet
>>> to maprfs:/user/hive/warehouse/is_20150617_test2/part-r-353768.gz.parquet
>>> java.io.IOException: Invalid source or target
>>> at com.mapr.fs.MapRFileSystem.rename(MapRFileSystem.java:952)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:201)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:225)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:167)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.performCommit$1(SparkHadoopMapRedUtil.scala:100)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.commitTask(SparkHadoopMapRedUtil.scala:137)
>>> at
>>> org.apache.spark.sql.sources.BaseWriterContainer.commitTask(commands.scala:357)
>>> at
>>> org.apache.spark.sql.sources.DefaultWriterContainer.commitTask(commands.scala:394)
>>> at org.apache.spark.sql.sources.InsertIntoHadoopFsRelation.org
>>> $apache$spark$sql$sources$InsertIntoHadoopFsRelation$$writeRows$1(commands.scala:157)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> 15/06/16 13:43:16 ERROR mapred.SparkHadoopMapRedUtil: Error committing
>>> the output of task: attempt_201506161341_0000_m_010323_0
>>> java.io.IOException: Invalid source or target
>>> at com.mapr.fs.MapRFileSystem.rename(MapRFileSystem.java:952)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:201)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:225)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:167)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.performCommit$1(SparkHadoopMapRedUtil.scala:100)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.commitTask(SparkHadoopMapRedUtil.scala:137)
>>> at
>>> org.apache.spark.sql.sources.BaseWriterContainer.commitTask(commands.scala:357)
>>> at
>>> org.apache.spark.sql.sources.DefaultWriterContainer.commitTask(commands.scala:394)
>>> at org.apache.spark.sql.sources.InsertIntoHadoopFsRelation.org
>>> $apache$spark$sql$sources$InsertIntoHadoopFsRelation$$writeRows$1(commands.scala:157)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> 15/06/16 13:43:20 INFO codec.CodecConfig: Compression: GZIP
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Parquet block size to
>>> 134217728
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Parquet page size to
>>> 1048576
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Parquet dictionary
>>> page size to 1048576
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Dictionary is on
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Validation is off
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Writer version is:
>>> PARQUET_1_0
>>> 15/06/16 13:43:20 INFO codec.CodecConfig: Compression: GZIP
>>> 15/06/16 13:43:20 INFO codec.CodecConfig: Compression: GZIP
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Parquet block size to
>>> 134217728
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Parquet block size to
>>> 134217728
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Parquet page size to
>>> 1048576
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Parquet page size to
>>> 1048576
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Parquet dictionary
>>> page size to 1048576
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Parquet dictionary
>>> page size to 1048576
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Dictionary is on
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Dictionary is on
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Validation is off
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Validation is off
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Writer version is:
>>> PARQUET_1_0
>>> 15/06/16 13:43:20 INFO hadoop.ParquetOutputFormat: Writer version is:
>>> PARQUET_1_0
>>> 15/06/16 13:43:20 ERROR fs.MapRFileSystem: Failed to delete path
>>> maprfs:/user/hive/warehouse/is_20150617_test2/_temporary/_attempt_201506161340_0000_m_010181_0,
>>> error: No such file or directory (2)
>>> 15/06/16 13:43:21 ERROR sources.DefaultWriterContainer: Task attempt
>>> attempt_201506161340_0000_m_010181_0 aborted.
>>> 15/06/16 13:43:21 ERROR sources.InsertIntoHadoopFsRelation: Aborting
>>> task.
>>> java.lang.RuntimeException: Failed to commit task
>>> at
>>> org.apache.spark.sql.sources.DefaultWriterContainer.commitTask(commands.scala:398)
>>> at org.apache.spark.sql.sources.InsertIntoHadoopFsRelation.org
>>> $apache$spark$sql$sources$InsertIntoHadoopFsRelation$$writeRows$1(commands.scala:157)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: java.io.IOException: Invalid source or target
>>> at com.mapr.fs.MapRFileSystem.rename(MapRFileSystem.java:952)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:201)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:225)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:167)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.performCommit$1(SparkHadoopMapRedUtil.scala:100)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.commitTask(SparkHadoopMapRedUtil.scala:137)
>>> at
>>> org.apache.spark.sql.sources.BaseWriterContainer.commitTask(commands.scala:357)
>>> at
>>> org.apache.spark.sql.sources.DefaultWriterContainer.commitTask(commands.scala:394)
>>> ... 9 more
>>> 15/06/16 13:43:21 ERROR fs.MapRFileSystem: Failed to delete path
>>> maprfs:/user/hive/warehouse/is_20150617_test2/_temporary/_attempt_201506161341_0000_m_010323_0,
>>> error: No such file or directory (2)
>>> 15/06/16 13:43:21 INFO compress.CodecPool: Got brand-new compressor [.gz]
>>> 15/06/16 13:43:21 INFO compress.CodecPool: Got brand-new compressor [.gz]
>>> 15/06/16 13:43:21 INFO compress.CodecPool: Got brand-new compressor [.gz]
>>> 15/06/16 13:43:21 INFO hadoop.InternalParquetRecordReader: at row 0.
>>> reading next block
>>> 15/06/16 13:43:21 INFO hadoop.InternalParquetRecordReader: at row 0.
>>> reading next block
>>> 15/06/16 13:43:21 INFO hadoop.InternalParquetRecordReader: at row 0.
>>> reading next block
>>> 15/06/16 13:43:21 INFO hadoop.InternalParquetRecordReader: block read in
>>> memory in 124 ms. row count = 998525
>>> 15/06/16 13:43:21 INFO hadoop.InternalParquetRecordReader: block read in
>>> memory in 201 ms. row count = 983534
>>> 15/06/16 13:43:21 INFO hadoop.InternalParquetRecordReader: block read in
>>> memory in 217 ms. row count = 970355
>>> 15/06/16 13:43:22 ERROR sources.DefaultWriterContainer: Task attempt
>>> attempt_201506161341_0000_m_010323_0 aborted.
>>> 15/06/16 13:43:22 ERROR sources.InsertIntoHadoopFsRelation: Aborting
>>> task.
>>> java.lang.RuntimeException: Failed to commit task
>>> at
>>> org.apache.spark.sql.sources.DefaultWriterContainer.commitTask(commands.scala:398)
>>> at org.apache.spark.sql.sources.InsertIntoHadoopFsRelation.org
>>> $apache$spark$sql$sources$InsertIntoHadoopFsRelation$$writeRows$1(commands.scala:157)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: java.io.IOException: Invalid source or target
>>> at com.mapr.fs.MapRFileSystem.rename(MapRFileSystem.java:952)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:201)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:225)
>>> at
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:167)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.performCommit$1(SparkHadoopMapRedUtil.scala:100)
>>> at
>>> org.apache.spark.mapred.SparkHadoopMapRedUtil$.commitTask(SparkHadoopMapRedUtil.scala:137)
>>> at
>>> org.apache.spark.sql.sources.BaseWriterContainer.commitTask(commands.scala:357)
>>> at
>>> org.apache.spark.sql.sources.DefaultWriterContainer.commitTask(commands.scala:394)
>>> ... 9 more
>>> 15/06/16 13:43:22 ERROR fs.MapRFileSystem: Failed to delete path
>>> maprfs:/user/hive/warehouse/is_20150617_test2/_temporary/_attempt_201506161341_0000_m_010323_0,
>>> error: No such file or directory (2)
>>> 15/06/16 13:43:22 ERROR fs.MapRFileSystem: Failed to delete path
>>> maprfs:/user/hive/warehouse/is_20150617_test2/_temporary/_attempt_201506161340_0000_m_010181_0,
>>> error: No such file or directory (2)
>>> 15/06/16 13:43:22 ERROR sources.DefaultWriterContainer: Task attempt
>>> attempt_201506161341_0000_m_010323_0 aborted.
>>> 15/06/16 13:43:22 ERROR sources.DefaultWriterContainer: Task attempt
>>> attempt_201506161340_0000_m_010181_0 aborted.
>>> 15/06/16 13:43:22 ERROR executor.Executor: Exception in task 10323.0 in
>>> stage 0.0 (TID 8896)
>>> java.lang.NullPointerException
>>> at
>>> parquet.hadoop.InternalParquetRecordWriter.flushRowGroupToStore(InternalParquetRecordWriter.java:146)
>>> at
>>> parquet.hadoop.InternalParquetRecordWriter.close(InternalParquetRecordWriter.java:112)
>>> at parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:73)
>>> at
>>> org.apache.spark.sql.parquet.ParquetOutputWriter.close(newParquet.scala:116)
>>> at
>>> org.apache.spark.sql.sources.DefaultWriterContainer.abortTask(commands.scala:404)
>>> at org.apache.spark.sql.sources.InsertIntoHadoopFsRelation.org
>>> $apache$spark$sql$sources$InsertIntoHadoopFsRelation$$writeRows$1(commands.scala:160)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> 15/06/16 13:43:22 ERROR executor.Executor: Exception in task 10181.0 in
>>> stage 0.0 (TID 8835)
>>> java.lang.NullPointerException
>>> at
>>> parquet.hadoop.InternalParquetRecordWriter.flushRowGroupToStore(InternalParquetRecordWriter.java:146)
>>> at
>>> parquet.hadoop.InternalParquetRecordWriter.close(InternalParquetRecordWriter.java:112)
>>> at parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:73)
>>> at
>>> org.apache.spark.sql.parquet.ParquetOutputWriter.close(newParquet.scala:116)
>>> at
>>> org.apache.spark.sql.sources.DefaultWriterContainer.abortTask(commands.scala:404)
>>> at org.apache.spark.sql.sources.InsertIntoHadoopFsRelation.org
>>> $apache$spark$sql$sources$InsertIntoHadoopFsRelation$$writeRows$1(commands.scala:160)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at
>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> 15/06/16 13:43:22 INFO executor.CoarseGrainedExecutorBackend: Got
>>> assigned task 9552
>>> 15/06/16 13:43:22 INFO executor.Executor: Running task 11093.0 in stage
>>> 0.0 (TID 9552)
>>> 15/06/16 13:43:22 INFO executor.CoarseGrainedExecutorBackend: Got
>>> assigned task 9553
>>> 15/06/16 13:43:22 INFO executor.Executor: Running task 10323.1 in stage
>>> 0.0 (TID 9553)
>>>
>>> On Tue, Jun 16, 2015 at 1:47 PM, Night Wolf <nightwolfzor@gmail.com>
>>> wrote:
>>>
>>>> Hi guys,
>>>>
>>>> Using Spark 1.4, trying to save a dataframe as a table, a really simple
>>>> test, but I'm getting a bunch of NPEs;
>>>>
>>>> The code Im running is very simple;
>>>>
>>>>
>>>>  qc.read.parquet("/user/sparkuser/data/staged/item_sales_basket_id.parquet").write.format("parquet").saveAsTable("is_20150617_test2")
>>>>
>>>> Logs of tasks lost;
>>>>
>>>> [Stage 0:=================================>                (8771 + 450)
>>>> / 13000]15/06/16 03:42:30 WARN TaskSetManager: Lost task 10681.0 in stage
>>>> 0.0 (TID 8757, qtausc-pphd0146): java.lang.NullPointerException
>>>> at
>>>> parquet.hadoop.InternalParquetRecordWriter.flushRowGroupToStore(InternalParquetRecordWriter.java:146)
>>>> at
>>>> parquet.hadoop.InternalParquetRecordWriter.close(InternalParquetRecordWriter.java:112)
>>>> at parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:73)
>>>> at
>>>> org.apache.spark.sql.parquet.ParquetOutputWriter.close(newParquet.scala:116)
>>>> at
>>>> org.apache.spark.sql.sources.DefaultWriterContainer.abortTask(commands.scala:404)
>>>> at org.apache.spark.sql.sources.InsertIntoHadoopFsRelation.org
>>>> $apache$spark$sql$sources$InsertIntoHadoopFsRelation$$writeRows$1(commands.scala:160)
>>>> at
>>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>>> at
>>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>>
>>>> [Stage 0:==================================>               (9006 + 490)
>>>> / 13000]15/06/16 03:43:22 WARN TaskSetManager: Lost task 10323.0 in stage
>>>> 0.0 (TID 8896, qtausc-pphd0167): java.lang.NullPointerException
>>>> at
>>>> parquet.hadoop.InternalParquetRecordWriter.flushRowGroupToStore(InternalParquetRecordWriter.java:146)
>>>> at
>>>> parquet.hadoop.InternalParquetRecordWriter.close(InternalParquetRecordWriter.java:112)
>>>> at parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:73)
>>>> at
>>>> org.apache.spark.sql.parquet.ParquetOutputWriter.close(newParquet.scala:116)
>>>> at
>>>> org.apache.spark.sql.sources.DefaultWriterContainer.abortTask(commands.scala:404)
>>>> at org.apache.spark.sql.sources.InsertIntoHadoopFsRelation.org
>>>> $apache$spark$sql$sources$InsertIntoHadoopFsRelation$$writeRows$1(commands.scala:160)
>>>> at
>>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>>> at
>>>> org.apache.spark.sql.sources.InsertIntoHadoopFsRelation$$anonfun$insert$1.apply(commands.scala:132)
>>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message