spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 李铖 <lidali...@gmail.com>
Subject hadoop.ParquetOutputCommitter: could not write summary file
Date Tue, 29 Mar 2016 10:38:27 GMT
an error occured when write parquet files to disk.
any advise?
I want to know the reason.thanks
```
16/03/29 18:31:48 WARN hadoop.ParquetOutputCommitter: could not write
summary file for file:/tmp/goods/2015-6
java.lang.NullPointerException
at
org.apache.parquet.hadoop.ParquetFileWriter.mergeFooters(ParquetFileWriter.java:456)
at
org.apache.parquet.hadoop.ParquetFileWriter.writeMetadataFile(ParquetFileWriter.java:420)
at
org.apache.parquet.hadoop.ParquetOutputCommitter.writeMetaDataFile(ParquetOutputCommitter.java:58)
at
org.apache.parquet.hadoop.ParquetOutputCommitter.commitJob(ParquetOutputCommitter.java:48)
at
org.apache.spark.sql.execution.datasources.BaseWriterContainer.commitJob(WriterContainer.scala:230)
```

Mime
View raw message