spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (Jira)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-29313) Failure on writing to `noop` datasource in benchmarks
Date Wed, 02 Oct 2019 04:06:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-29313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dongjoon Hyun reassigned SPARK-29313:
-------------------------------------

    Assignee: Maxim Gekk

> Failure on writing to `noop` datasource in benchmarks
> -----------------------------------------------------
>
>                 Key: SPARK-29313
>                 URL: https://issues.apache.org/jira/browse/SPARK-29313
>             Project: Spark
>          Issue Type: Test
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Maxim Gekk
>            Assignee: Maxim Gekk
>            Priority: Minor
>
> This failure happens on overwriting results of ExtractBenchmark via command:
> {code}
> SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain org.apache.spark.sql.execution.benchmark.ExtractBenchmark"
> {code}
> with the exception:
> {code}
> [info] Running benchmark: Invoke extract for timestamp
> [info]   Running case: cast to timestamp
> [error] Exception in thread "main" org.apache.spark.sql.AnalysisException: TableProvider
implementation noop cannot be written with ErrorIfExists mode, please use Append or Overwrite
modes instead.;
> [error] 	at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:284)
> [error] 	at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$doBenchmark$1(ExtractBenchmark.scala:46)
> [error] 	at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:47)
> [error] 	at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:31)
> [error] 	at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.withSQLConf(ExtractBenchmark.scala:37)
> [error] 	at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.doBenchmark(ExtractBenchmark.scala:47)
> [error] 	at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$run$1(ExtractBenchmark.scala:57)
> [error] 	at org.apache.spark.benchmark.Benchmark.$anonfun$addCase$1(Benchmark.scala:77)
> [error] 	at org.apache.spark.benchmark.Benchmark.$anonfun$addCase$1$adapted(Benchmark.scala:75)
> [error] 	at org.apache.spark.benchmark.Benchmark.measure(Benchmark.scala:139)
> [error] 	at org.apache.spark.benchmark.Benchmark.$anonfun$run$1(Benchmark.scala:106)
> [error] 	at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
> [error] 	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
> [error] 	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
> [error] 	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
> [error] 	at scala.collection.TraversableLike.map(TraversableLike.scala:238)
> [error] 	at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
> [error] 	at scala.collection.AbstractTraversable.map(Traversable.scala:108)
> [error] 	at org.apache.spark.benchmark.Benchmark.run(Benchmark.scala:104)
> [error] 	at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$runBenchmarkSuite$2(ExtractBenchmark.scala:101)
> [error] 	at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$runBenchmarkSuite$2$adapted(ExtractBenchmark.scala:95)
> [error] 	at scala.collection.immutable.List.foreach(List.scala:392)
> [error] 	at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$runBenchmarkSuite$1(ExtractBenchmark.scala:95)
> [error] 	at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.$anonfun$runBenchmarkSuite$1$adapted(ExtractBenchmark.scala:94)
> [error] 	at scala.collection.immutable.List.foreach(List.scala:392)
> [error] 	at org.apache.spark.sql.execution.benchmark.ExtractBenchmark$.runBenchmarkSuite(ExtractBenchmark.scala:94)
> [error] 	at org.apache.spark.benchmark.BenchmarkBase.main(BenchmarkBase.scala:57)
> [error] 	at org.apache.spark.sql.execution.benchmark.ExtractBenchmark.main(ExtractBenchmark.scala)
> java.lang.RuntimeException: Nonzero exit code returned from runner: 1
> 	at scala.sys.package$.error(package.scala:27)
> [trace] Stack trace suppressed: run last sql/test:runMain for the full output.
> [error] (sql/test:runMain) Nonzero exit code returned from runner: 1
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message