spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ragini Manjaiah <ragini.manja...@gmail.com>
Subject [Spark SQL] : at org.apache.spark.sql.execution.datasources.orc.OrcColumnVector.getDecimal(OrcColumnVector.java:158)
Date Thu, 15 Jul 2021 09:19:44 GMT
<https://stackoverflow.com/posts/68391105/timeline>

Hi Team

I am trying to read from the HDFS path which is a partition on a sales date
and then selecting only one particular column spark job fails which is of
type decimal(32,20) (nullable = true). when I exclude this column and
select others it works fine.

hitting with at
org.apache.spark.sql.execution.datasources.orc.OrcColumnVector.getDecimal(OrcColumnVector.java:158)

alternativeApproach I took: I took a sample of data into another HDFS path
and queried it is working fine

what could be scenarios we can hit the above issue

Mime
View raw message