spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "1427357147@qq.com" <1427357...@qq.com>
Subject Hot to filter the datatime in dataset with java code please?
Date Wed, 11 Apr 2018 08:19:35 GMT
HI  all,

I want to filter the data by the datatime.
In mysql, the colume is the DATETIME type, named A. 
I write my code like:
import java.util.Date;
newX.filter(newX.col("A").isNull().or(newX.col("A").lt(new Date()))).show();

I got error :
Exception in thread "main" java.lang.RuntimeException: Unsupported literal type class java.util.Date
Wed Apr 11 16:17:31 CST 2018
at org.apache.spark.sql.catalyst.expressions.Literal$.apply(literals.scala:77)
at org.apache.spark.sql.catalyst.expressions.Literal$$anonfun$create$2.apply(literals.scala:163)
at org.apache.spark.sql.catalyst.expressions.Literal$$anonfun$create$2.apply(literals.scala:163)
at scala.util.Try.getOrElse(Try.scala:79)
at org.apache.spark.sql.catalyst.expressions.Literal$.create(literals.scala:162)
at org.apache.spark.sql.functions$.typedLit(functions.scala:113)
at org.apache.spark.sql.functions$.lit(functions.scala:96)
at org.apache.spark.sql.Column.$less(Column.scala:384)
at org.apache.spark.sql.Column.lt(Column.scala:399)
at Main.main(Main.java:38)

How should I do to filter the datetime in dataset filter please?






1427357147@qq.com
Mime
View raw message