spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From saurabh3d <>
Subject Window Functions with SQLContext
Date Thu, 01 Sep 2016 05:16:21 GMT
Hi All,

As per  SPARK-11001 <>   ,
Window functions should be supported by SQLContext. But when i try to run

SQLContext sqlContext = new SQLContext(jsc);
WindowSpec w = Window.partitionBy("assetId").orderBy("assetId");
DataFrame df_2 = df1.withColumn("row_number", row_number().over(w));;

it fails with:
Exception in thread "main" org.apache.spark.sql.AnalysisException: Could not
resolve window function 'row_number'. Note that, using window functions
currently requires a HiveContext;

This code runs fine with HiveContext.
Any idea what’s going on?  Is this a known issue and is there a workaround
to make Window function work without HiveContext.


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe e-mail:

View raw message