spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sree Eedupuganti <>
Subject Spark Select Statement
Date Wed, 04 May 2016 09:39:19 GMT
Hello Spark users, can we query the SQL SELECT statement in Spark using
if it is possible any suggestions please. I tried like this.How to pass the
database name.
Here my database name is nimbus and table name is winbox_opens.

*Source Code :*

*public class Select { public static class SquareKey implements
Function<Row, Integer> {   public Integer call(Row row) throws Exception {
   return row.getInt(0) * row.getInt(0);   } } public static void
main(String[] args) throws Exception {  SparkConf s = new
SparkConf().setMaster("local[2]").setAppName("Select");  SparkContext sc =
new SparkContext(s);  HiveContext hc = new HiveContext(sc);   DataFrame rdd
= hc.sql("SELECT * FROM winbox_opens");   JavaRDD<Integer> squaredKeys =
rdd.toJavaRDD().map(new SquareKey());   List<Integer> result =
squaredKeys.collect();   for (Integer elem : result) {
 System.out.println(elem);   }  }}*

*Error: Exception in thread "main" org.apache.spark.sql.AnalysisException:
no such table winbox_prod_action_logs_1; line 1 pos 14*

Best Regards,
Sreeharsha Eedupuganti
Data Engineer
innData Analytics Private Limited

View raw message