spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "barge.nilesh" <barge.nil...@gmail.com>
Subject Re: Schema change on Spark Hive (Parquet file format) table not working
Date Tue, 30 Sep 2014 16:55:49 GMT
code snippet in short:

hiveContext.sql("*CREATE EXTERNAL TABLE IF NOT EXISTS people_table (name
String, age INT) ROW FORMAT SERDE 'parquet.hive.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT 'parquet.hive.DeprecatedParquetInputFormat'
OUTPUTFORMAT 'parquet.hive.DeprecatedParquetOutputFormat'*"); 
hiveContext.sql("*INSERT INTO TABLE people_table SELECT name, age FROM
temp_table_people1*"); 
hiveContext.sql("*SELECT * FROM people_table*"); ///Here, data read was
successful./ 
hiveContext.sql("*ALTER TABLE people_table ADD COLUMNS (gender STRING)*");
hiveContext.sql("*SELECT * FROM people_table*"); ///Not able to read
existing data and ArrayIndexOutOfBoundsException is thrown./



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Schema-change-on-Spark-Hive-Parquet-file-format-table-not-working-tp15360p15415.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message