spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "barge.nilesh" <>
Subject Re: Schema change on Spark Hive (Parquet file format) table not working
Date Tue, 30 Sep 2014 16:55:49 GMT
code snippet in short:

hiveContext.sql("*CREATE EXTERNAL TABLE IF NOT EXISTS people_table (name
String, age INT) ROW FORMAT SERDE 'parquet.hive.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT 'parquet.hive.DeprecatedParquetInputFormat'
OUTPUTFORMAT 'parquet.hive.DeprecatedParquetOutputFormat'*"); 
hiveContext.sql("*INSERT INTO TABLE people_table SELECT name, age FROM
hiveContext.sql("*SELECT * FROM people_table*"); ///Here, data read was
hiveContext.sql("*ALTER TABLE people_table ADD COLUMNS (gender STRING)*");
hiveContext.sql("*SELECT * FROM people_table*"); ///Not able to read
existing data and ArrayIndexOutOfBoundsException is thrown./

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message