When I do that , and then do a select, full of errors. I think Hive table to read.

select * from mine;
OK
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
at parquet.hadoop.codec.SnappyDecompressor.decompress(SnappyDecompressor.java:62)
at parquet.hadoop.codec.NonBlockedDecompressorStream.read(NonBlockedDecompressorStream.java:51)
at java.io.DataInputStream.readFully(DataInputStream.java:195)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at parquet.bytes.BytesInput$StreamBytesInput.toByteArray(BytesInput.java:204)
at parquet.column.impl.ColumnReaderImpl.readPageV1(ColumnReaderImpl.java:557)
at parquet.column.impl.ColumnReaderImpl.access$300(ColumnReaderImpl.java:57)
at parquet.column.impl.ColumnReaderImpl$3.visit(ColumnReaderImpl.java:516)
at parquet.column.impl.ColumnReaderImpl$3.visit(ColumnReaderImpl.java:513)
at parquet.column.page.DataPageV1.accept(DataPageV1.java:96)
at parquet.column.impl.ColumnReaderImpl.readPage(ColumnReaderImpl.java:513)
at parquet.column.impl.ColumnReaderImpl.checkRead(ColumnReaderImpl.java:505)
at parquet.column.impl.ColumnReaderImpl.consume(ColumnReaderImpl.java:607)
at parquet.column.impl.ColumnReaderImpl.<init>(ColumnReaderImpl.java:351)
at parquet.column.impl.ColumnReadStoreImpl.newMemColumnReader(ColumnReadStoreImpl.java:66)
at parquet.column.impl.ColumnReadStoreImpl.getColumnReader(ColumnReadStoreImpl.java:61)
at parquet.io.RecordReaderImplementation.<init>(RecordReaderImplementation.java:270)
at parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:134)
at parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:99)
at parquet.filter2.compat.FilterCompat$NoOpFilter.accept(FilterCompat.java:154)
at parquet.io.MessageColumnIO.getRecordReader(MessageColumnIO.java:99)
at parquet.hadoop.InternalParquetRecordReader.checkRead(InternalParquetRecordReader.java:137)
at parquet.hadoop.InternalParquetRecordReader.nextKeyValue(InternalParquetRecordReader.java:208)
at parquet.hadoop.ParquetRecordReader.nextKeyValue(ParquetRecordReader.java:201)
at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:122)
at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:85)
at org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:72)
at org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:673)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:323)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:445)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:414)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1670)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at org.xerial.snappy.SnappyNativeLoader.loadLibrary(SnappyNativeLoader.java:52)
... 52 more
Exception in thread "main" org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] null
at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
at parquet.hadoop.codec.SnappyDecompressor.decompress(SnappyDecompressor.java:62)
at parquet.hadoop.codec.NonBlockedDecompressorStream.read(NonBlockedDecompressorStream.java:51)
at java.io.DataInputStream.readFully(DataInputStream.java:195)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at parquet.bytes.BytesInput$StreamBytesInput.toByteArray(BytesInput.java:204)
at parquet.column.impl.ColumnReaderImpl.readPageV1(ColumnReaderImpl.java:557)
at parquet.column.impl.ColumnReaderImpl.access$300(ColumnReaderImpl.java:57)
at parquet.column.impl.ColumnReaderImpl$3.visit(ColumnReaderImpl.java:516)
at parquet.column.impl.ColumnReaderImpl$3.visit(ColumnReaderImpl.java:513)
at parquet.column.page.DataPageV1.accept(DataPageV1.java:96)
at parquet.column.impl.ColumnReaderImpl.readPage(ColumnReaderImpl.java:513)
at parquet.column.impl.ColumnReaderImpl.checkRead(ColumnReaderImpl.java:505)
at parquet.column.impl.ColumnReaderImpl.consume(ColumnReaderImpl.java:607)
at parquet.column.impl.ColumnReaderImpl.<init>(ColumnReaderImpl.java:351)
at parquet.column.impl.ColumnReadStoreImpl.newMemColumnReader(ColumnReadStoreImpl.java:66)
at parquet.column.impl.ColumnReadStoreImpl.getColumnReader(ColumnReadStoreImpl.java:61)
at parquet.io.RecordReaderImplementation.<init>(RecordReaderImplementation.java:270)
at parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:134)
at parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:99)
at parquet.filter2.compat.FilterCompat$NoOpFilter.accept(FilterCompat.java:154)
at parquet.io.MessageColumnIO.getRecordReader(MessageColumnIO.java:99)
at parquet.hadoop.InternalParquetRecordReader.checkRead(InternalParquetRecordReader.java:137)
at parquet.hadoop.InternalParquetRecordReader.nextKeyValue(InternalParquetRecordReader.java:208)
at parquet.hadoop.ParquetRecordReader.nextKeyValue(ParquetRecordReader.java:201)
at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:122)
at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:85)
at org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:72)
at org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:673)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:323)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:445)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:414)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1670)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Feb 11, 2018 3:14:06 AM WARNING: parquet.hadoop.ParquetRecordReader: Can not initialize counter due to context is not a instance of TaskInputOutputContext, but is org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl
Feb 11, 2018 3:14:06 AM INFO: parquet.hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 36635 records.
Feb 11, 2018 3:14:06 AM INFO: parquet.hadoop.InternalParquetRecordReader: at row 0. reading next block
Feb 11, 2018 3:14:06 AM INFO: parquet.hadoop.InternalParquetRecordReader: block read in memory in 27 ms. row count = 36635


On Sun, Feb 11, 2018 at 3:10 AM, Deepak Sharma <deepakmca05@gmail.com> wrote:
There was a typo:
Instead of :
alter table mine set locations "hdfs://localhost:8020/user/hive/warehouse/mine";

Use :
alter table mine set location "hdfs://localhost:8020/user/hive/warehouse/mine";

On Sun, Feb 11, 2018 at 1:38 PM, Deepak Sharma <deepakmca05@gmail.com> wrote:
Try this in hive:
alter table mine set locations "hdfs://localhost:8020/user/hive/warehouse/mine";

Thanks
Deepak

On Sun, Feb 11, 2018 at 1:24 PM, ☼ R Nair (रविशंकर नायर) <ravishankar.nair@gmail.com> wrote:
Hi,
Here you go:

hive> show create table mine;
OK
CREATE TABLE `mine`(
  `policyid` int, 
  `statecode` string, 
  `socialid` string, 
  `county` string, 
  `eq_site_limit` decimal(10,2), 
  `hu_site_limit` decimal(10,2), 
  `fl_site_limit` decimal(10,2), 
  `fr_site_limit` decimal(10,2), 
  `tiv_2014` decimal(10,2), 
  `tiv_2015` decimal(10,2), 
  `eq_site_deductible` int, 
  `hu_site_deductible` int, 
  `fl_site_deductible` int, 
  `fr_site_deductible` int, 
  `latitude` decimal(6,6), 
  `longitude` decimal(6,6), 
  `line` string, 
  `construction` string, 
  `point_granularity` int)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe
WITH SERDEPROPERTIES ( 
  'path'='hdfs://localhost:8020/user/hive/warehouse/mine') 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
  'file:/Users/ravishankarnair/spark-warehouse/mine'
TBLPROPERTIES (
  'spark.sql.sources.provider'='parquet', 
  'spark.sql.sources.schema.numParts'='1', 
  'spark.sql.sources.schema.part.0'='{\"type\":\"struct\",\"fields\":[{\"name\":\"policyid\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"policyid\",\"scale\":0}},{\"name\":\"statecode\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"name\":\"statecode\",\"scale\":0}},{\"name\":\"Socialid\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"name\":\"Socialid\",\"scale\":0}},{\"name\":\"county\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"name\":\"county\",\"scale\":0}},{\"name\":\"eq_site_limit\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"eq_site_limit\",\"scale\":2}},{\"name\":\"hu_site_limit\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"hu_site_limit\",\"scale\":2}},{\"name\":\"fl_site_limit\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"fl_site_limit\",\"scale\":2}},{\"name\":\"fr_site_limit\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"fr_site_limit\",\"scale\":2}},{\"name\":\"tiv_2014\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"tiv_2014\",\"scale\":2}},{\"name\":\"tiv_2015\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"tiv_2015\",\"scale\":2}},{\"name\":\"eq_site_deductible\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"eq_site_deductible\",\"scale\":0}},{\"name\":\"hu_site_deductible\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"hu_site_deductible\",\"scale\":0}},{\"name\":\"fl_site_deductible\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"fl_site_deductible\",\"scale\":0}},{\"name\":\"fr_site_deductible\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"fr_site_deductible\",\"scale\":0}},{\"name\":\"latitude\",\"type\":\"decimal(6,6)\",\"nullable\":true,\"metadata\":{\"name\":\"latitude\",\"scale\":6}},{\"name\":\"longitude\",\"type\":\"decimal(6,6)\",\"nullable\":true,\"metadata\":{\"name\":\"longitude\",\"scale\":6}},{\"name\":\"line\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"name\":\"line\",\"scale\":0}},{\"name\":\"construction\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"name\":\"construction\",\"scale\":0}},{\"name\":\"point_granularity\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"point_granularity\",\"scale\":0}}]}'
  'transient_lastDdlTime'='1518335598')
Time taken: 0.13 seconds, Fetched: 35 row(s)

On Sun, Feb 11, 2018 at 2:36 AM, Shmuel Blitz <shmuel.blitz@similarweb.com> wrote:
Please run the following command, and paste the result:
SHOW CREATE TABLE <<TABLE-NAME>>

On Sun, Feb 11, 2018 at 7:56 AM, ☼ R Nair (रविशंकर नायर) <ravishankar.nair@gmail.com> wrote:
No, No luck.

Thanks

On Sun, Feb 11, 2018 at 12:48 AM, Deepak Sharma <deepakmca05@gmail.com> wrote:
In hive cli:
msck repair table 《table_name》;

Thanks
Deepak

On Feb 11, 2018 11:14, "☼ R Nair (रविशंकर नायर)" <ravishankar.nair@gmail.com> wrote:
NO, can you pease explain the command ? Let me try now.

Best,

On Sun, Feb 11, 2018 at 12:40 AM, Deepak Sharma <deepakmca05@gmail.com> wrote:
I am not sure about the exact issue bjt i see you are partioning while writing from spark.
Did you tried msck repair on the table before reading it in hive ?

Thanks
Deepak

On Feb 11, 2018 11:06, "☼ R Nair (रविशंकर नायर)" <ravishankar.nair@gmail.com> wrote:
All,

Thanks for the inputs. Again I am not successful. I think, we need to resolve this, as this is a very common requirement.

Please go through my complete code:

STEP 1:  Started Spark shell as spark-shell --master yarn

STEP 2: Flowing code is being given as inout to shark shell

import org.apache.spark.sql.Row
import org.apache.spark.sql.SparkSession
val warehouseLocation ="/user/hive/warehouse"

val spark = SparkSession.builder().appName("Spark Hive Example").config("spark.sql.warehouse.dir", warehouseLocation).enableHiveSupport().getOrCreate()

import org.apache.spark.sql._
var passion_df = spark.read.
format("jdbc").
option("url", "jdbc:mysql://localhost:3307/policies").
option("driver" ,"com.mysql.jdbc.Driver").
option("user", "root").
option("password", "root").
option("dbtable", "insurancedetails").
option("partitionColumn", "policyid").
option("lowerBound", "1").
option("upperBound", "100000").
option("numPartitions", "4").
load()
//Made sure that passion_df is created, as passion_df.show(5) shows me correct data.
passion_df.write.saveAsTable("default.mine") //Default parquet

STEP 3: Went to HIVE. Started HIVE prompt. 

hive> show tables;
OK
callcentervoicelogs
mine      
Time taken: 0.035 seconds, Fetched: 2 row(s)
//As you can see HIVE is showing the table "mine" in default schema.

STEP 4: HERE IS THE PROBLEM. 

hive> select * from mine;
OK
Time taken: 0.354 seconds
hive> 
//Where is the data ???

STEP 5:

See the below command on HIVE

describe formatted mine;
OK
# col_name             data_type           comment             
 
policyid             int                                    
statecode           string                                  
socialid             string                                  
county               string                                  
eq_site_limit       decimal(10,2)                          
hu_site_limit       decimal(10,2)                          
fl_site_limit       decimal(10,2)                          
fr_site_limit       decimal(10,2)                          
tiv_2014             decimal(10,2)                          
tiv_2015             decimal(10,2)                          
eq_site_deductible   int                                    
hu_site_deductible   int                                    
fl_site_deductible   int                                    
fr_site_deductible   int                                    
latitude             decimal(6,6)                            
longitude           decimal(6,6)                            
line                 string                                  
construction         string                                  
point_granularity   int                                    
 
# Detailed Table Information  
Database:           default              
Owner:               ravishankarnair      
CreateTime:         Sun Feb 11 00:26:40 EST 2018  
LastAccessTime:     UNKNOWN              
Protect Mode:       None                  
Retention:           0                    
Location:           file:/Users/ravishankarnair/spark-warehouse/mine  
Table Type:         MANAGED_TABLE        
Table Parameters:  
spark.sql.sources.provider parquet             
spark.sql.sources.schema.numParts 1                   
spark.sql.sources.schema.part.0 {\"type\":\"struct\",\"fields\":[{\"name\":\"policyid\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"policyid\",\"scale\":0}},{\"name\":\"statecode\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"name\":\"statecode\",\"scale\":0}},{\"name\":\"Socialid\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"name\":\"Socialid\",\"scale\":0}},{\"name\":\"county\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"name\":\"county\",\"scale\":0}},{\"name\":\"eq_site_limit\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"eq_site_limit\",\"scale\":2}},{\"name\":\"hu_site_limit\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"hu_site_limit\",\"scale\":2}},{\"name\":\"fl_site_limit\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"fl_site_limit\",\"scale\":2}},{\"name\":\"fr_site_limit\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"fr_site_limit\",\"scale\":2}},{\"name\":\"tiv_2014\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"tiv_2014\",\"scale\":2}},{\"name\":\"tiv_2015\",\"type\":\"decimal(10,2)\",\"nullable\":true,\"metadata\":{\"name\":\"tiv_2015\",\"scale\":2}},{\"name\":\"eq_site_deductible\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"eq_site_deductible\",\"scale\":0}},{\"name\":\"hu_site_deductible\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"hu_site_deductible\",\"scale\":0}},{\"name\":\"fl_site_deductible\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"fl_site_deductible\",\"scale\":0}},{\"name\":\"fr_site_deductible\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"fr_site_deductible\",\"scale\":0}},{\"name\":\"latitude\",\"type\":\"decimal(6,6)\",\"nullable\":true,\"metadata\":{\"name\":\"latitude\",\"scale\":6}},{\"name\":\"longitude\",\"type\":\"decimal(6,6)\",\"nullable\":true,\"metadata\":{\"name\":\"longitude\",\"scale\":6}},{\"name\":\"line\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"name\":\"line\",\"scale\":0}},{\"name\":\"construction\",\"type\":\"string\",\"nullable\":true,\"metadata\":{\"name\":\"construction\",\"scale\":0}},{\"name\":\"point_granularity\",\"type\":\"integer\",\"nullable\":true,\"metadata\":{\"name\":\"point_granularity\",\"scale\":0}}]}
transient_lastDdlTime 1518326800          
 
# Storage Information  
SerDe Library:       org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe  
InputFormat:         org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat  
OutputFormat:       org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat  
Compressed:         No                    
Num Buckets:         -1                    
Bucket Columns:     []                    
Sort Columns:       []                    
Storage Desc Params:  
path                 hdfs://localhost:8020/user/hive/warehouse/mine
serialization.format 1                   
Time taken: 0.077 seconds, Fetched: 48 row(s)

Now, I see your advise and support. Whats the issue? Am I doing wrong, it it a bug ? I am using Spark 2.2.1, HIVE 1.2.1, HADOOP 2.7.3. All class path, configuration are set properly.

Best,

Ravion

On Fri, Feb 9, 2018 at 1:29 PM, Nicholas Hakobian <nicholas.hakobian@rallyhealth.com> wrote:
Its possible that the format of your table is not compatible with your version of hive, so Spark saved it in a way such that only Spark can read it. When this happens it prints out a very visible warning letting you know this has happened.

We've seen it most frequently when trying to save a parquet file with a column in date format into a Hive table. In older versions of hive, its parquet reader/writer did not support Date formats (among a couple others).

Nicholas Szandor Hakobian, Ph.D.
Staff Data Scientist
Rally Health


On Fri, Feb 9, 2018 at 9:59 AM, Prakash Joshi <prakashcjoshi5@gmail.com> wrote:
Ravi,

Can you send the result of 
Show create table your_table_name

Thanks 
Prakash

On Feb 9, 2018 8:20 PM, "☼ R Nair (रविशंकर नायर)" <ravishankar.nair@gmail.com> wrote:
All,

It has been three days continuously I am on this issue. Not getting any clue. 

Environment: Spark 2.2.x, all configurations are correct. hive-site.xml is in spark's conf.

1) Step 1: I created a data frame DF1 reading a csv file.

2) Did  manipulations on DF1. Resulting frame is passion_df.

3) passion_df.write.format("orc").saveAsTable("sampledb.passion")

4) The metastore shows the hive table., when I do "show tables" in HIVE, I can see table name

5) I can't select in HIVE, though I can select from SPARK as spark.sql("select * from sampledb.passion")

Whats going on here? Please help. Why I am not seeing data from HIVE prompt?
The "describe formatted " command on the table in HIVE shows he data is is in default warehouse location ( /user/hive/warehouse) since I set it.

I am not getting any definite answer anywhere. Many suggestions and answers given in Stackoverflow et al.Nothing really works.

So asking experts here for some light on this, thanks

Best,
Ravion 






--




--




--




--
Shmuel Blitz
Big Data Developer
Email: shmuel.blitz@similarweb.com
www.similarweb.com



--




--



--



--