hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaowei Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12303) HCatRecordSerDe throw a IndexOutOfBoundsException
Date Mon, 02 Nov 2015 11:19:27 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14985077#comment-14985077
] 

Xiaowei Wang commented on HIVE-12303:
-------------------------------------

The schema is 
{noformat}

# col_name              data_type               comment             
                 
ip                      string                  from deserializer   
manualtime              string                  from deserializer   
timezone                string                  from deserializer   
pbparams                map<string,string>      from deserializer   
pageurl                 string                  from deserializer   
useragent               string                  from deserializer   
yyid                    string                  from deserializer   
suv                     string                  from deserializer   
line                    string                  from deserializer   
applogs                 array<struct<productid:string,version:string,brand:string,model:string,device:string,product:string,sdk:string,os:string,deviceid:string,net:string,channel:string,w:string,h:string,imsi:string,source_channel:string,reserved_field0:string,reserved_field1:string,reserved_field2:string,reserved_field3:string,reserved_field4:string,type:string,parent:string,content:string,ts:string,extra:string>>
      from deserializer   
                 
# Partition Information          
# col_name              data_type               comment             
                 
logdate                 string                                      
                 
# Detailed Table Information             
Database:               default                  
Owner:                  hive                     
CreateTime:             Fri Nov 08 11:38:00 CST 2013     
LastAccessTime:         UNKNOWN                  
Protect Mode:           None                     
Retention:              0                        
Location:               viewfs://nsX/user/hive/warehouse/default.db/web/uigs/web_uigs_wapsearch
 
Table Type:             EXTERNAL_TABLE           
Table Parameters:                
        EXTERNAL                TRUE                
        last_modified_by        slave               
        last_modified_time      1414463853          
        transient_lastDdlTime   1414463853          
                 
# Storage Information            
SerDe Library:          com.custom.datacat.hive.DataCatSerde      
InputFormat:            com.custom.datadir.plugin.SymlinkLzoTextInputFormat       
OutputFormat:           org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat       
Compressed:             No                       
Num Buckets:            -1                       
Bucket Columns:         []                       
Sort Columns:           []                       
Storage Desc Params:             
        datacat.fieldInspector  applogs:com.custom.datacat.hive.DataCatListObjectInspector:\t&pbparams:com.custom.datacat.hive.DataCatMapObjectInspector
        datacat.lineInspector   com.custom.datacat.wapapp.WapAppSearchInspector: 
        serialization.format    1   

{noformat} 

>  HCatRecordSerDe  throw a IndexOutOfBoundsException 
> ----------------------------------------------------
>
>                 Key: HIVE-12303
>                 URL: https://issues.apache.org/jira/browse/HIVE-12303
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog
>    Affects Versions: 0.14.0, 1.2.1
>            Reporter: Xiaowei Wang
>            Assignee: Sushanth Sowmyan
>             Fix For: 1.2.1
>
>         Attachments: HIVE-12303.0.patch
>
>
> When access hive table using hcatlog in Pig,sometime it throws a exception !
> Exception
> {noformat}
> 2015-10-30 06:44:35,219 WARN [Thread-4] org.apache.hadoop.mapred.YarnChild: Exception
running child : org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error converting
read value to tuple
>         at org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
>         at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:59)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:204)
>         at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>         at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1892)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> Caused by: java.lang.IndexOutOfBoundsException: Index: 24, Size: 24
>         at java.util.ArrayList.rangeCheck(ArrayList.java:635)
>         at java.util.ArrayList.get(ArrayList.java:411)
>         at org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeStruct(HCatRecordSerDe.java:175)
>         at org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeList(HCatRecordSerDe.java:244)
>         at org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:196)
>         at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
>         at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
>         at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
>         at org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:63)
>         ... 13 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message