spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jai Kumar Singh (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-1452) dynamic partition creation not working on cached table
Date Wed, 09 Apr 2014 09:21:14 GMT

     [ https://issues.apache.org/jira/browse/SPARK-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jai Kumar Singh updated SPARK-1452:
-----------------------------------

    Description: 
dynamic partition creation via shark QL command not working with cached table. Though it works
fine with non-cached tables.
Also static partition is working fine with cached table. 


shark> desc sample;
OK
cid                     string                  None                
host                    string                  None                
url                     string                  None                
bytes                   int                     None                
pckts                   int                     None                
app                     string                  None                
cat                     string                  None                
Time taken: 0.149 seconds
shark> 

shark> desc sample_cached;
OK
cat                     string                  from deserializer   
host                    string                  from deserializer   
cid                     string                  None                
                 
# Partition Information          
# col_name              data_type               comment             
                 
cid                     string                  None                
Time taken: 0.15 seconds
shark> 

shark> insert into table sample_cached partition(cid) select cat,host,cid from sample;
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
shark> 

shark> insert into table sample_cached partition(cid="my-cid") select cat,host from sample
limit 20;
java.lang.InstantiationException: scala.Some
Continuing ...
java.lang.RuntimeException: failed to evaluate: <unbound>=Class.new();
Continuing ...
Loading data to table default.sample_cached partition (cid=my-cid)
OK
Time taken: 64.268 seconds



I am logging this issue over here because https://spark-project.atlassian.net/browse/SHARK/?selectedTab=com.atlassian.jira.jira-projects-plugin:issues-panel
not allowing me to log the issue there.

  was:
dynamic partition creation via shark QL command not working with cached table. Though it works
fine with non-cached tables.

shark> desc sample;
OK
cid                     string                  None                
host                    string                  None                
url                     string                  None                
bytes                   int                     None                
pckts                   int                     None                
app                     string                  None                
cat                     string                  None                
Time taken: 0.149 seconds
shark> 

shark> desc sample_cached;
OK
cat                     string                  from deserializer   
host                    string                  from deserializer   
cid                     string                  None                
                 
# Partition Information          
# col_name              data_type               comment             
                 
cid                     string                  None                
Time taken: 0.15 seconds
shark> 

shark> insert into table sample_cached partition(cid) select cat,host,cid from sample;
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
shark> 

shark> insert into table sample_cached partition(cid="my-cid") select cat,host from sample
limit 20;
java.lang.InstantiationException: scala.Some
Continuing ...
java.lang.RuntimeException: failed to evaluate: <unbound>=Class.new();
Continuing ...
Loading data to table default.sample_cached partition (cid=my-cid)
OK
Time taken: 64.268 seconds



> dynamic partition creation not working on cached table
> ------------------------------------------------------
>
>                 Key: SPARK-1452
>                 URL: https://issues.apache.org/jira/browse/SPARK-1452
>             Project: Spark
>          Issue Type: Bug
>    Affects Versions: 0.9.0
>         Environment: Shark git 
> commit dfc0e81366c0e1d0293ecf9b490eeabcc2a9c904
> Merge: 517ebca 7652f0d
>            Reporter: Jai Kumar Singh
>              Labels: Shark
>
> dynamic partition creation via shark QL command not working with cached table. Though
it works fine with non-cached tables.
> Also static partition is working fine with cached table. 
> shark> desc sample;
> OK
> cid                     string                  None                
> host                    string                  None                
> url                     string                  None                
> bytes                   int                     None                
> pckts                   int                     None                
> app                     string                  None                
> cat                     string                  None                
> Time taken: 0.149 seconds
> shark> 
> shark> desc sample_cached;
> OK
> cat                     string                  from deserializer   
> host                    string                  from deserializer   
> cid                     string                  None                
>                  
> # Partition Information          
> # col_name              data_type               comment             
>                  
> cid                     string                  None                
> Time taken: 0.15 seconds
> shark> 
> shark> insert into table sample_cached partition(cid) select cat,host,cid from sample;
> FAILED: Hive Internal Error: java.lang.NullPointerException(null)
> shark> 
> shark> insert into table sample_cached partition(cid="my-cid") select cat,host from
sample limit 20;
> java.lang.InstantiationException: scala.Some
> Continuing ...
> java.lang.RuntimeException: failed to evaluate: <unbound>=Class.new();
> Continuing ...
> Loading data to table default.sample_cached partition (cid=my-cid)
> OK
> Time taken: 64.268 seconds
> I am logging this issue over here because https://spark-project.atlassian.net/browse/SHARK/?selectedTab=com.atlassian.jira.jira-projects-plugin:issues-panel
not allowing me to log the issue there.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message