drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andries Engelbrecht <aengelbre...@maprtech.com>
Subject Re: Drill dir0 issue
Date Sun, 23 Aug 2015 17:31:23 GMT
Hi Sungwook,

Can you create a different directory with a few files in each sub directory, but use 2012,
2013, 2014instead of year=2012, etc.
Might be a good test to see if the directory naming structure of year=xxxx is tripping up
Drill on directory pruning.

—Andries


> On Aug 23, 2015, at 9:47 AM, Kristine Hahn <khahn@maprtech.com> wrote:
> 
> If you set up your data in directories like the log data in the
> Querying Directories example on
> http://drill.apache.org/docs/querying-directories, which uses WHERE
> dir0='2013' LIMIT 10 in the query, and you are having intermittent
> Table Not Found results, look for hidden files in the directory you
> are querying. The files must be compatible--they must have comparable
> data types and columns in the same order. Hidden files that do not
> have comparable data types can cause a Table Not Found error.
> Kristine Hahn
> Sr. Technical Writer
> 415-497-8107 @krishahn skype:krishahn
> 
> 
> 
> On Sun, Aug 23, 2015 at 9:01 AM, USC <hsuanyi@usc.edu> wrote:
>> Hi Sungwook,
>> In your where clause, you only need to say year=2012.
>> 
>> The directory column (e.g., dir0) is used when users query a directory.
>> 
>> 
>>> On Aug 23, 2015, at 7:07 AM, Sungwook Yoon <syoon@maprtech.com> wrote:
>>> 
>>> Hi,
>>> 
>>> I am trying to use Hive parquet stored files partitioned by some column.
>>> So, the directory structure is partitioned with the column.
>>> 
>>> The column is actually year.
>>> Let's say there are 5 years, so dir0 are like year=2010,
>>> year=2011,year=2012,year=2013,year=2014
>>> 
>>> We did like following
>>> select * from dfs.root.`/user/hive/warehouse/table` d where d.dir0 =
>>> 'year=2012';
>>> 
>>> I get nothing.
>>> Apparently, there are parquet files in the directory though.
>>> 
>>> Sometimes it picks up e.g., year=2010, but not year=2012..
>>> 
>>> Where am I going wrong with this?
>>> 
>>> Thanks,
>>> 
>>> Sungwook


Mime
View raw message