drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdel Hakim Deneche <adene...@maprtech.com>
Subject Re: Convert Hbase Values from Bytes to Actual Strings
Date Fri, 07 Aug 2015 15:10:12 GMT
>> Are
>> we suffering performance hits when we do queries on 60+ columns?
>
> No.  Extra information referenced in the view is pruned away if it isn't
> directly in your query so you don't pay any penalty for excess complexity
> described in the view.

Unless it's a bug of course, like DRILL-2412
<https://issues.apache.org/jira/browse/DRILL-3412>. If you see any
performance hits, check the query's plan, if the scan is reading more
columns than needed it may be a bug.


On Thu, Aug 6, 2015 at 5:22 PM, Ted Dunning <ted.dunning@gmail.com> wrote:

> On Sat, Aug 1, 2015 at 5:53 PM, John Omernik <john@omernik.com> wrote:
>
> > I am working with this as well and I find the "abstraction" frustrating
> > from a user perspective as well.
>
>
> Yes.  It can be a pain.
>
>
> > I understand we can create views, I've
> > done that, but it's a tedious process of ensuring my syntax is correct
> and
> > one that I did have the Schema on hand to easily do this.  Could a
> feature
> > be added to "register" a view doing this for us based on the schema" ?
>
>
> It should be relatively easy to build a tool to convert schema information
> that you have into a view definition.
>
>
>
> > Are
> > we suffering performance hits when we do queries on 60+ columns?
>
>
> No.  Extra information referenced in the view is pruned away if it isn't
> directly in your query so you don't pay any penalty for excess complexity
> described in the view.
>
>
> > This
> > would make this integration much more intuitive for users who know sql,
> > know their data is in an hbase table, and just want to go query. (Trying
> to
> > make this more self serve for users)
> >
>
> Yes. A GUI would be a nice-to-have here.  It isn't clear if this will be
> high enough priority to happen right away.
>
>
>
>
> >
> > John
> >
> > On Sat, Aug 1, 2015 at 2:53 AM, Usman Ali <usman.ali@platalytics.com>
> > wrote:
> >
> > > I understand creating views and querying on them. But my problem is, I
> > don
> > > not have schema of the table, therefore I can not create views on the
> > > table. The only thing I know about the table is that each column
> > contains a
> > > "utf-8" encoded string. Now what I want to do to is to query the table
> > (not
> > > the view) through Restful APIs and get column values in bytes in my
> > > response json. Then I want to convert those  byte values to my original
> > > string. For eample my rresponse json is :
> > > {
> > >       myFamily: "{ "name" : "name string  in bytes" , "address" :
> > "address
> > >  string in bytes"}'
> > >
> > > }
> > >
> > > Now , I want to convert byte values of name and address to original
> > string.
> > > I hope I am clear in stating my problem.
> > > Any other work around this problem is also welcome.
> > >
> > > Thanks,
> > > Usman Ali
> > >
> > >
> > > On Sat, Aug 1, 2015 at 8:12 AM, Jacques Nadeau <jacques@dremio.com>
> > wrote:
> > >
> > > > Got it.  Was just concerned that the person might be confused by your
> > > > example using CAST when we were talking about CONVERT.  I think we
> need
> > > to
> > > > come up with a simpler/clearer way to express the differences since
> > > people
> > > > keep tripping up on this.
> > > >
> > > > --
> > > > Jacques Nadeau
> > > > CTO and Co-Founder, Dremio
> > > >
> > > > On Fri, Jul 31, 2015 at 7:59 PM, Carol McDonald <
> > cmcdonald@maprtech.com>
> > > > wrote:
> > > >
> > > > > the point I was trying to make is :
> > > > > The good news is, Drill does provide a nice simple way to abstract
> > > these
> > > > > details away.  You simply create a view on top of HBase [1].  The
> > view
> > > > can
> > > > > contain the physical conversions.  Then users can interact with the
> > > view
> > > > > rather than the underlying table.
> > > > >
> > > > > On Fri, Jul 31, 2015 at 11:36 AM, Jacques Nadeau <
> jacques@dremio.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Carol, your statements are misleading.  There are some situations
> > > where
> > > > > > people have stored in HBase encoded as text.  In other cases,
> > they've
> > > > > used
> > > > > > a straight binary encoding.  CAST is used generally used when
the
> > > data
> > > > is
> > > > > > already encoded as a UTF8 string.  Otherwise, the user should
use
> > > > > > CONVERT_FROM to decode the binary value.
> > > > > >
> > > > > > --
> > > > > > Jacques Nadeau
> > > > > > CTO and Co-Founder, Dremio
> > > > > >
> > > > > > On Fri, Jul 31, 2015 at 7:42 AM, Carol McDonald <
> > > > cmcdonald@maprtech.com>
> > > > > > wrote:
> > > > > >
> > > > > > > This blog shows an  example using REST , drill , and HBase.
In
> > this
> > > > > > example
> > > > > > > a view converts the HBase byte values  to strings
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://www.mapr.com/blog/how-use-sql-hadoop-drill-rest-json-nosql-and-hbase-simple-rest-client
> > > > > > >
> > > > > > > 0: jdbc:drill:> use dfs.mydata;
> > > > > > >
> > > > > > > create or replace view prodview as SELECT CAST(row_key
AS
> > INTEGER)
> > > AS
> > > > > > > prod_id, CAST(t.details.category AS VARCHAR(40)) AS category,
> > > > > > > CAST(t.details.name AS VARCHAR(129)) AS name FROM
> > maprdb.products
> > > t;
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Jul 28, 2015 at 11:35 AM, Usman Ali <
> > > > usman.ali@platalytics.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >      When we  query an Hbase table  like "select *
from
> > > > > > > > hbasePluginName.`tableName`" using Restful APIs, the
json it
> > > > returns
> > > > > > > > contains  values in the form of bytes. I know that
all the
> > values
> > > > in
> > > > > my
> > > > > > > > hbase table were originally of String form but I do
not know
> > > exact
> > > > > > schema
> > > > > > > > (column families and column names) of my table. How
can I
> > convert
> > > > > those
> > > > > > > > bytes stored in strings back to my original strings.
> > > > > > > >
> > > > > > > > Thanks.
> > > > > > > > Usman Ali
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>



-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message