hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kranthi reddy <kranthili2...@gmail.com>
Subject Re: Porting SQL DB into HBASE
Date Wed, 14 Apr 2010 09:08:41 GMT
Hi,

The amount of data being added is around 6-8GB per day. If we keep redundant
data the size increases exponentially and we are expecting it to increase by
atleast twice if not more.

Eg: Table 1 has 50 columns with unique entries and suppose "Column X" is the
primary key.
      Suppose we have Table 2 with 15 columns each with foreign key "Column
X".

      If for an entry "Y" in Table 1, we have 15 entries in Table 2 with
foreign key as "Y". Here we end up having 1 row in Table 1(50 cells filled)
and 15 rows in Table 2(15*15=225 cells filled).


    If these 2 tables are denormalized, we end up with 15 rows having
redundant data (15*50 cells + 15*15 cells = 975 cells filled).

Hope my example is clear.

Regards,
kranthi

On Wed, Apr 14, 2010 at 11:33 AM, Imran M Yousuf <imyousuf@gmail.com> wrote:

> On Mon, Apr 12, 2010 at 2:55 PM, kranthi reddy <kranthili2020@gmail.com>
> wrote:
> >
> > <snip />
> > The problem is denormalising these 20% tables is also extremely difficult
> > and we are planning to port them directly into hbase. And also
> denormalising
> > these tables would lead to a lot of redundant data.
> >
>
> When denormalisation is been mentioned, it is implied having redundant
> data. The idea is as there is no join instead of doing N lookups (to
> replace N joins) keeping redundant data will allow you to do a single
> lookup and furthermore, HBase is great in scaling huge data sets.
>
> When I started reading http://wiki.apache.org/hadoop/Hbase/FAQ#A20 it
> helped me understand it further.
>
> Hope this helps.
>
> Best regards,
>
> --
> Imran M Yousuf
> Entrepreneur & Software Engineer
> Smart IT Engineering
> Dhaka, Bangladesh
> Email: imran@smartitengineering.com
> Blog: http://imyousuf-tech.blogs.smartitengineering.com/
> Mobile: +880-1711402557
>



-- 
Kranthi Reddy. B

http://www.setusoftware.com/setu/index.htm

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message