hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: MOB integration
Date Sat, 19 May 2018 16:38:39 GMT
If you have a chance to look at region server log, you would see some line
such as the following:

2018-05-19 16:31:23,548 INFO  [MemStoreFlusher.0] regionserver.HMobStore:
Renaming flushed file from
hdfs://mycluster/apps/hbase/data/mobdir/.tmp/28e252d7f013973174750d483d358fa020180519dd8e7c3d67814eb0b5fb06fb9e800377
to
hdfs://mycluster/apps/hbase/data/mobdir/data/default/IntegrationTestIngestWithMOB/e9b5d936e7f55a4f1c3246a8d5ce53c2/test_cf/28e252d7f013973174750d483d358fa020180519dd8e7c3d67814eb0b5fb06fb9e800377

Meaning Mob store file is first saved under /apps/hbase/data/mobdir/.tmp/ ,
then renamed to under the usual location under region directory for the
table.

>From high level, as long as you can query what you ingested, you can be
assured that Mob data is persisted.

Cheers

On Sat, May 19, 2018 at 8:43 AM, Nicolas Paris <niparisco@gmail.com> wrote:

> Hi
>
> ​Yes the data comes back as expected.
> My table is not called "mlob" however since I found such folder
> I thought it was storing mob objects.
>
> I do have 500 folder hashed as you mentionned. They contains the
> whole dataset (2TO)
> However, how beeing sure the data is actually stored as MOB (and not
> as traditional binary)
>
> Thanks
>
>
> 2018-05-19 15:59 GMT+02:00 Ted Yu <yuzhihong@gmail.com>:
>
> > bq. look into hdfs hbase/data/mlob
> >
> > Is 'mlob' name of your table ?
> >
> > bq. nearly empty folder
> >
> > Here is listing under a one region table:
> >
> > drwxr-xr-x   - hbase hdfs          0 2018-05-16 23:51
> > /apps/hbase/data/data/default/atlas_janus/.tabledesc
> > drwxr-xr-x   - hbase hdfs          0 2018-05-16 23:51
> > /apps/hbase/data/data/default/atlas_janus/.tmp
> > drwxr-xr-x   - hbase hdfs          0 2018-05-17 00:55
> > /apps/hbase/data/data/default/atlas_janus/8033ea259cb7272d43bc137ca0ab29
> 06
> >
> > Not sure if the above matches your description of being nearly empty.
> > Here data is stored under 8033ea259cb7272d43bc137ca0ab2906
> >
> > If you query the table, does the data come back as expected ?
> >
> > Thanks
> >
> > On Sat, May 19, 2018 at 12:23 AM, Nicolas Paris <niparisco@gmail.com>
> > wrote:
> >
> > > Hi
> > >
> > >
> > > I am using hbase 1.1 and hive 1.2
> > >
> > > I created an hbase table with a mob column with the default
> > > threshold (100K)
> > > I mapped the table into hive with a binary format, and loaded
> > > 20M of pdf of size between 50k and 20mb
> > >
> > > Apparently the mob is not populated because when I look into
> > > hdfs hbase/data/mlob, it is a nearly empty folder.
> > >
> > > Does it mean hive cannot populate hbase mob columns  ?
> > >
> > > Thanks
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message