gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From alx...@aim.com
Subject Re: gora-hbase query
Date Thu, 28 Feb 2013 01:54:30 GMT

I am mostly interested in fetcher job. In this job I see this code

StorageUtils.initMapperJob(currentJob, fields, IntWritable.class,  FetchEntry.class, FetcherMapper.class,
FetchEntryPartitioner.class, false);

In storage utils this function has

DataStore<String, WebPage> store = createWebStore(job.getConfiguration(), String.class,
    if (store==null) throw new RuntimeException("Could not create datastore");
 Query<String, WebPage> query = store.newQuery();
 GoraMapper.initMapperJob(job, query, store, outKeyClass, outValueClass, mapperClass, partitionerClass,

I followed all these functions but did not find actual code that sends query to hbase table.
I believe it is somewhere in gora-hbase.




-----Original Message-----
From: Renato MarroquĂ­n Mog
 rovejo <renatoj.marroquin@gmail.com>
To: Gora Dev <dev@gora.apache.org>
Sent: Tue, Feb 26, 2013 8:01 pm
Subject: Re: gora-hbase query

Hi Alex,

The Gora-HBase module is only in charge of querying and persisting
data from any where, not only Nutch. That being said, you want the
part where Nutch populates a map used in different Nutch jobs? Which
jobs are you talking about? Generator? Fetcher? You can probably get
some more lights over in NutchLand.
I am happy to go over the code with you anyways, just please be a
little bit more specific.

Renato M.

2013/2/26  <alxsss@aim.com>:
> Hello,
> Can someone point me the code in gora-hbase that queries hbase and populates 
nutch map key values for varies nutch jobs?
> I plan to use SingleColumnValueFilter to see if it selects only subset of 
> Thanks.
> Alex.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message