hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Stack <st...@duboce.net>
Subject Re: implementing selection/projection using mapreduce
Date Fri, 05 Dec 2008 05:22:15 GMT
The #configure method gets called with JobConf before your map task 
starts up.  Get what you need from JobConf at that time and save them 
off into data members.  Then pull on them inside in your map.  See 
GroupingTableMap in mapred package for example or the examples in hadoop.

St.Ack


abhinit wrote:
> Thanks. ONe more thing.
>
> How do you get the current configuration object in the map method.
> I couldn't find it.
>
> Thanks,
> -Abhinit
>
>
> Try passing the variables into the mapper through the JobConf.set
> and .get methods.
>
> On Dec 4, 2008, at 3:45 AM, abhinit wrote:
>
>   
>> Hi,
>>
>> I am implementing basic selection/projection using mapreduce on an
>> HBase
>> table.
>>
>> I have an outer class SelectProject which implements the tool
>> interface and
>> my mapper class,
>> SelectProjectOperator extends TableMap.
>>
>> In my run method, I am parsing the command line argument for things
>> like
>> column names
>> and predicate condition(say key = 10000 etc.) and I have to use
>> these values
>> inside the map
>> method of my mapper class. All the variables to be used in the map
>> function
>> for selection/projection
>> are static variables of the enclosing outer class(SelectProject). I
>> et these
>> variables in run method
>> after parsing the comman-line arguments.
>>
>> However, in the map method I get a NullPointerException because non-
>> of those
>> static member variables
>> are created. I understand this is not exactlt an HBase related issue
>> and has
>> something to do with java inner classes.
>> But I thought this can be a very common case in mapreduce programming
>> (passing a condition variable from the
>> outer class to enclosed mapper class' map method).
>>
>> Thanks a lot for help.
>> -Abhinit
>>
>> --
>>     
>
>   


Mime
View raw message