mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Lyubimov <dlie...@gmail.com>
Subject Re: where i can set -Dmapred.map.tasks=X
Date Mon, 03 Jan 2011 23:17:48 GMT
Jeff,
now it stopped complaining about first -D but started doing so about the
second one.

bin/mahout seqdirectory -Dmapred.job.tracker=local -Dfs.default.name=file:///
-c UTF-8 -i /home/dmitriy/projects/testcollections/reuters-extracted/ -o
/home/dmitriy/projects/testcollections/reuters-seqfiles
Running on hadoop, using HADOOP_HOME=/home/dmitriy/tools/hadoop
No HADOOP_CONF_DIR set, using /home/dmitriy/tools/hadoop/conf
11/01/03 15:16:13 ERROR text.SequenceFilesFromDirectory: Exception
org.apache.commons.cli2.OptionException: Unexpected -Dfs.default.name=file:///
while processing Options
        at org.apache.commons.cli2.commandline.Parser.parse(Parser.java:99)
        at
org.apache.mahout.text.SequenceFilesFromDirectory.main(SequenceFilesFromDirectory.java:201)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at
org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:183)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:186)


On Mon, Jan 3, 2011 at 1:04 PM, Jeff Eastman <jdog@windwardsolutions.com>wrote:

> Yes, I committed a small patch on the 29th. Try a new trunk build.
>
>
> On 1/3/11 12:37 PM, Dmitriy Lyubimov wrote:
>
>> Hi Jeff,
>>
>> so did you get around to fixing this? i am having this little bugger all
>> over the place, including book examples that don't work directly if i have
>> hadoop setup on my machine such as in the following:
>>
>> bin/mahout seqdirectory -Dmapred.job.tracker=local -Dfs.default.name
>> =file:///
>> -c UTF-8 -i /home/dmitriy/projects/testcollections/reuters-extracted/ -o
>> /home/dmitriy/projects/testcollections/reuters-seqfiles
>> Running on hadoop, using HADOOP_HOME=/home/dmitriy/tools/hadoop
>> No HADOOP_CONF_DIR set, using /home/dmitriy/tools/hadoop/conf
>> 11/01/03 12:32:06 ERROR text.SequenceFilesFromDirectory: Exception
>> org.apache.commons.cli2.OptionException: Unexpected
>> -Dmapred.job.tracker=local while processing Options
>>         at
>> org.apache.commons.cli2.commandline.Parser.parse(Parser.java:99)
>>         at
>>
>> org.apache.mahout.text.SequenceFilesFromDirectory.main(SequenceFilesFromDirectory.java:187)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at
>>
>> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>>         at
>> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>>         at
>> org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:182)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>>
>>
>> Thanks.
>> -Dmitriy
>>
>> On Wed, Dec 29, 2010 at 11:50 AM, Jeff Eastman<jeastman@narus.com>
>>  wrote:
>>
>>  The patch to MahoutDriver involves the code in the for loop at lines
>>> 203-216. If the arg.startsWith("-D") then the arg needs to be added to
>>> argsList at position 1, else at the end. I will commit a patch for this
>>> tonight as I have not got my Narus CLA signed yet.
>>>
>>> -----Original Message-----
>>> From: Dmitriy Lyubimov [mailto:dlieu.7@gmail.com]
>>> Sent: Wednesday, December 29, 2010 11:46 AM
>>> To: user@mahout.apache.org
>>> Cc: dev@mahout.apache.org
>>> Subject: Re: where i can set -Dmapred.map.tasks=X
>>>
>>> ok, thank you, Jeff. Good to know. I actually expected to rely on this
>>> for
>>> a
>>> wide range of issues (most common being task jvm parameters override).
>>>
>>>
>>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message