whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Ramakrishnan <sinchronized.a...@gmail.com>
Subject Re: issues upgrading to 0.6 from 0.5
Date Fri, 07 Oct 2011 22:25:04 GMT
yup,  I did a "mvn clean install assembly:assembly" on a  fresh checkout
with it pointing to a current build of jclouds 1.2.0-SNAPSHOT and
jsr305-1.3.9 added as a dependency to whirr-core.
And I get the same errors while using the command line.

thanks
Arun

On Fri, Oct 7, 2011 at 5:29 AM, Andrei Savu <savu.andrei@gmail.com> wrote:

> Looks like a guava classpath ordering issue. Are you seeing this when
> building a clean checkout?
> On Oct 7, 2011 2:55 AM, "Arun Ramakrishnan" <sinchronized.arun@gmail.com>
> wrote:
>
>> Alas! While trying to run from the command line , I run into the following
>> issue. Any ideas ?
>>
>> ########################
>> whirr-0.7.0-SNAPSHOT>bin/whirr launch-cluster --config
>> ../whirr_cdh.properties
>> Exception in thread "main" com.google.common.collect.ComputationException:
>> java.lang.NoClassDefFoundError: com/google/common/cache/CacheLoader
>>     at
>> com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference.compute(ComputingConcurrentHashMap.java:325)
>>     at
>> com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.compute(ComputingConcurrentHashMap.java:140)
>>     at
>> com.google.common.collect.ComputingConcurrentHashMap.apply(ComputingConcurrentHashMap.java:71)
>>     at
>> com.google.common.collect.MapMaker$ComputingMapAdapter.get(MapMaker.java:848)
>>     at org.apache.whirr.service.ComputeCache.apply(ComputeCache.java:66)
>>     at org.apache.whirr.service.ComputeCache.apply(ComputeCache.java:57)
>>     at
>> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:62)
>>     at
>> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:106)
>>     at
>> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:62)
>>     at org.apache.whirr.cli.Main.run(Main.java:64)
>>     at org.apache.whirr.cli.Main.main(Main.java:97)
>> Caused by: java.lang.NoClassDefFoundError:
>> com/google/common/cache/CacheLoader
>>     at
>> org.jclouds.aws.ec2.AWSEC2ContextBuilder.addClientModule(AWSEC2ContextBuilder.java:42)
>>     at
>> org.jclouds.rest.RestContextBuilder.addClientModuleIfNotPresent(RestContextBuilder.java:267)
>>     at
>> org.jclouds.rest.RestContextBuilder.buildInjector(RestContextBuilder.java:179)
>>     at
>> org.jclouds.compute.ComputeServiceContextBuilder.buildInjector(ComputeServiceContextBuilder.java:52)
>>     at
>> org.jclouds.compute.ComputeServiceContextBuilder.buildComputeServiceContext(ComputeServiceContextBuilder.java:65)
>>     at
>> org.jclouds.compute.ComputeServiceContextFactory.buildContextUnwrappingExceptions(ComputeServiceContextFactory.java:70)
>>     at
>> org.jclouds.compute.ComputeServiceContextFactory.createContext(ComputeServiceContextFactory.java:121)
>>     at org.apache.whirr.service.ComputeCache$1.apply(ComputeCache.java:82)
>>     at org.apache.whirr.service.ComputeCache$1.apply(ComputeCache.java:72)
>>     at
>> com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference.compute(ComputingConcurrentHashMap.java:316)
>>     ... 10 more
>> Caused by: java.lang.ClassNotFoundException:
>> com.google.common.cache.CacheLoader
>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>     ... 20 more
>> ###########################################################
>>
>> On Thu, Oct 6, 2011 at 3:51 PM, Arun Ramakrishnan <
>> sinchronized.arun@gmail.com> wrote:
>>
>>> Hurray! Then I included jsr305 in whirr-core and everything builds fine.
>>>
>>> While following your suggestion, I deleted my ~/.m2 directory and did a
>>> mvn clean install. Is that what you meant by clean the cache ? . I still had
>>> the same problem.
>>>
>>> here is some info about my environment.
>>> #######################################
>>> /Users/arun/workspace/whirr-trunk>mvn --version
>>> Apache Maven 3.0.3 (r1075438; 2011-02-28 09:31:09-0800)
>>> Maven home: /usr/share/maven
>>> Java version: 1.6.0_26, vendor: Apple Inc.
>>> Java home:
>>> /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
>>> Default locale: en_US, platform encoding: MacRoman
>>> OS name: "mac os x", version: "10.7.1", arch: "x86_64", family: "mac"
>>> #######################################
>>>
>>>
>>> thanks
>>> Arun
>>>
>>>
>>>
>>> On Thu, Oct 6, 2011 at 1:41 PM, Andrei Savu <savu.andrei@gmail.com>wrote:
>>>
>>>> It seems like this is somehow related to Guava.
>>>>
>>>> http://stackoverflow.com/questions/3800033/guava-r07-gwt-and-javax-annotation-nullable
>>>>
>>>> Can you retry to build Whirr after removing the maven cache? I'm not
>>>> sure this is the problem you are seeing.
>>>>
>>>> -- Andrei Savu
>>>>
>>>> On Thu, Oct 6, 2011 at 11:40 PM, Andrei Savu <savu.andrei@gmail.com>wrote:
>>>>
>>>>> I think I've seen this in the past on a different project. Can you tell
>>>>> me more about os, IDE, jdk version and maven version?
>>>>>
>>>>> -- Andrei Savu
>>>>>
>>>>> On Thu, Oct 6, 2011 at 10:21 PM, Arun Ramakrishnan <
>>>>> sinchronized.arun@gmail.com> wrote:
>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> I applied the patch to jclouds trunk and I ran into this problem.
I
>>>>>> didn't spend too much time investigating it. But, let me know if
you an idea
>>>>>> whats wrong.
>>>>>>
>>>>>> #############################################################
>>>>>> [INFO] Apache Whirr Core ................................. FAILURE
>>>>>> [7.074s]
>>>>>>
>>>>>> ...
>>>>>>
>>>>>> [ERROR] Failed to execute goal
>>>>>> org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile
>>>>>> (default-compile) on project whirr-core: Compilation failure: Compilation
>>>>>> failure:
>>>>>> [ERROR]
>>>>>> /Users/arun/workspace/whirr-trunk/core/src/main/java/org/apache/whirr/RolePredicates.java:[29,23]
>>>>>> cannot find symbol
>>>>>> [ERROR] symbol  : class Nullable
>>>>>> [ERROR] location: package javax.annotation
>>>>>> [ERROR]
>>>>>> [ERROR]
>>>>>> /Users/arun/workspace/whirr-trunk/core/src/main/java/org/apache/whirr/RolePredicates.java:[108,28]
>>>>>> cannot find symbol
>>>>>> [ERROR] symbol: class Nullable
>>>>>> [ERROR] public boolean apply(@Nullable Instance input) {
>>>>>> [ERROR] -> [Help 1]
>>>>>> ###########################################################
>>>>>>
>>>>>> thanks
>>>>>> Arun
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 4, 2011 at 10:45 PM, Andrei Savu <savu.andrei@gmail.com>wrote:
>>>>>>
>>>>>>> mvn package assembly:assembly will generate the release tarball
from
>>>>>>> the source code.
>>>>>>>
>>>>>>> -- Andrei
>>>>>>>
>>>>>>> On Wed, Oct 5, 2011 at 4:07 AM, Arun Ramakrishnan <
>>>>>>> sinchronized.arun@gmail.com> wrote:
>>>>>>>
>>>>>>>> Its always been less than 10 security groups in total. I
will give
>>>>>>>> this a try however.
>>>>>>>> I am able to "mvn install" successfully in whirr-trunk. Is
there a
>>>>>>>> maven goal or a script that does the final packaging for
command line use ?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Oct 4, 2011 at 2:08 PM, Tom White <tom@cloudera.com>
wrote:
>>>>>>>>
>>>>>>>>> Hi Arun,
>>>>>>>>>
>>>>>>>>> I had this issue too - I think it is a consequence of
using an
>>>>>>>>> account
>>>>>>>>> with a large number of existing groups. There's a fix
in
>>>>>>>>> https://issues.apache.org/jira/browse/WHIRR-381 that
you might
>>>>>>>>> like to
>>>>>>>>> try - this will be fixed in the next release.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Tom
>>>>>>>>>
>>>>>>>>> On Tue, Oct 4, 2011 at 1:30 PM, Arun Ramakrishnan
>>>>>>>>> <sinchronized.arun@gmail.com> wrote:
>>>>>>>>> > I am having this issue upgrading to 0.6.
>>>>>>>>> >
>>>>>>>>> > During the security group creation, the internal
access rules
>>>>>>>>> within the
>>>>>>>>> > cluster don't get added, consequentially blocking
the datanodes
>>>>>>>>> from being
>>>>>>>>> > able to connect to the namenode. I added these rules
manually and
>>>>>>>>> it seems
>>>>>>>>> > fine.
>>>>>>>>> >
>>>>>>>>> > This bug recurs for me consistently in my environment.
>>>>>>>>> >
>>>>>>>>> > log
>>>>>>>>> > ########################################################
>>>>>>>>> >
>>>>>>>>> > 2011-10-04 12:23:20,949 DEBUG [jclouds.compute]
(pool-3-thread-4)
>>>>>>>>> >>
>>>>>>>>> > creating keyPair region(us-east-1) group(ccore27)
>>>>>>>>> > 2011-10-04 12:23:20,949 DEBUG [jclouds.compute]
(pool-3-thread-2)
>>>>>>>>> >>
>>>>>>>>> > creating keyPair region(us-east-1) group(ccore27)
>>>>>>>>> > 2011-10-04 12:23:21,414 DEBUG [jclouds.compute]
(pool-3-thread-4)
>>>>>>>>> << created
>>>>>>>>> > keyPair(jclouds#ccore27#us-east-1#72)
>>>>>>>>> > 2011-10-04 12:23:21,414 DEBUG [jclouds.compute]
(pool-3-thread-4)
>>>>>>>>> >>
>>>>>>>>> > creating securityGroup region(us-east-1)
>>>>>>>>> name(jclouds#ccore27#us-east-1)
>>>>>>>>> > 2011-10-04 12:23:21,692 DEBUG [jclouds.compute]
(pool-3-thread-4)
>>>>>>>>> << created
>>>>>>>>> > securityGroup(jclouds#ccore27#us-east-1)
>>>>>>>>> > 2011-10-04 12:23:21,692 DEBUG [jclouds.compute]
(pool-3-thread-4)
>>>>>>>>> >>
>>>>>>>>> > authorizing securityGroup region(us-east-1)
>>>>>>>>> name(jclouds#ccore27#us-east-1)
>>>>>>>>> > port(22)
>>>>>>>>> > 2011-10-04 12:23:21,926 DEBUG [jclouds.compute]
(pool-3-thread-4)
>>>>>>>>> <<
>>>>>>>>> > authorized securityGroup(jclouds#ccore27#us-east-1)
>>>>>>>>> > 2011-10-04 12:23:21,926 DEBUG [jclouds.compute]
(pool-3-thread-4)
>>>>>>>>> >>
>>>>>>>>> > authorizing securityGroup region(us-east-1)
>>>>>>>>> name(jclouds#ccore27#us-east-1)
>>>>>>>>> > permission to itself
>>>>>>>>> > 2011-10-04 12:23:22,306 ERROR
>>>>>>>>> > [org.apache.whirr.actions.BootstrapClusterAction]
>>>>>>>>> (pool-3-thread-3)
>>>>>>>>> > Unexpected error while starting 4 nodes, minimum
4 nodes for
>>>>>>>>> > [hadoop-datanode, hadoop-tasktracker] of cluster
ccore27
>>>>>>>>> > java.util.concurrent.ExecutionException:
>>>>>>>>> java.lang.RuntimeException:
>>>>>>>>> > request: POST https://ec2.us-east-1.amazonaws.com/
HTTP/1.1;
>>>>>>>>> cause:
>>>>>>>>> > java.lang.NullPointerException
>>>>>>>>> >     at
>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>>>>>>> >     at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>>>>>>> >     at
>>>>>>>>> >
>>>>>>>>> org.apache.whirr.actions.BootstrapClusterAction$StartupProcess.waitForOutcomes(BootstrapClusterAction.java:320)
>>>>>>>>> >     at
>>>>>>>>> >
>>>>>>>>> org.apache.whirr.actions.BootstrapClusterAction$StartupProcess.call(BootstrapClusterAction.java:273)
>>>>>>>>> >     at
>>>>>>>>> >
>>>>>>>>> org.apache.whirr.actions.BootstrapClusterAction$StartupProcess.call(BootstrapClusterAction.java:234)
>>>>>>>>> >     at
>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>>>> >     at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>>>> >     at
>>>>>>>>> >
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>> >     at
>>>>>>>>> >
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>> >     at java.lang.Thread.run(Thread.java:680)
>>>>>>>>> > Caused by: java.lang.RuntimeException: request:
POST
>>>>>>>>> > https://ec2.us-east-1.amazonaws.com/ HTTP/1.1; cause:
>>>>>>>>> > java.lang.NullPointerException
>>>>>>>>> >     at
>>>>>>>>> >
>>>>>>>>> org.jclouds.http.functions.ParseSax.addDetailsAndPropagate(ParseSax.java:152)
>>>>>>>>> >     at
>>>>>>>>> org.jclouds.http.functions.ParseSax.parse(ParseSax.java:116)
>>>>>>>>> >     at
>>>>>>>>> org.jclouds.http.functions.ParseSax.apply(ParseSax.java:78)
>>>>>>>>> >     at
>>>>>>>>> org.jclouds.http.functions.ParseSax.apply(ParseSax.java:51)
>>>>>>>>> >     at
>>>>>>>>> com.google.common.util.concurrent.Futures$4.apply(Futures.java:439)
>>>>>>>>> >     at
>>>>>>>>> com.google.common.util.concurrent.Futures$4.apply(Futures.java:437)
>>>>>>>>> >     at
>>>>>>>>> >
>>>>>>>>> com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:713)
>>>>>>>>> >     at
>>>>>>>>> >
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>> >     at
>>>>>>>>> >
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>> >     at java.lang.Thread.run(Thread.java:680)
>>>>>>>>> >     at
>>>>>>>>> >
>>>>>>>>> org.jclouds.http.functions.ParseSax.addDetailsAndPropagate(ParseSax.java:152)
>>>>>>>>> >     at
>>>>>>>>> org.jclouds.http.functions.ParseSax.apply(ParseSax.java:80)
>>>>>>>>> >     at
>>>>>>>>> org.jclouds.http.functions.ParseSax.apply(ParseSax.java:51)
>>>>>>>>> >     at
>>>>>>>>> com.google.common.util.concurrent.Futures$4.apply(Futures.java:439)
>>>>>>>>> >     at
>>>>>>>>> com.google.common.util.concurrent.Futures$4.apply(Futures.java:437)
>>>>>>>>> >     at
>>>>>>>>> >
>>>>>>>>> com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:713)
>>>>>>>>> >     ... 3 more
>>>>>>>>> >
>>>>>>>>> > ###########################################################
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > This are the rules that existed for the cluster
created with
>>>>>>>>> whirr 0.6
>>>>>>>>> > GROUP    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>> > jclouds#ccore27#us-east-1
>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>> ALLOWS    tcp
>>>>>>>>> > 22    22    FROM    CIDR    0.0.0.0/0
>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>> ALLOWS    tcp
>>>>>>>>> > 8020    8020    FROM    CIDR    184.72.183.32/32
>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>> ALLOWS    tcp
>>>>>>>>> > 8021    8021    FROM    CIDR    184.72.183.32/32
>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>> ALLOWS    tcp
>>>>>>>>> > 50030    50030    FROM    CIDR    24.43.39.218/32
>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>> ALLOWS    tcp
>>>>>>>>> > 50070    50070    FROM    CIDR    24.43.39.218/32
>>>>>>>>> >
>>>>>>>>> > I had to manually add
>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>> ALLOWS    all
>>>>>>>>> >         FROM    USER    673040621396    GRPNAME
>>>>>>>>> jclouds#ccore27#us-east-1
>>>>>>>>> > to allow the datanodes to talk to the namenode.
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > Here is my config file with updated property names
to match 0.6
>>>>>>>>> > ##########################################
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > whirr.cluster-name=ccore27
>>>>>>>>> > whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,4
>>>>>>>>> > hadoop-datanode+hadoop-tasktracker
>>>>>>>>> >
>>>>>>>>> > whirr.provider=aws-ec2
>>>>>>>>> > whirr.identity=*************************
>>>>>>>>> > whirr.credential=**********************
>>>>>>>>> > whirr.private-key-file=/Users/arun/.ec2/hadoopkey
>>>>>>>>> > whirr.public-key-file=/Users/arun/.ec2/hadoopkey.pub
>>>>>>>>> > whirr.client-cidrs=24.43.39.218/32
>>>>>>>>> >
>>>>>>>>> > whirr.location-id=us-east-1
>>>>>>>>> > whirr.hardware-id=c1.xlarge
>>>>>>>>> > #c1.xlarge
>>>>>>>>> >
>>>>>>>>> > # Ubuntu 10.04 LTS Lucid. See http://alestic.com/
or
>>>>>>>>> > http://aws.amazon.com/amis/4348
>>>>>>>>> > # ebs root only
>>>>>>>>> > whirr.image-id=us-east-1/ami-4a0df923
>>>>>>>>> >
>>>>>>>>> > whirr.hadoop.install-function=install_cdh_hadoop
>>>>>>>>> > whirr.hadoop.configure-function=configure_cdh_hadoop
>>>>>>>>> >
>>>>>>>>> > #######################################
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>

Mime
View raw message