whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrei Savu <savu.and...@gmail.com>
Subject Re: issues upgrading to 0.6 from 0.5
Date Sat, 08 Oct 2011 09:16:43 GMT
I think that if you are upgrading jclouds you also need to upgrade the guava
version in Whirr (root pom.xml) to match the one used by jclouds.

-- Andrei Savu

On Sat, Oct 8, 2011 at 1:25 AM, Arun Ramakrishnan <
sinchronized.arun@gmail.com> wrote:

> yup,  I did a "mvn clean install assembly:assembly" on a  fresh checkout
> with it pointing to a current build of jclouds 1.2.0-SNAPSHOT and
> jsr305-1.3.9 added as a dependency to whirr-core.
> And I get the same errors while using the command line.
>
> thanks
> Arun
>
>
> On Fri, Oct 7, 2011 at 5:29 AM, Andrei Savu <savu.andrei@gmail.com> wrote:
>
>> Looks like a guava classpath ordering issue. Are you seeing this when
>> building a clean checkout?
>> On Oct 7, 2011 2:55 AM, "Arun Ramakrishnan" <sinchronized.arun@gmail.com>
>> wrote:
>>
>>> Alas! While trying to run from the command line , I run into the
>>> following issue. Any ideas ?
>>>
>>> ########################
>>> whirr-0.7.0-SNAPSHOT>bin/whirr launch-cluster --config
>>> ../whirr_cdh.properties
>>> Exception in thread "main"
>>> com.google.common.collect.ComputationException:
>>> java.lang.NoClassDefFoundError: com/google/common/cache/CacheLoader
>>>     at
>>> com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference.compute(ComputingConcurrentHashMap.java:325)
>>>     at
>>> com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.compute(ComputingConcurrentHashMap.java:140)
>>>     at
>>> com.google.common.collect.ComputingConcurrentHashMap.apply(ComputingConcurrentHashMap.java:71)
>>>     at
>>> com.google.common.collect.MapMaker$ComputingMapAdapter.get(MapMaker.java:848)
>>>     at org.apache.whirr.service.ComputeCache.apply(ComputeCache.java:66)
>>>     at org.apache.whirr.service.ComputeCache.apply(ComputeCache.java:57)
>>>     at
>>> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:62)
>>>     at
>>> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:106)
>>>     at
>>> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:62)
>>>     at org.apache.whirr.cli.Main.run(Main.java:64)
>>>     at org.apache.whirr.cli.Main.main(Main.java:97)
>>> Caused by: java.lang.NoClassDefFoundError:
>>> com/google/common/cache/CacheLoader
>>>     at
>>> org.jclouds.aws.ec2.AWSEC2ContextBuilder.addClientModule(AWSEC2ContextBuilder.java:42)
>>>     at
>>> org.jclouds.rest.RestContextBuilder.addClientModuleIfNotPresent(RestContextBuilder.java:267)
>>>     at
>>> org.jclouds.rest.RestContextBuilder.buildInjector(RestContextBuilder.java:179)
>>>     at
>>> org.jclouds.compute.ComputeServiceContextBuilder.buildInjector(ComputeServiceContextBuilder.java:52)
>>>     at
>>> org.jclouds.compute.ComputeServiceContextBuilder.buildComputeServiceContext(ComputeServiceContextBuilder.java:65)
>>>     at
>>> org.jclouds.compute.ComputeServiceContextFactory.buildContextUnwrappingExceptions(ComputeServiceContextFactory.java:70)
>>>     at
>>> org.jclouds.compute.ComputeServiceContextFactory.createContext(ComputeServiceContextFactory.java:121)
>>>     at
>>> org.apache.whirr.service.ComputeCache$1.apply(ComputeCache.java:82)
>>>     at
>>> org.apache.whirr.service.ComputeCache$1.apply(ComputeCache.java:72)
>>>     at
>>> com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference.compute(ComputingConcurrentHashMap.java:316)
>>>     ... 10 more
>>> Caused by: java.lang.ClassNotFoundException:
>>> com.google.common.cache.CacheLoader
>>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>     at java.security.AccessController.doPrivileged(Native Method)
>>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>>     ... 20 more
>>> ###########################################################
>>>
>>> On Thu, Oct 6, 2011 at 3:51 PM, Arun Ramakrishnan <
>>> sinchronized.arun@gmail.com> wrote:
>>>
>>>> Hurray! Then I included jsr305 in whirr-core and everything builds fine.
>>>>
>>>>
>>>> While following your suggestion, I deleted my ~/.m2 directory and did a
>>>> mvn clean install. Is that what you meant by clean the cache ? . I still
had
>>>> the same problem.
>>>>
>>>> here is some info about my environment.
>>>> #######################################
>>>> /Users/arun/workspace/whirr-trunk>mvn --version
>>>> Apache Maven 3.0.3 (r1075438; 2011-02-28 09:31:09-0800)
>>>> Maven home: /usr/share/maven
>>>> Java version: 1.6.0_26, vendor: Apple Inc.
>>>> Java home:
>>>> /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
>>>> Default locale: en_US, platform encoding: MacRoman
>>>> OS name: "mac os x", version: "10.7.1", arch: "x86_64", family: "mac"
>>>> #######################################
>>>>
>>>>
>>>> thanks
>>>> Arun
>>>>
>>>>
>>>>
>>>> On Thu, Oct 6, 2011 at 1:41 PM, Andrei Savu <savu.andrei@gmail.com>wrote:
>>>>
>>>>> It seems like this is somehow related to Guava.
>>>>>
>>>>> http://stackoverflow.com/questions/3800033/guava-r07-gwt-and-javax-annotation-nullable
>>>>>
>>>>> Can you retry to build Whirr after removing the maven cache? I'm not
>>>>> sure this is the problem you are seeing.
>>>>>
>>>>> -- Andrei Savu
>>>>>
>>>>> On Thu, Oct 6, 2011 at 11:40 PM, Andrei Savu <savu.andrei@gmail.com>wrote:
>>>>>
>>>>>> I think I've seen this in the past on a different project. Can you
>>>>>> tell me more about os, IDE, jdk version and maven version?
>>>>>>
>>>>>> -- Andrei Savu
>>>>>>
>>>>>> On Thu, Oct 6, 2011 at 10:21 PM, Arun Ramakrishnan <
>>>>>> sinchronized.arun@gmail.com> wrote:
>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> I applied the patch to jclouds trunk and I ran into this problem.
I
>>>>>>> didn't spend too much time investigating it. But, let me know
if you an idea
>>>>>>> whats wrong.
>>>>>>>
>>>>>>> #############################################################
>>>>>>> [INFO] Apache Whirr Core ................................. FAILURE
>>>>>>> [7.074s]
>>>>>>>
>>>>>>> ...
>>>>>>>
>>>>>>> [ERROR] Failed to execute goal
>>>>>>> org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile
>>>>>>> (default-compile) on project whirr-core: Compilation failure:
Compilation
>>>>>>> failure:
>>>>>>> [ERROR]
>>>>>>> /Users/arun/workspace/whirr-trunk/core/src/main/java/org/apache/whirr/RolePredicates.java:[29,23]
>>>>>>> cannot find symbol
>>>>>>> [ERROR] symbol  : class Nullable
>>>>>>> [ERROR] location: package javax.annotation
>>>>>>> [ERROR]
>>>>>>> [ERROR]
>>>>>>> /Users/arun/workspace/whirr-trunk/core/src/main/java/org/apache/whirr/RolePredicates.java:[108,28]
>>>>>>> cannot find symbol
>>>>>>> [ERROR] symbol: class Nullable
>>>>>>> [ERROR] public boolean apply(@Nullable Instance input) {
>>>>>>> [ERROR] -> [Help 1]
>>>>>>> ###########################################################
>>>>>>>
>>>>>>> thanks
>>>>>>> Arun
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Oct 4, 2011 at 10:45 PM, Andrei Savu <savu.andrei@gmail.com>wrote:
>>>>>>>
>>>>>>>> mvn package assembly:assembly will generate the release tarball
from
>>>>>>>> the source code.
>>>>>>>>
>>>>>>>> -- Andrei
>>>>>>>>
>>>>>>>> On Wed, Oct 5, 2011 at 4:07 AM, Arun Ramakrishnan <
>>>>>>>> sinchronized.arun@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Its always been less than 10 security groups in total.
I will give
>>>>>>>>> this a try however.
>>>>>>>>> I am able to "mvn install" successfully in whirr-trunk.
Is there a
>>>>>>>>> maven goal or a script that does the final packaging
for command line use ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Oct 4, 2011 at 2:08 PM, Tom White <tom@cloudera.com>wrote:
>>>>>>>>>
>>>>>>>>>> Hi Arun,
>>>>>>>>>>
>>>>>>>>>> I had this issue too - I think it is a consequence
of using an
>>>>>>>>>> account
>>>>>>>>>> with a large number of existing groups. There's a
fix in
>>>>>>>>>> https://issues.apache.org/jira/browse/WHIRR-381 that
you might
>>>>>>>>>> like to
>>>>>>>>>> try - this will be fixed in the next release.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Tom
>>>>>>>>>>
>>>>>>>>>> On Tue, Oct 4, 2011 at 1:30 PM, Arun Ramakrishnan
>>>>>>>>>> <sinchronized.arun@gmail.com> wrote:
>>>>>>>>>> > I am having this issue upgrading to 0.6.
>>>>>>>>>> >
>>>>>>>>>> > During the security group creation, the internal
access rules
>>>>>>>>>> within the
>>>>>>>>>> > cluster don't get added, consequentially blocking
the datanodes
>>>>>>>>>> from being
>>>>>>>>>> > able to connect to the namenode. I added these
rules manually
>>>>>>>>>> and it seems
>>>>>>>>>> > fine.
>>>>>>>>>> >
>>>>>>>>>> > This bug recurs for me consistently in my environment.
>>>>>>>>>> >
>>>>>>>>>> > log
>>>>>>>>>> > ########################################################
>>>>>>>>>> >
>>>>>>>>>> > 2011-10-04 12:23:20,949 DEBUG [jclouds.compute]
>>>>>>>>>> (pool-3-thread-4) >>
>>>>>>>>>> > creating keyPair region(us-east-1) group(ccore27)
>>>>>>>>>> > 2011-10-04 12:23:20,949 DEBUG [jclouds.compute]
>>>>>>>>>> (pool-3-thread-2) >>
>>>>>>>>>> > creating keyPair region(us-east-1) group(ccore27)
>>>>>>>>>> > 2011-10-04 12:23:21,414 DEBUG [jclouds.compute]
>>>>>>>>>> (pool-3-thread-4) << created
>>>>>>>>>> > keyPair(jclouds#ccore27#us-east-1#72)
>>>>>>>>>> > 2011-10-04 12:23:21,414 DEBUG [jclouds.compute]
>>>>>>>>>> (pool-3-thread-4) >>
>>>>>>>>>> > creating securityGroup region(us-east-1)
>>>>>>>>>> name(jclouds#ccore27#us-east-1)
>>>>>>>>>> > 2011-10-04 12:23:21,692 DEBUG [jclouds.compute]
>>>>>>>>>> (pool-3-thread-4) << created
>>>>>>>>>> > securityGroup(jclouds#ccore27#us-east-1)
>>>>>>>>>> > 2011-10-04 12:23:21,692 DEBUG [jclouds.compute]
>>>>>>>>>> (pool-3-thread-4) >>
>>>>>>>>>> > authorizing securityGroup region(us-east-1)
>>>>>>>>>> name(jclouds#ccore27#us-east-1)
>>>>>>>>>> > port(22)
>>>>>>>>>> > 2011-10-04 12:23:21,926 DEBUG [jclouds.compute]
>>>>>>>>>> (pool-3-thread-4) <<
>>>>>>>>>> > authorized securityGroup(jclouds#ccore27#us-east-1)
>>>>>>>>>> > 2011-10-04 12:23:21,926 DEBUG [jclouds.compute]
>>>>>>>>>> (pool-3-thread-4) >>
>>>>>>>>>> > authorizing securityGroup region(us-east-1)
>>>>>>>>>> name(jclouds#ccore27#us-east-1)
>>>>>>>>>> > permission to itself
>>>>>>>>>> > 2011-10-04 12:23:22,306 ERROR
>>>>>>>>>> > [org.apache.whirr.actions.BootstrapClusterAction]
>>>>>>>>>> (pool-3-thread-3)
>>>>>>>>>> > Unexpected error while starting 4 nodes, minimum
4 nodes for
>>>>>>>>>> > [hadoop-datanode, hadoop-tasktracker] of cluster
ccore27
>>>>>>>>>> > java.util.concurrent.ExecutionException:
>>>>>>>>>> java.lang.RuntimeException:
>>>>>>>>>> > request: POST https://ec2.us-east-1.amazonaws.com/
HTTP/1.1;
>>>>>>>>>> cause:
>>>>>>>>>> > java.lang.NullPointerException
>>>>>>>>>> >     at
>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>>>>>>>> >     at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>>>>>>>> >     at
>>>>>>>>>> >
>>>>>>>>>> org.apache.whirr.actions.BootstrapClusterAction$StartupProcess.waitForOutcomes(BootstrapClusterAction.java:320)
>>>>>>>>>> >     at
>>>>>>>>>> >
>>>>>>>>>> org.apache.whirr.actions.BootstrapClusterAction$StartupProcess.call(BootstrapClusterAction.java:273)
>>>>>>>>>> >     at
>>>>>>>>>> >
>>>>>>>>>> org.apache.whirr.actions.BootstrapClusterAction$StartupProcess.call(BootstrapClusterAction.java:234)
>>>>>>>>>> >     at
>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>>>>> >     at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>>>>> >     at
>>>>>>>>>> >
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>> >     at
>>>>>>>>>> >
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>> >     at java.lang.Thread.run(Thread.java:680)
>>>>>>>>>> > Caused by: java.lang.RuntimeException: request:
POST
>>>>>>>>>> > https://ec2.us-east-1.amazonaws.com/ HTTP/1.1;
cause:
>>>>>>>>>> > java.lang.NullPointerException
>>>>>>>>>> >     at
>>>>>>>>>> >
>>>>>>>>>> org.jclouds.http.functions.ParseSax.addDetailsAndPropagate(ParseSax.java:152)
>>>>>>>>>> >     at
>>>>>>>>>> org.jclouds.http.functions.ParseSax.parse(ParseSax.java:116)
>>>>>>>>>> >     at
>>>>>>>>>> org.jclouds.http.functions.ParseSax.apply(ParseSax.java:78)
>>>>>>>>>> >     at
>>>>>>>>>> org.jclouds.http.functions.ParseSax.apply(ParseSax.java:51)
>>>>>>>>>> >     at
>>>>>>>>>> com.google.common.util.concurrent.Futures$4.apply(Futures.java:439)
>>>>>>>>>> >     at
>>>>>>>>>> com.google.common.util.concurrent.Futures$4.apply(Futures.java:437)
>>>>>>>>>> >     at
>>>>>>>>>> >
>>>>>>>>>> com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:713)
>>>>>>>>>> >     at
>>>>>>>>>> >
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>> >     at
>>>>>>>>>> >
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>> >     at java.lang.Thread.run(Thread.java:680)
>>>>>>>>>> >     at
>>>>>>>>>> >
>>>>>>>>>> org.jclouds.http.functions.ParseSax.addDetailsAndPropagate(ParseSax.java:152)
>>>>>>>>>> >     at
>>>>>>>>>> org.jclouds.http.functions.ParseSax.apply(ParseSax.java:80)
>>>>>>>>>> >     at
>>>>>>>>>> org.jclouds.http.functions.ParseSax.apply(ParseSax.java:51)
>>>>>>>>>> >     at
>>>>>>>>>> com.google.common.util.concurrent.Futures$4.apply(Futures.java:439)
>>>>>>>>>> >     at
>>>>>>>>>> com.google.common.util.concurrent.Futures$4.apply(Futures.java:437)
>>>>>>>>>> >     at
>>>>>>>>>> >
>>>>>>>>>> com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:713)
>>>>>>>>>> >     ... 3 more
>>>>>>>>>> >
>>>>>>>>>> > ###########################################################
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > This are the rules that existed for the cluster
created with
>>>>>>>>>> whirr 0.6
>>>>>>>>>> > GROUP    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>>> > jclouds#ccore27#us-east-1
>>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>>> ALLOWS    tcp
>>>>>>>>>> > 22    22    FROM    CIDR    0.0.0.0/0
>>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>>> ALLOWS    tcp
>>>>>>>>>> > 8020    8020    FROM    CIDR    184.72.183.32/32
>>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>>> ALLOWS    tcp
>>>>>>>>>> > 8021    8021    FROM    CIDR    184.72.183.32/32
>>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>>> ALLOWS    tcp
>>>>>>>>>> > 50030    50030    FROM    CIDR    24.43.39.218/32
>>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>>> ALLOWS    tcp
>>>>>>>>>> > 50070    50070    FROM    CIDR    24.43.39.218/32
>>>>>>>>>> >
>>>>>>>>>> > I had to manually add
>>>>>>>>>> > PERMISSION    673040621396    jclouds#ccore27#us-east-1
>>>>>>>>>> ALLOWS    all
>>>>>>>>>> >         FROM    USER    673040621396    GRPNAME
>>>>>>>>>> jclouds#ccore27#us-east-1
>>>>>>>>>> > to allow the datanodes to talk to the namenode.
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > Here is my config file with updated property
names to match 0.6
>>>>>>>>>> > ##########################################
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > whirr.cluster-name=ccore27
>>>>>>>>>> > whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,4
>>>>>>>>>> > hadoop-datanode+hadoop-tasktracker
>>>>>>>>>> >
>>>>>>>>>> > whirr.provider=aws-ec2
>>>>>>>>>> > whirr.identity=*************************
>>>>>>>>>> > whirr.credential=**********************
>>>>>>>>>> > whirr.private-key-file=/Users/arun/.ec2/hadoopkey
>>>>>>>>>> > whirr.public-key-file=/Users/arun/.ec2/hadoopkey.pub
>>>>>>>>>> > whirr.client-cidrs=24.43.39.218/32
>>>>>>>>>> >
>>>>>>>>>> > whirr.location-id=us-east-1
>>>>>>>>>> > whirr.hardware-id=c1.xlarge
>>>>>>>>>> > #c1.xlarge
>>>>>>>>>> >
>>>>>>>>>> > # Ubuntu 10.04 LTS Lucid. See http://alestic.com/
or
>>>>>>>>>> > http://aws.amazon.com/amis/4348
>>>>>>>>>> > # ebs root only
>>>>>>>>>> > whirr.image-id=us-east-1/ami-4a0df923
>>>>>>>>>> >
>>>>>>>>>> > whirr.hadoop.install-function=install_cdh_hadoop
>>>>>>>>>> > whirr.hadoop.configure-function=configure_cdh_hadoop
>>>>>>>>>> >
>>>>>>>>>> > #######################################
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>

Mime
View raw message