cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tyler Hobbs (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6398) nodetool removenode error 'Endpoint /x.x.x.x generation changed while trying to remove it' occurs regularly.
Date Thu, 02 Jan 2014 19:55:51 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13860673#comment-13860673
] 

Tyler Hobbs commented on CASSANDRA-6398:
----------------------------------------

[~enigmacurry] do you have a somewhat reliable way to repro this?  So far I haven't seen this.

The exception is thrown when a new heartbeat version for the removed node comes in over gossip
during a 30s sleep period.  I'm not 100% sure why this check exists, but I would guess it's
to avoid removing a node that's still alive.  (We also have another check for this.)

> nodetool removenode error 'Endpoint /x.x.x.x generation changed while trying to remove
it' occurs regularly.
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-6398
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6398
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tools
>            Reporter: Ryan McGuire
>            Assignee: Tyler Hobbs
>            Priority: Minor
>             Fix For: 2.0.5
>
>
> I see this error somewhat regularly when running a *{{nodetool removenode}}* command.
The command completes successfully, ie, the node is removed, so I'm not sure what this message
is telling me.
>  
> {code}
> $ nodetool -p 7100 removenode bff9072e-4bb6-42fa-937c-bb73bcc094bc
> Exception in thread "main" java.lang.RuntimeException: Endpoint /127.0.0.2 generation
changed while trying to remove it
> 	at org.apache.cassandra.gms.Gossiper.advertiseRemoving(Gossiper.java:421)
> 	at org.apache.cassandra.service.StorageService.removeNode(StorageService.java:3080)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:111)
> 	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:45)
> 	at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:235)
> 	at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> 	at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:250)
> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:791)
> 	at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1486)
> 	at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:96)
> 	at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1327)
> 	at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1419)
> 	at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:847)
> 	at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
> 	at sun.rmi.transport.Transport$1.run(Transport.java:177)
> 	at sun.rmi.transport.Transport$1.run(Transport.java:174)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
> 	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
> 	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
> 	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:722)
> {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message