cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonas Borgström (Commented) (JIRA) <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3466) Hinted handoff not working after rolling upgrade from 0.8.7 to 1.0.2
Date Wed, 09 Nov 2011 14:47:52 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147070#comment-13147070
] 

Jonas Borgström commented on CASSANDRA-3466:
--------------------------------------------

Brandon, you're right. After commenting out those two lines I can no longer reproduce the
hint delivery problem.

The AssertionError is still there though and I managed to reproduce it with my two node setup.
I'll attach a tarball with my config, DEBUG-level log files and data directories shortly.
What I did was basically:

# Start a new cluster with a patched 1.0.2 and create keyspace and column family.
# Bring down node2, trigger hints on node1
# Bring up node2, notice that hints are delivered, YAY!
# Bring down node1, trigger hints on node2
# Bring up node1, noticed AssertionError in log file for node1   (node1/system.log:2036)

To me it looks like the assertionError on node1 is triggered when node2 connects to deliver
its hints.
                
> Hinted handoff not working after rolling upgrade from 0.8.7 to 1.0.2
> --------------------------------------------------------------------
>
>                 Key: CASSANDRA-3466
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3466
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.0.0
>            Reporter: Jonas Borgström
>            Assignee: Brandon Williams
>              Labels: hintedhandoff
>             Fix For: 1.0.3
>
>
> While testing rolling upgrades from 0.8.7 to 1.0.2 on a test cluster I've noticed that
hinted hand-off didn't always work properly. Hints generated on an upgraded node does not
seem to be delivered to other newly upgraded nodes once they rejoin the ring. They only way
I've found to get a node to deliver its hints is to restart it.
> Here's some steps to reproduce this issue:
> 1. Install cassandra 0.8.7 on node1 and node2 using default settings.
> 2. Create keyspace foo with {replication_factor: 2}. Create column family bar
> 3. Shutdown node2 
> 4. Insert data into bar and verify that HintsColumnFamily on node2 contains hints
> 5. Start node2 and verify that hinted handoff is performed and HintsColumnFamily becomes
empty again.
> 6. Upgrade and restart node1
> 7. Shutdown node2 
> 8. Insert data into bar and verify that HintsColumnFamily on node2 contains hints
> 9. Upgrade and start node2
> 10. Notice that hinted handoff is *not* performed when "node2" comes back. (Only if node1
is restarted)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message