cassandra-pr mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [cassandra] adelapena commented on a change in pull request #570: CASSANDRA-13606 Improve handling of 2i initialization failures
Date Thu, 07 May 2020 13:03:37 GMT

adelapena commented on a change in pull request #570:
URL: https://github.com/apache/cassandra/pull/570#discussion_r421413678



##########
File path: src/java/org/apache/cassandra/index/Index.java
##########
@@ -136,6 +135,7 @@
  */
 public interface Index
 {
+    public static enum LoadType {READ, WRITE, ALL, NONE};

Review comment:
       +1 to the new enum names. It would be nice to add a very brief comment about what it
is, for the sake of the unaware reader. The static and the semicolon are not needed:
   ```suggestion
       public enum LoadType {READ, WRITE, ALL, NONE}
   ```

##########
File path: src/java/org/apache/cassandra/index/internal/CassandraIndex.java
##########
@@ -146,6 +146,22 @@ protected abstract ByteBuffer getIndexedValue(ByteBuffer partitionKey,
                                                   CellPath path,
                                                   ByteBuffer cellValue);
 
+    
+    public boolean supportsLoad(LoadType load)
+    {
+        switch (load)
+        {
+            case ALL:
+                return supportedLoads == LoadType.ALL;
+            case READ:
+                return supportedLoads == LoadType.ALL || supportedLoads == LoadType.READ;
+            case WRITE:
+                return supportedLoads == LoadType.ALL || supportedLoads == LoadType.WRITE;
+            default:
+                return false;
+        }

Review comment:
       This block of code is repeated in a couple of test index implementations, and it's
to expect that other implementations would need to repeat it. I think that defining that `ALL`
includes `READ` and `WRITE`, etc. seems more a characteristic of `LoadType` than the index
itself, so perhaps we could move this logic to the enum itself, for example:
   ```java
   public enum LoadType {
       READ, WRITE, ALL, NONE;
   
       public boolean accepts(LoadType load)
       {
           switch (this)
           {
               case ALL:
                   return true;
               case READ:
                   return load == LoadType.READ || load == LoadType.NONE;
               case WRITE:
                   return load == LoadType.WRITE || load == LoadType.NONE;
               default:
                   return false;
           }
       }
   }
   ```
   That way we would simplify usage later:
   ```java
   @Override
   public boolean supportsLoad(LoadType load)
   {
       return supportedLoadType.accepts(load);
   }
   ```
   

##########
File path: src/java/org/apache/cassandra/index/SecondaryIndexManager.java
##########
@@ -338,20 +355,20 @@ public void markAllIndexesRemoved()
     public void rebuildIndexesBlocking(Set<String> indexNames)
     {
         try (ColumnFamilyStore.RefViewFragment viewFragment = baseCfs.selectAndReference(View.selectFunction(SSTableSet.CANONICAL));
-             Refs<SSTableReader> allSSTables = viewFragment.refs)
-        {
-            Set<Index> toRebuild = indexes.values().stream()
-                                          .filter(index -> indexNames.contains(index.getIndexMetadata().name))
-                                          .filter(Index::shouldBuildBlocking)
-                                          .collect(Collectors.toSet());
-            if (toRebuild.isEmpty())
-            {
-                logger.info("No defined indexes with the supplied names: {}", Joiner.on(',').join(indexNames));
-                return;
-            }
-
-            buildIndexesBlocking(allSSTables, toRebuild, true);
-        }
+                Refs<SSTableReader> allSSTables = viewFragment.refs)
+           {
+               Set<Index> toRebuild = indexes.values().stream()
+                                             .filter(index -> indexNames.contains(index.getIndexMetadata().name))
+                                             .filter(Index::shouldBuildBlocking)
+                                             .collect(Collectors.toSet());
+               if (toRebuild.isEmpty())
+               {
+                   logger.info("No defined indexes with the supplied names: {}", Joiner.on(',').join(indexNames));
+                   return;
+               }
+
+               buildIndexesBlocking(allSSTables, toRebuild, true);
+           }

Review comment:
       Nit: this seems to have missed alignment with an extra tab

##########
File path: src/java/org/apache/cassandra/index/SecondaryIndexManager.java
##########
@@ -619,8 +642,16 @@ private synchronized void markIndexesBuilding(Set<Index> indexes,
boolean isFull
     private synchronized void markIndexBuilt(Index index, boolean isFullRebuild)
     {
         String indexName = index.getIndexMetadata().name;
-        if (isFullRebuild)
+        if (isFullRebuild && index.supportsLoad(Index.LoadType.READ))
+        {
             queryableIndexes.add(indexName);
+            logger.info("Index [" + indexName + "] became queryable.");
+        }
+        if (isFullRebuild && index.supportsLoad(Index.LoadType.WRITE))
+        {
+            writableIndexes.add(indexName);
+            logger.info("Index [" + indexName + "] became writable.");

Review comment:
       I think that `became` suggests a change of status, as if it weren't writable before.
We could either say that is writable without telling if it was so before or, even better,
use the boolean returned by `queryableIndexes.add`:
   ```suggestion
               if (writableIndexes.add(indexName))
                   logger.info("Index [" + indexName + "] became writable.");
   ```
   The same applies to queryable indexes above.

##########
File path: src/java/org/apache/cassandra/index/SecondaryIndexManager.java
##########
@@ -424,10 +441,13 @@ public static String getIndexName(String cfName)
         assert isIndexColumnFamily(cfName);
         return StringUtils.substringAfter(cfName, Directories.SECONDARY_INDEX_NAME_SEPARATOR);
     }
-
+    
     /**
-     * Performs a blocking (re)indexing of the specified SSTables for the specified indexes.
+     * Performs a blocking (re)indexing/recovery of the specified SSTables for the specified
indexes.
      *
+     * If the index doesn't support ALL {@link Index.LoadType} it performs a recovery {@link
getRecoveryTaskSupport()}
+     * instead of a build {@link getBuildTaskSupport()}

Review comment:
       ```suggestion
        * If the index doesn't support ALL {@link Index.LoadType} it performs a recovery {@link
Index#getRecoveryTaskSupport()}
        * instead of a build {@link Index#getBuildTaskSupport()}
   ```

##########
File path: src/java/org/apache/cassandra/index/SecondaryIndexManager.java
##########
@@ -1356,7 +1396,7 @@ public void commit()
                 for (Index index : indexes)
                 {
                     Index.Indexer indexer = index.indexerFor(key, columns, nowInSec, ctx,
Type.COMPACTION);
-                    if (indexer == null)
+                    if (indexer == null || !indexManager.isIndexWritable(index))

Review comment:
       See comment above about either calling `isIndexWritable` before creating the indexer,
or directly passing `writableIndexes` to the `IndexGCTransaction ` constructor, so we don't
need to this check.

##########
File path: src/java/org/apache/cassandra/index/SecondaryIndexManager.java
##########
@@ -1126,17 +1158,21 @@ public UpdateTransaction newUpdateTransaction(PartitionUpdate update,
WriteConte
     {
         if (!hasIndexes())
             return UpdateTransaction.NO_OP;
-
-        Index.Indexer[] indexers = indexes.values().stream()
-                                          .map(i -> i.indexerFor(update.partitionKey(),
-                                                                 update.columns(),
-                                                                 nowInSec,
-                                                                 ctx,
-                                                                 IndexTransaction.Type.UPDATE))
-                                          .filter(Objects::nonNull)
-                                          .toArray(Index.Indexer[]::new);
-
-        return indexers.length == 0 ? UpdateTransaction.NO_OP : new WriteTimeTransaction(indexers);
+        
+        ArrayList<Index.Indexer> idxrs = new ArrayList<>();
+        for (Index i : indexes.values())
+        {
+            Index.Indexer idxr = i.indexerFor(update.partitionKey(), update.columns(), nowInSec,
ctx, IndexTransaction.Type.UPDATE);
+            if (idxr != null && isIndexWritable(i))

Review comment:
       We could check if the index is writable before creating the indexer:
   ```suggestion
               if (!isIndexWritable(i))
                   continue;
   
               Index.Indexer idxr = i.indexerFor(update.partitionKey(), update.columns(),
nowInSec, ctx, IndexTransaction.Type.UPDATE);
               if (idxr != null)
   ```
   The same applies to the other usages of `isIndexWritable` in `IndexGCTransaction#commit()`
and `CleanupGCTransaction#commit()`. Alternatively, we could transform the set `SIM#writableIndexes`
into a `Map<String, Index>`, and iterate its values this instead of `SIM#indexes` values
in the three places. That way we wouldn't even need the `isIndexWritable` method.

##########
File path: src/java/org/apache/cassandra/index/internal/CassandraIndex.java
##########
@@ -79,6 +78,7 @@
     protected ColumnFamilyStore indexCfs;
     protected ColumnMetadata indexedColumn;
     protected CassandraIndexFunctions functions;
+    protected LoadType supportedLoads = LoadType.ALL;

Review comment:
       Nit:
   ```suggestion
       protected LoadType supportedLoadType = LoadType.ALL;
   ```

##########
File path: test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
##########
@@ -1052,6 +1052,58 @@ public void testIndexQueriesWithIndexNotReady() throws Throwable
             execute("DROP index " + KEYSPACE + ".testIndex");
         }
     }
+    
+    @Test
+    public void testIndexWritesWithIndexNotReady() throws Throwable
+    {
+        createTable("CREATE TABLE %s (pk int, ck int, value int, PRIMARY KEY (pk, ck))");
+        String indexName = createIndex("CREATE CUSTOM INDEX ON %s (value) USING '" + BlockingStubIndex.class.getName()
+ "'");
+
+        execute("INSERT INTO %s (pk, ck, value) VALUES (?, ?, ?)", 1, 1, 1);
+        BlockingStubIndex index = (BlockingStubIndex) getCurrentColumnFamilyStore().indexManager.getIndexByName(indexName);
+        assertEquals(0, index.rowsInserted.size());
+        execute("DROP index " + KEYSPACE + "." + indexName);
+    }
+    
+    @Test // A Bad init could leave an index only accepting reads
+    public void testReadOnlyIndex() throws Throwable
+    {
+        String tableName = createTable("CREATE TABLE %s (pk int, ck int, value int, PRIMARY
KEY (pk, ck))");
+        String indexName = createIndex("CREATE CUSTOM INDEX ON %s (value) USING '" + ReadOnlyIndex.class.getName()
+ "'");
+        assertTrue(waitForIndex(keyspace(), tableName, indexName));
+
+        execute("SELECT value FROM %s WHERE value = 1");
+        execute("INSERT INTO %s (pk, ck, value) VALUES (?, ?, ?)", 1, 1, 1);
+
+        ReadOnlyIndex index = (ReadOnlyIndex) getCurrentColumnFamilyStore().indexManager.getIndexByName(indexName);
+        assertEquals(0, index.rowsInserted.size());
+        execute("DROP index " + KEYSPACE + "." + indexName);
+    }
+    
+    @Test  // A Bad init could leave an index only accepting writes
+    public void testWriteOnlyIndex() throws Throwable
+    {
+        String tableName = createTable("CREATE TABLE %s (pk int, ck int, value int, PRIMARY
KEY (pk, ck))");
+        String indexName = createIndex("CREATE CUSTOM INDEX ON %s (value) USING '" + WriteOnlyIndex.class.getName()
+ "'");
+        assertTrue(waitForIndex(keyspace(), tableName, indexName));
+
+        execute("INSERT INTO %s (pk, ck, value) VALUES (?, ?, ?)", 1, 1, 1);
+        WriteOnlyIndex index = (WriteOnlyIndex) getCurrentColumnFamilyStore().indexManager.getIndexByName(indexName);
+        assertEquals(1, index.rowsInserted.size());
+        try
+        {
+            execute("SELECT value FROM %s WHERE value = 1");    
+            fail();
+        }
+        catch (IndexNotAvailableException e)
+        {
+            assertTrue(true);
+        }
+        finally
+        {
+            execute("DROP index " + KEYSPACE + "." + indexName);

Review comment:
       Similarly to the comment before, we could try a recovery, and simulate the initializitation
failure throwing an exception in `WriteOnlyIndex#getInitializationTask`.

##########
File path: test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
##########
@@ -1052,6 +1052,58 @@ public void testIndexQueriesWithIndexNotReady() throws Throwable
             execute("DROP index " + KEYSPACE + ".testIndex");
         }
     }
+    
+    @Test
+    public void testIndexWritesWithIndexNotReady() throws Throwable
+    {
+        createTable("CREATE TABLE %s (pk int, ck int, value int, PRIMARY KEY (pk, ck))");
+        String indexName = createIndex("CREATE CUSTOM INDEX ON %s (value) USING '" + BlockingStubIndex.class.getName()
+ "'");
+
+        execute("INSERT INTO %s (pk, ck, value) VALUES (?, ?, ?)", 1, 1, 1);
+        BlockingStubIndex index = (BlockingStubIndex) getCurrentColumnFamilyStore().indexManager.getIndexByName(indexName);
+        assertEquals(0, index.rowsInserted.size());
+        execute("DROP index " + KEYSPACE + "." + indexName);
+    }
+    
+    @Test // A Bad init could leave an index only accepting reads
+    public void testReadOnlyIndex() throws Throwable
+    {
+        String tableName = createTable("CREATE TABLE %s (pk int, ck int, value int, PRIMARY
KEY (pk, ck))");
+        String indexName = createIndex("CREATE CUSTOM INDEX ON %s (value) USING '" + ReadOnlyIndex.class.getName()
+ "'");
+        assertTrue(waitForIndex(keyspace(), tableName, indexName));
+
+        execute("SELECT value FROM %s WHERE value = 1");
+        execute("INSERT INTO %s (pk, ck, value) VALUES (?, ?, ?)", 1, 1, 1);
+
+        ReadOnlyIndex index = (ReadOnlyIndex) getCurrentColumnFamilyStore().indexManager.getIndexByName(indexName);
+        assertEquals(0, index.rowsInserted.size());
+        execute("DROP index " + KEYSPACE + "." + indexName);

Review comment:
       We could try a recovery here, and verify that the index starts to accepts writes after
that recovery. Also, in `ReadOnlyIndex`, we could throw an exception in `getInitializationTask`,
and provide a mock implementation `getRecoveryTaskSupport` that enables writes. Note that
forcing an exception during initialization would make us to miss the call to `markIndexBuilt`,
so the index won't be added to `SIM#queryableIndexes` and it won't accepts reads independently
of its `ReadOnlyIndex#supportsLoad` implementation. That's why I think we need some new logic
in `SIM#markIndexFailed` too.

##########
File path: test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
##########
@@ -1375,9 +1427,10 @@ public void testIndexOnFrozenUDT() throws Throwable
 
         execute("INSERT INTO %s (k, v) VALUES (?, ?)", 0, udt1);
         String indexName = createIndex("CREATE INDEX ON %s (v)");
+        assertTrue(waitForIndex(keyspace(), tableName, indexName));

Review comment:
       I think that this is an important change of behaviour in indexes, they are missing
writes done during their initialization. I think that this problem can be solved if we add
them to `SIM#writableIndexes` at the beginning of their registration (before initialization),
and we remove them from `writableIndexes` in `SIM#markIndexFailed`. WDYT?

##########
File path: test/unit/org/apache/cassandra/index/SecondaryIndexManagerTest.java
##########
@@ -119,37 +124,47 @@ public void addingSSTablesMarksTheIndexAsBuilt() throws Throwable
     }
 
     @Test
-    public void cannotRebuildWhileInitializationIsInProgress() throws Throwable
+    public void cannotRebuilRecoverdWhileInitializationIsInProgress() throws Throwable

Review comment:
       ```suggestion
       public void cannotRebuildRecoverWhileInitializationIsInProgress() throws Throwable
   ```

##########
File path: test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
##########
@@ -1560,4 +1613,77 @@ public IndexBlockingOnInitialization(ColumnFamilyStore baseCfs, IndexMetadata
in
             return super.getInvalidateTask();
         }
     }
+    
+    /**
+     * <code>StubIndex</code> that blocks during the initialization.
+     */
+    public static class BlockingStubIndex extends StubIndex
+    {
+        private final CountDownLatch latch = new CountDownLatch(1);
+
+        public BlockingStubIndex(ColumnFamilyStore baseCfs, IndexMetadata indexDef)
+        {
+            super(baseCfs, indexDef);
+        }
+
+        @Override
+        public Callable<?> getInitializationTask()
+        {
+            return () -> {
+                latch.await();
+                return null;
+            };
+        }
+
+        @Override
+        public Callable<?> getInvalidateTask()
+        {
+            latch.countDown();
+            return super.getInvalidateTask();
+        }
+    }
+
+    /**
+     * <code>CassandraIndex</code> that only supports reads. Could be intentional
or a result of a bad init
+     */
+    public static class ReadOnlyIndex extends StubIndex
+    {
+        public ReadOnlyIndex(ColumnFamilyStore baseCfs, IndexMetadata indexDef)
+        {
+            super(baseCfs, indexDef);
+        }
+
+        @Override
+        public Callable<?> getInitializationTask()
+        {
+            return null;
+        }
+
+        public boolean supportsLoad(LoadType load)
+        {
+            return load.equals(LoadType.READ);
+        }
+    }
+
+    /**
+     * <code>CassandraIndex</code> that only supports writes. Could be intentional
or a result of a bad init

Review comment:
       Nit: is not a `CassandraIndex`:
   ```suggestion
        * {@code StubIndex} that only supports writes. Could be intentional or a result of
a bad init.
   ```

##########
File path: test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
##########
@@ -1560,4 +1613,77 @@ public IndexBlockingOnInitialization(ColumnFamilyStore baseCfs, IndexMetadata
in
             return super.getInvalidateTask();
         }
     }
+    
+    /**
+     * <code>StubIndex</code> that blocks during the initialization.
+     */
+    public static class BlockingStubIndex extends StubIndex
+    {
+        private final CountDownLatch latch = new CountDownLatch(1);
+
+        public BlockingStubIndex(ColumnFamilyStore baseCfs, IndexMetadata indexDef)
+        {
+            super(baseCfs, indexDef);
+        }
+
+        @Override
+        public Callable<?> getInitializationTask()
+        {
+            return () -> {
+                latch.await();
+                return null;
+            };
+        }
+
+        @Override
+        public Callable<?> getInvalidateTask()
+        {
+            latch.countDown();
+            return super.getInvalidateTask();
+        }
+    }
+
+    /**
+     * <code>CassandraIndex</code> that only supports reads. Could be intentional
or a result of a bad init
+     */
+    public static class ReadOnlyIndex extends StubIndex
+    {
+        public ReadOnlyIndex(ColumnFamilyStore baseCfs, IndexMetadata indexDef)
+        {
+            super(baseCfs, indexDef);
+        }
+
+        @Override
+        public Callable<?> getInitializationTask()
+        {
+            return null;
+        }
+
+        public boolean supportsLoad(LoadType load)
+        {
+            return load.equals(LoadType.READ);

Review comment:
       ```suggestion
               return load == LoadType.READ;
   ```

##########
File path: test/unit/org/apache/cassandra/index/SecondaryIndexManagerTest.java
##########
@@ -719,4 +746,20 @@ public boolean shouldBuildBlocking()
             return true;
         }
     }
+
+    /**
+     * <code>CassandraIndex</code> that only supports reads. Could be intentional
or a result of a bad init

Review comment:
       ```suggestion
        * <code>TestingIndex</code> that only supports reads. Could be intentional
or a result of a bad init
   ```

##########
File path: test/unit/org/apache/cassandra/index/SecondaryIndexManagerTest.java
##########
@@ -719,4 +746,20 @@ public boolean shouldBuildBlocking()
             return true;
         }
     }
+
+    /**
+     * <code>CassandraIndex</code> that only supports reads. Could be intentional
or a result of a bad init
+     */
+    public static class ReadOnlyIndex extends TestingIndex
+    {
+        public ReadOnlyIndex(ColumnFamilyStore baseCfs, IndexMetadata indexDef)
+        {
+            super(baseCfs, indexDef);
+        }
+
+        public boolean supportsLoad(LoadType load)
+        {
+            return load.equals(LoadType.READ);

Review comment:
       ```suggestion
               return load == LoadType.READ;
   ```

##########
File path: test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
##########
@@ -1560,4 +1613,77 @@ public IndexBlockingOnInitialization(ColumnFamilyStore baseCfs, IndexMetadata
in
             return super.getInvalidateTask();
         }
     }
+    
+    /**
+     * <code>StubIndex</code> that blocks during the initialization.
+     */
+    public static class BlockingStubIndex extends StubIndex
+    {
+        private final CountDownLatch latch = new CountDownLatch(1);
+
+        public BlockingStubIndex(ColumnFamilyStore baseCfs, IndexMetadata indexDef)
+        {
+            super(baseCfs, indexDef);
+        }
+
+        @Override
+        public Callable<?> getInitializationTask()
+        {
+            return () -> {
+                latch.await();
+                return null;
+            };
+        }
+
+        @Override
+        public Callable<?> getInvalidateTask()
+        {
+            latch.countDown();
+            return super.getInvalidateTask();
+        }
+    }
+
+    /**
+     * <code>CassandraIndex</code> that only supports reads. Could be intentional
or a result of a bad init
+     */
+    public static class ReadOnlyIndex extends StubIndex
+    {
+        public ReadOnlyIndex(ColumnFamilyStore baseCfs, IndexMetadata indexDef)
+        {
+            super(baseCfs, indexDef);
+        }
+
+        @Override
+        public Callable<?> getInitializationTask()
+        {
+            return null;
+        }
+
+        public boolean supportsLoad(LoadType load)
+        {
+            return load.equals(LoadType.READ);
+        }
+    }
+
+    /**
+     * <code>CassandraIndex</code> that only supports writes. Could be intentional
or a result of a bad init
+     */
+    public static class WriteOnlyIndex extends StubIndex
+    {
+        public WriteOnlyIndex(ColumnFamilyStore baseCfs, IndexMetadata indexDef)
+        {
+            super(baseCfs, indexDef);
+        }
+
+        @Override
+        public Callable<?> getInitializationTask()
+        {
+            return null;
+        }
+
+        public boolean supportsLoad(LoadType load)
+        {
+            return load.equals(LoadType.WRITE);

Review comment:
       ```suggestion
               return load == LoadType.WRITE;
   ```

##########
File path: test/unit/org/apache/cassandra/cql3/validation/entities/SecondaryIndexTest.java
##########
@@ -1560,4 +1613,77 @@ public IndexBlockingOnInitialization(ColumnFamilyStore baseCfs, IndexMetadata
in
             return super.getInvalidateTask();
         }
     }
+    
+    /**
+     * <code>StubIndex</code> that blocks during the initialization.
+     */
+    public static class BlockingStubIndex extends StubIndex
+    {
+        private final CountDownLatch latch = new CountDownLatch(1);
+
+        public BlockingStubIndex(ColumnFamilyStore baseCfs, IndexMetadata indexDef)
+        {
+            super(baseCfs, indexDef);
+        }
+
+        @Override
+        public Callable<?> getInitializationTask()
+        {
+            return () -> {
+                latch.await();
+                return null;
+            };
+        }
+
+        @Override
+        public Callable<?> getInvalidateTask()
+        {
+            latch.countDown();
+            return super.getInvalidateTask();
+        }
+    }
+
+    /**
+     * <code>CassandraIndex</code> that only supports reads. Could be intentional
or a result of a bad init

Review comment:
       Nit: it's not a `CassandraIndex `:
   ```suggestion
        * {@code StubIndex} that only supports reads. Could be intentional or a result of
a bad init.
   ```

##########
File path: test/unit/org/apache/cassandra/index/internal/CustomCassandraIndex.java
##########
@@ -77,6 +77,7 @@
     protected ColumnFamilyStore indexCfs;
     protected ColumnMetadata indexedColumn;
     protected CassandraIndexFunctions functions;
+    protected LoadType supportedLoads = LoadType.ALL;

Review comment:
       ```suggestion
       protected LoadType supportedLoadType = LoadType.ALL;
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Mime
View raw message