cassandra-pr mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From iamaleksey <...@git.apache.org>
Subject [GitHub] cassandra pull request #239: [CASSANDRA-14556] Optimize Streaming
Date Thu, 26 Jul 2018 20:09:33 GMT
Github user iamaleksey commented on a diff in the pull request:

    https://github.com/apache/cassandra/pull/239#discussion_r205586651
  
    --- Diff: src/java/org/apache/cassandra/db/streaming/ComponentManifest.java ---
    @@ -0,0 +1,130 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.cassandra.db.streaming;
    +
    +import java.io.IOException;
    +import java.util.ArrayList;
    +import java.util.Iterator;
    +import java.util.LinkedHashMap;
    +import java.util.List;
    +import java.util.Map;
    +
    +import com.google.common.collect.Iterators;
    +
    +import org.apache.cassandra.db.TypeSizes;
    +import org.apache.cassandra.io.IVersionedSerializer;
    +import org.apache.cassandra.io.sstable.Component;
    +import org.apache.cassandra.io.util.DataInputPlus;
    +import org.apache.cassandra.io.util.DataOutputPlus;
    +
    +public final class ComponentManifest implements Iterable<Component>
    +{
    +    private final LinkedHashMap<Component, Long> components;
    +
    +    public ComponentManifest(Map<Component, Long> components)
    +    {
    +        this.components = new LinkedHashMap<>(components);
    +    }
    +
    +    public long sizeOf(Component component)
    +    {
    +        Long size = components.get(component);
    +        if (size == null)
    +            throw new IllegalArgumentException("Component " + component + " is not present
in the manifest");
    +        return size;
    +    }
    +
    +    public long totalSize()
    +    {
    +        long totalSize = 0;
    +        for (Long size : components.values())
    +            totalSize += size;
    +        return totalSize;
    +    }
    +
    +    public List<Component> components()
    +    {
    +        return new ArrayList<>(components.keySet());
    +    }
    +
    +    @Override
    +    public boolean equals(Object o)
    +    {
    +        if (this == o)
    +            return true;
    +
    +        if (!(o instanceof ComponentManifest))
    +            return false;
    +
    +        ComponentManifest that = (ComponentManifest) o;
    +        return components.equals(that.components);
    +    }
    +
    +    @Override
    +    public int hashCode()
    +    {
    +        return components.hashCode();
    +    }
    +
    +    public static final IVersionedSerializer<ComponentManifest> serializer = new
IVersionedSerializer<ComponentManifest>()
    +    {
    +        public void serialize(ComponentManifest manifest, DataOutputPlus out, int version)
throws IOException
    +        {
    +            out.writeUnsignedVInt(manifest.components.size());
    +            for (Map.Entry<Component, Long> entry : manifest.components.entrySet())
    +            {
    +                out.writeByte(entry.getKey().type.id);
    --- End diff --
    
    Talked to @dineshjoshi offline, and we realised that this is incomplete - and neither
was my proposed version. For completeness, when want to serialize the whole component info,
not just its type. And it has two important fields - type and name. Name will usually be derived
from the type, but not always. And even though we don't support streaming those components
(custom and SI), we might want to change it in the future, and the protocol should allow it.
    
    So I suggest we encode`component.type.name()`, the full enum name, followed by `component.name()`.
It's a little heavier, but this is completely irrelevant in the big picture, size-wise.
    
    The upside is that we can handle encode/decode any component necessary in the future,
loss-free. And, again, we don't really need to assign ids. `valueOf()` is plenty good, and
allows extension without overlap risk like in `Verb`.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Mime
View raw message