From commits-return-148161-apmail-cassandra-commits-archive=cassandra.apache.org@cassandra.apache.org Fri Oct 9 18:52:05 2015 Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B612A183E8 for ; Fri, 9 Oct 2015 18:52:05 +0000 (UTC) Received: (qmail 87620 invoked by uid 500); 9 Oct 2015 18:52:05 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 87584 invoked by uid 500); 9 Oct 2015 18:52:05 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 87494 invoked by uid 99); 9 Oct 2015 18:52:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Oct 2015 18:52:05 +0000 Date: Fri, 9 Oct 2015 18:52:05 +0000 (UTC) From: "Jonathan Shook (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CASSANDRA-10495) Improve the way we do streaming with vnodes MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-10495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14950983#comment-14950983 ] Jonathan Shook commented on CASSANDRA-10495: -------------------------------------------- What if the streaming protocol were enhanced to allow sending nodes to provide an offer manifest, blocking until the receiver responded with a preferred ordering and grouping. Does this help address any of the planning issues better? > Improve the way we do streaming with vnodes > ------------------------------------------- > > Key: CASSANDRA-10495 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10495 > Project: Cassandra > Issue Type: Improvement > Reporter: Marcus Eriksson > Fix For: 3.x > > > Streaming with vnodes usually creates a large amount of sstables on the target node - for example if each source node has 100 sstables and we use num_tokens = 256, the bootstrapping (for example) node might get 100*256 sstables > One approach could be to do an on-the-fly compaction on the source node, meaning we would only stream out one sstable per range. Note that we will want the compaction strategy to decide how to combine the sstables, for example LCS will not want to mix sstables from different levels while STCS can probably just combine everything > cc [~yukim] -- This message was sent by Atlassian JIRA (v6.3.4#6332)