From dev-return-22481-apmail-ignite-dev-archive=ignite.apache.org@ignite.apache.org Tue Jun 6 03:02:36 2017 Return-Path: X-Original-To: apmail-ignite-dev-archive@minotaur.apache.org Delivered-To: apmail-ignite-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 51DDE1908C for ; Tue, 6 Jun 2017 03:02:36 +0000 (UTC) Received: (qmail 49223 invoked by uid 500); 6 Jun 2017 03:02:36 -0000 Delivered-To: apmail-ignite-dev-archive@ignite.apache.org Received: (qmail 49151 invoked by uid 500); 6 Jun 2017 03:02:36 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 49097 invoked by uid 99); 6 Jun 2017 03:02:35 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Jun 2017 03:02:35 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 8254F1AFC64 for ; Tue, 6 Jun 2017 03:02:35 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.514 X-Spam-Level: X-Spam-Status: No, score=-0.514 tagged_above=-999 required=6.31 tests=[RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-2.8, SPF_SOFTFAIL=0.972, URIBL_BLOCKED=0.001, URI_HEX=1.313] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id a--bvE_hb9hh for ; Tue, 6 Jun 2017 03:02:33 +0000 (UTC) Received: from resqmta-po-12v.sys.comcast.net (resqmta-po-12v.sys.comcast.net [96.114.154.171]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 1DFB15F4EE for ; Tue, 6 Jun 2017 03:02:32 +0000 (UTC) Received: from resomta-po-17v.sys.comcast.net ([96.114.154.241]) by resqmta-po-12v.sys.comcast.net with SMTP id I4l9dD6KBjQNSI4lGd2qbK; Tue, 06 Jun 2017 03:02:26 +0000 Received: from tinybb.boudnik.org ([107.3.179.163]) by resomta-po-17v.sys.comcast.net with SMTP id I4lFdA8PBijgRI4lFdeaNh; Tue, 06 Jun 2017 03:02:26 +0000 Received: by tinybb.boudnik.org (Postfix, from userid 1002) id B636549F0; Mon, 5 Jun 2017 20:02:47 -0700 (PDT) Date: Mon, 5 Jun 2017 20:02:47 -0700 From: Konstantin Boudnik To: dev@ignite.apache.org Subject: Re: Fwd: Persistent Distributed Store Metrics Message-ID: <20170606030247.GC16350@boudnik.org> Mail-Followup-To: dev@ignite.apache.org References: <20170606014625.GB16350@boudnik.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Organization: It's something of 'Cos User-Agent: Mutt/1.5.21 (2010-09-15) X-CMAE-Envelope: MS4wfE/gWaAHSEIIVlLUb4J7HxOVnRu/lYgwGP+uGcrQONEEUNNIFyhj4XrauBsQ3oGqblE/dxknSqitQMYY+oMl3iGZpL1xsKFvgpWfNXYumAj1OcKiyIVV gwOyt5rxRH2/CSL9myB7twcnY5pHNDKFa5wcT8LAN2JvdEG5UMOsdTQl On Mon, Jun 05, 2017 at 07:41PM, Dmitriy Setrakyan wrote: > On Mon, Jun 5, 2017 at 6:46 PM, Konstantin Boudnik wrote: > > > Wow, hold on - as far as I remember there was a VOTE to accept the > > contribution of the code into the project _on a branch_. We haven't vetted > > its > > inclusion into the next release, We are still at the phase of getting > > familiar > > with the code. > > > > Cos, the community has been vetting the inclusion of the new code for over > 3 weeks already (the process and dates are documented here [2]). To be > honest, I am not sure what the appropriate time frame should be, but I > would think that a month would be a good check-in point. I would think it should be as long as we, as the community, are comfortable with the massive change coming into the release. We already been through the discussion on the timing, etc. and I don't want to harp on that. > There is also an active stabilization thread for the persistence branch > [3]. I encouraged the community to get involved and post any questions or > concerns there as well. > > There is an upcoming in-memory computing conference that is coming up in > June in Amsterdam [4], where there are many Ignite talks scheduled. It > would be great to be able to talk about the persistence features of Ignite > there as well. I would like to ask the community to mobilize with reviewing > the donated code, so we could have something concrete to tell the audience > on the conference. I am sure that having the code on the branch is good enough to be able to talk about this. Having this in the release isn't really a contingency to be able to make a presentation, right? Cos > [2] > http://incubator.apache.org/ip-clearance/persistent-distributed-store-ignite.html > [3] > http://apache-ignite-developers.2346864.n4.nabble.com/Persistent-Store-Stabilization-for-release-td18288.html > [4] https://imcsummit.org/ > > > > > And from what I am seeing in the discussions like this [1], we need to be > > extra careful. > > > > I would keep the discussion in [1] separate from the persistence store. > These are 2 unrelated issues. I will respond on [1] either today or > tomorrow, but I agree in general that it should be fixed ASAP. > > > > BTW, you have sent this email 9 days before the vote had happened! A bit > > too > > soon, if you ask me. > > > > Cos, this email was sent 1 week after the initial donated code branch was > presented to the community (see [2] above). The developers involved were > eager to make it available to the users as soon as possible, but no code > has been merged to the master branch yet. > > I would like to encourage everyone in the community to participate in the > persistence branch coding discussions, like the one in this thread, and > familiarize themselves with the code. > > > > > > [1] https://is.gd/UQCr51 > > > > Cos > > > > On Wed, May 17, 2017 at 11:16AM, Dmitriy Govorukhin wrote: > > > Folk, > > > > > > As you know, ignite 2.1 will contain new module (pds), it will be > > > provide ability to store data on disk. Let's discuss what type of > > > metrics we need for this? > > > I think it must be metrics per memory policy, per cache, checkpoint, > > > and global metrics which will be aggregate all metrics. > > > > > > I did sketch. > > > > > > PersistentStoreMetrics.java > > > > > > public interface PersistentStoreMetrics { > > > > > > // Global metrics. > > > > > > public long getMemorySize(); > > > > > > public long getDiskSize(); > > > > > > public long getPagesInMemory(); > > > > > > public long getPagesSizeInMemory(); > > > > > > public long getPagesOnDisk(); > > > > > > public long getPagesSizeOnDisk(); > > > > > > public long getFreePages(); > > > > > > public long getFreePagesSize(); > > > > > > public long getDirtyPages(); > > > > > > public long getDirtyPagesSize(); > > > > > > public long walLog(); > > > > > > public long walLogSize(); > > > > > > // Frequency. > > > > > > public long getPagesRead(); > > > > > > public long getPagesWrite(); > > > > > > public long getFsync(); > > > > > > public long getWal(); > > > > > > public long getAverageWalFsyncTime(); > > > > > > // Per cache. > > > > > > public PersistentStoreCacheMetrics cache(String name); > > > > > > public PersistentStoreCacheMetrics cache(int cacheId); > > > > > > // For last checkpoint. > > > > > > public PersistentStoreCheckpointMetrics getLastCheckPoint(); > > > } > > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > PersistentStoreCacheMetrics.java > > > > > > public interface PersistentStoreCacheMetrics { > > > > > > public String name(); > > > > > > public double getFillFactor(); > > > > > > public double getFillFactor(int part); > > > > > > public long getMemorySize(); > > > > > > public long getDiskSize(); > > > > > > public long getPagesInMemory(); > > > > > > public long getPagesSizeInMemory(); > > > > > > public long getPagesOnDisk(); > > > > > > public long getPagesSizeOnDisk(); > > > > > > public long getFreePages(); > > > > > > public long getFreePagesSize(); > > > > > > public long getDirtyPages(); > > > > > > public long getDirtyPagesSize(); > > > > > > public long getPagesRead(); > > > > > > public long getPagesWritten(); > > > } > > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > PersistentStoreCheckpointMetrics.java > > > > > > public interface PersistentStoreCheckpointMetrics { > > > > > > public long getTotalPages(); > > > > > > //TODO Page type is internal? > > > public long[] pagesType(); > > > > > > public long getExecutingTime(); > > > > > > public long getFsyncTime(); > > > > > > public long getPagesCopyOnWrite(); > > > } > >