ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Goncharuk <alexey.goncha...@gmail.com>
Subject Re: Persistent Distributed Store Metrics
Date Wed, 07 Jun 2017 10:10:40 GMT
Denis,

Thanks for the review. I addressed your comments and merged the changes to
ignite-5267 branch.

2017-06-06 22:56 GMT+03:00 Denis Magda <dmagda@apache.org>:

> > On Jun 5, 2017, at 8:02 PM, Konstantin Boudnik <cos@apache.org> wrote:
> >
> > On Mon, Jun 05, 2017 at 07:41PM, Dmitriy Setrakyan wrote:
> >> On Mon, Jun 5, 2017 at 6:46 PM, Konstantin Boudnik <cos@apache.org>
> wrote:
> >>
> >>> Wow, hold on - as far as I remember there was a VOTE to accept the
> >>> contribution of the code into the project _on a branch_. We haven't
> vetted
> >>> its
> >>> inclusion into the next release, We are still at the phase of getting
> >>> familiar
> >>> with the code.
> >>>
> >>
> >> Cos, the community has been vetting the inclusion of the new code for
> over
> >> 3 weeks already (the process and dates are documented here [2]). To be
> >> honest, I am not sure what the appropriate time frame should be, but I
> >> would think that a month would be a good check-in point.
> >
> > I would think it should be as long as we, as the community, are
> comfortable
> > with the massive change coming into the release. We already been through
> the
> > discussion on the timing, etc. and I don't want to harp on that.
>
> Just in case I’m working on the high-level storage documentation. Here is
> a draft for your reference:
> https://apacheignite.readme.io/docs/distributed-persistent-store <
> https://apacheignite.readme.io/docs/distributed-persistent-store>
>
> The page is invisible for wide audience at the moment. Only for Ignite
> committers.
>
> —
> Denis
>
> > On Jun 5, 2017, at 8:02 PM, Konstantin Boudnik <cos@apache.org> wrote:
> >
> > On Mon, Jun 05, 2017 at 07:41PM, Dmitriy Setrakyan wrote:
> >> On Mon, Jun 5, 2017 at 6:46 PM, Konstantin Boudnik <cos@apache.org>
> wrote:
> >>
> >>> Wow, hold on - as far as I remember there was a VOTE to accept the
> >>> contribution of the code into the project _on a branch_. We haven't
> vetted
> >>> its
> >>> inclusion into the next release, We are still at the phase of getting
> >>> familiar
> >>> with the code.
> >>>
> >>
> >> Cos, the community has been vetting the inclusion of the new code for
> over
> >> 3 weeks already (the process and dates are documented here [2]). To be
> >> honest, I am not sure what the appropriate time frame should be, but I
> >> would think that a month would be a good check-in point.
> >
> > I would think it should be as long as we, as the community, are
> comfortable
> > with the massive change coming into the release. We already been through
> the
> > discussion on the timing, etc. and I don't want to harp on that.
> >
> >> There is also an active stabilization thread for the persistence branch
> >> [3]. I encouraged the community to get involved and post any questions
> or
> >> concerns there as well.
> >>
> >> There is an upcoming in-memory computing conference that is coming up in
> >> June in Amsterdam [4], where there are many Ignite talks scheduled. It
> >> would be great to be able to talk about the persistence features of
> Ignite
> >> there as well. I would like to ask the community to mobilize with
> reviewing
> >> the donated code, so we could have something concrete to tell the
> audience
> >> on the conference.
> >
> > I am sure that having the code on the branch is good enough to be able
> to talk
> > about this. Having this in the release isn't really a contingency to be
> able
> > to make a presentation, right?
> >
> > Cos
> >
> >> [2]
> >> http://incubator.apache.org/ip-clearance/persistent-
> distributed-store-ignite.html
> >> [3]
> >> http://apache-ignite-developers.2346864.n4.nabble.com/Persistent-Store-
> Stabilization-for-release-td18288.html
> >> [4] https://imcsummit.org/
> >>
> >>
> >>
> >>> And from what I am seeing in the discussions like this [1], we need to
> be
> >>> extra careful.
> >>>
> >>
> >> I would keep the discussion in [1] separate from the persistence store.
> >> These are 2 unrelated issues. I will respond on [1] either today or
> >> tomorrow, but I agree in general that it should be fixed ASAP.
> >>
> >>
> >>> BTW, you have sent this email 9 days before the vote had happened! A
> bit
> >>> too
> >>> soon, if you ask me.
> >>>
> >>
> >> Cos, this email was sent 1 week after the initial donated code branch
> was
> >> presented to the community (see [2] above). The developers involved were
> >> eager to make it available to the users as soon as possible, but no code
> >> has been merged to the master branch yet.
> >>
> >> I would like to encourage everyone in the community to participate in
> the
> >> persistence branch coding discussions, like the one in this thread, and
> >> familiarize themselves with the code.
> >>
> >>
> >>>
> >>> [1] https://is.gd/UQCr51
> >>>
> >>> Cos
> >>>
> >>> On Wed, May 17, 2017 at 11:16AM, Dmitriy Govorukhin wrote:
> >>>> Folk,
> >>>>
> >>>> As you know, ignite 2.1 will contain new module (pds), it will be
> >>>> provide ability to store data on disk. Let's discuss what type of
> >>>> metrics we need for this?
> >>>> I think it must be metrics per memory policy, per cache, checkpoint,
> >>>> and global metrics which will be aggregate all metrics.
> >>>>
> >>>> I did sketch.
> >>>>
> >>>> PersistentStoreMetrics.java
> >>>>
> >>>> public interface PersistentStoreMetrics {
> >>>>
> >>>>    // Global metrics.
> >>>>
> >>>>    public long getMemorySize();
> >>>>
> >>>>    public long getDiskSize();
> >>>>
> >>>>    public long getPagesInMemory();
> >>>>
> >>>>    public long getPagesSizeInMemory();
> >>>>
> >>>>    public long getPagesOnDisk();
> >>>>
> >>>>    public long getPagesSizeOnDisk();
> >>>>
> >>>>    public long getFreePages();
> >>>>
> >>>>    public long getFreePagesSize();
> >>>>
> >>>>    public long getDirtyPages();
> >>>>
> >>>>    public long getDirtyPagesSize();
> >>>>
> >>>>    public long walLog();
> >>>>
> >>>>    public long walLogSize();
> >>>>
> >>>>    // Frequency.
> >>>>
> >>>>    public long getPagesRead();
> >>>>
> >>>>    public long getPagesWrite();
> >>>>
> >>>>    public long getFsync();
> >>>>
> >>>>    public long getWal();
> >>>>
> >>>>    public long getAverageWalFsyncTime();
> >>>>
> >>>>    // Per cache.
> >>>>
> >>>>    public PersistentStoreCacheMetrics cache(String name);
> >>>>
> >>>>    public PersistentStoreCacheMetrics cache(int cacheId);
> >>>>
> >>>>    // For last checkpoint.
> >>>>
> >>>>    public PersistentStoreCheckpointMetrics getLastCheckPoint();
> >>>> }
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>
> >>>> PersistentStoreCacheMetrics.java
> >>>>
> >>>> public interface PersistentStoreCacheMetrics {
> >>>>
> >>>>    public String name();
> >>>>
> >>>>    public double getFillFactor();
> >>>>
> >>>>    public double getFillFactor(int part);
> >>>>
> >>>>    public long getMemorySize();
> >>>>
> >>>>    public long getDiskSize();
> >>>>
> >>>>    public long getPagesInMemory();
> >>>>
> >>>>    public long getPagesSizeInMemory();
> >>>>
> >>>>    public long getPagesOnDisk();
> >>>>
> >>>>    public long getPagesSizeOnDisk();
> >>>>
> >>>>    public long getFreePages();
> >>>>
> >>>>    public long getFreePagesSize();
> >>>>
> >>>>    public long getDirtyPages();
> >>>>
> >>>>    public long getDirtyPagesSize();
> >>>>
> >>>>    public long getPagesRead();
> >>>>
> >>>>    public long getPagesWritten();
> >>>> }
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>
> >>>> PersistentStoreCheckpointMetrics.java
> >>>>
> >>>> public interface PersistentStoreCheckpointMetrics {
> >>>>
> >>>>    public long getTotalPages();
> >>>>
> >>>>    //TODO Page type is internal?
> >>>>    public long[] pagesType();
> >>>>
> >>>>    public long getExecutingTime();
> >>>>
> >>>>    public long getFsyncTime();
> >>>>
> >>>>    public long getPagesCopyOnWrite();
> >>>> }
> >>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message