ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Apache Ignite 2.1 scope
Date Thu, 06 Jul 2017 08:04:54 GMT
Vyacheslav, Denis,

7 July is too abrupt date. Scope of 2.1 is still too broad, and what is
more important - persistent store has been merged only several days ago. We
need some room for stabilization. I propose the following timeline:
16 July - code freeze
17-21 July - QA
21-24 July - vote and release

On Thu, Jul 6, 2017 at 4:30 AM, Konstantin Boudnik <cos@apache.org> wrote:

> Thanks everyone for giving us enough time to take a look into the code
> and architecture of this new feature. The webinar was certainly quite
> helpful (thanks Denis!).
>
> It seems to be a good time to add the feature into the dot-release, so
> more users can have a taste of it "officially". I have a somewhat
> unrelated question though: it seems that 2.0 has significant
> performance degradation compared to 1.8 when it get to the working
> with external distributed storage (like Cassandra). Could it be caused
> by all the changes that were made between 1.8 and 2.0 in the
> preparation for the coming persistent store functionality? Are we
> publishing/collecting say yardstick reports for our own releases?
>
> Thanks!
>   Cos
> --
>   Take care,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
>
> Disclaimer: Opinions expressed in this email are those of the author,
> and do not necessarily represent the views of any company the author
> might be affiliated with at the moment of writing.
>
>
> On Tue, Jul 4, 2017 at 3:20 AM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
> > Igniters,
> >
> > Persistent store has been merged to master branch! "master-bak" branch
> was
> > created to keep the state before merge for safety. As release date for
> 2.1
> > is mid July, I created "ignite-2.1" branch where we will stabilize the
> > release as usual. Please push features and fixes planned for 2.1 release
> to
> > this branch. The rest commits should go to master.
> >
> > Vladimir.
> >
> > On Mon, Jul 3, 2017 at 4:18 PM, Vladimir Ozerov <vozerov@gridgain.com>
> > wrote:
> >
> >> Hi Denis,
> >>
> >> Awesome news! I'll take care of necessary release procedures if nobody
> >> minds.
> >>
> >> Vladimir.
> >>
> >> On Sat, Jul 1, 2017 at 12:25 AM, Denis Magda <dmagda@apache.org> wrote:
> >>
> >>> Igniters,
> >>>
> >>> It’s time to refresh this abandoned thread and finally rollout out all
> >>> the changes appeared in 2.1.
> >>>
> >>> In addition, recently donated Persistent Store got the green light [1]
> to
> >>> become a part of the master branch (no one asked for extra time to dive
> >>> into its details) and, personally, it’s absolutely fine to make it
> >>> available in the nearest release.
> >>>
> >>> My proposal is to do the release by mid of July (closer to July 15th).
> Is
> >>> there anyone who is ready to take over as a release manager creating
> the
> >>> page like this [2] and handling all release related activities?
> >>>
> >>>
> >>> [1] http://apache-ignite-developers.2346864.n4.nabble.com/
> >>> Ignite-Persistent-Store-Ready-for-merge-td19160.html
> >>> [2] https://cwiki.apache.org/confluence/display/IGNITE/
> Apache+Ignite+2.0
> >>>
> >>> —
> >>> Denis
> >>>
> >>> > On Jun 1, 2017, at 9:24 AM, Alexander Paschenko <
> >>> alexander.a.paschenko@gmail.com> wrote:
> >>> >
> >>> > IGNITE-5327 Create predefined cache templates for CREATE TABLE
> command
> >>> > - minor comments left, ETA is Friday.
> >>> >
> >>> > IGNITE-5380 Validate cache QueryEntities in discovery thread - in
> >>> > progress, the meat of code is written, but need to add lots of tests.
> >>> > ETA is Friday.
> >>> >
> >>> > IGNITE-5188 Support AFFINITY KEY keyword for CREATE TABLE command -
> in
> >>> > progress, made few first small steps, ETA is Friday.
> >>> >
> >>> > Rest is closed/patch available, ignite-4994 has been moved to 2.2.
> >>> >
> >>> > - Alex
> >>> >
> >>> > 2017-06-01 19:03 GMT+03:00 Sergey Chugunov <
> sergey.chugunov@gmail.com>:
> >>> >>   1. IGNITE-5386 Inactive mode must be forced on starting up grid
> with
> >>> >>   persistence is enabled
> >>> >>   It is important improvement to fix critical bug IGNITE-5363.
> >>> >>   Working on it, ETA - tomorrow.
> >>> >>   2. IGNITE-5375 New PersistentStoreMetrics, MemoryMetrics interface
> >>> >>   improvements
> >>> >>   A lot of discussions were on this topic, ticket created only
today
> >>> and
> >>> >>   requires several days to implement.
> >>> >>
> >>> >>
> >>> >>
> >>> >> On Thu, Jun 1, 2017 at 6:56 PM, Taras Ledkov <tledkov@gridgain.com>
> >>> wrote:
> >>> >>
> >>> >>> Folks,
> >>> >>>
> >>> >>> IGNITE-4922 JDBC Driver: renew thin client based solution:
> >>> >>>
> >>> >>> On 2.1 the functionality of the new thin client JDBC driver
will be
> >>> >>> between deprecated Ignite thin JDBC and Ignite JDBCv2.
> >>> >>> 1. The most functions of SQL query (include DML) are implemented
> and
> >>> ready
> >>> >>> for review;
> >>> >>> 2. The most functions of JDBC metadata are implemented and
ready
> for
> >>> >>> review;
> >>> >>> 3. Transactions, batching, streaming, blobs, scrollable / writable
> >>> cursors
> >>> >>> will not be supported in 2.1.
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> On 01.06.2017 18:43, Vladimir Ozerov wrote:
> >>> >>>
> >>> >>>> Folks,
> >>> >>>>
> >>> >>>> We are almost reached proposed feature-complete date (June
2),
> Could
> >>> you
> >>> >>>> please share current status of your major features?
> >>> >>>>
> >>> >>>> On Tue, May 16, 2017 at 3:51 AM, Dmitriy Setrakyan <
> >>> dsetrakyan@apache.org
> >>> >>>>>
> >>> >>>> wrote:
> >>> >>>>
> >>> >>>> Looks a little tight. Let's hope we can make it.
> >>> >>>>>
> >>> >>>>> On Mon, May 15, 2017 at 1:29 PM, Denis Magda <dmagda@apache.org>
> >>> wrote:
> >>> >>>>>
> >>> >>>>> Well, let me propose the following milestones for 2.1
release
> then.
> >>> >>>>>>
> >>> >>>>>> Code freeze: June 2nd.
> >>> >>>>>> Final QA and benchmarking: June 5 - June 8
> >>> >>>>>> Voting: ~ June 9
> >>> >>>>>> Release: ~ June 13
> >>> >>>>>>
> >>> >>>>>> Also I heard H2 has to be released once again to
support
> Ignite’s
> >>> CREATE
> >>> >>>>>> table command. Think that we should talk to H2
folks to make it
> >>> happen
> >>> >>>>>> in
> >>> >>>>>> June 22nd - June 2nd time frame.
> >>> >>>>>>
> >>> >>>>>> —
> >>> >>>>>> Denis
> >>> >>>>>>
> >>> >>>>>> On May 11, 2017, at 2:26 AM, Pavel Tupitsyn <
> ptupitsyn@apache.org>
> >>> >>>>>>>
> >>> >>>>>> wrote:
> >>> >>>>>>
> >>> >>>>>>> As for .NET, I would propose to concentrate
on peer deployment
> >>> >>>>>>>
> >>> >>>>>> (IGNITE-2492)
> >>> >>>>>>
> >>> >>>>>>> and related stuff, like IGNITE-1894 .NET: Delegate
support in
> the
> >>> API
> >>> >>>>>>>
> >>> >>>>>> via
> >>> >>>>>
> >>> >>>>>> extension methods.
> >>> >>>>>>>
> >>> >>>>>>> SQL Dependency does not look important to me,
we can reschedule
> >>> it for
> >>> >>>>>>> later versions.
> >>> >>>>>>>
> >>> >>>>>>> On Thu, May 11, 2017 at 12:01 PM, Dmitriy Setrakyan
<
> >>> >>>>>>>
> >>> >>>>>> dsetrakyan@apache.org>
> >>> >>>>>>
> >>> >>>>>>> wrote:
> >>> >>>>>>>
> >>> >>>>>>> Vyacheslav, I think it is worth the research,
but you should
> >>> always
> >>> >>>>>>>>
> >>> >>>>>>> keep
> >>> >>>>>
> >>> >>>>>> data querying and indexing in mind. For example,
I don't see how
> >>> >>>>>>>>
> >>> >>>>>>> by-page
> >>> >>>>>
> >>> >>>>>> compression will solve it.
> >>> >>>>>>>>
> >>> >>>>>>>> On Thu, May 11, 2017 at 1:52 AM, Vyacheslav
Daradur <
> >>> >>>>>>>>
> >>> >>>>>>> daradurvs@gmail.com>
> >>> >>>>>>
> >>> >>>>>>> wrote:
> >>> >>>>>>>>
> >>> >>>>>>>> Dmitriy,
> >>> >>>>>>>>>
> >>> >>>>>>>>> I'm researching a best way for this
future.
> >>> >>>>>>>>>
> >>> >>>>>>>>> At the moment I found only one way
(querying and indexing
> >>> >>>>>>>>>
> >>> >>>>>>>> compatible),
> >>> >>>>>
> >>> >>>>>> this
> >>> >>>>>>>>
> >>> >>>>>>>>> is per-objects-field compression.
> >>> >>>>>>>>>
> >>> >>>>>>>>> But there is a good proffit only for
long strings or fields
> with
> >>> >>>>>>>>>
> >>> >>>>>>>> large
> >>> >>>>>
> >>> >>>>>> objects.
> >>> >>>>>>>>>
> >>> >>>>>>>>> Maybe it makes sense just to introduce
compression for string
> >>> fileds.
> >>> >>>>>>>>>
> >>> >>>>>>>>> I'm researching the new page-memory
architecture as applied
> to
> >>> >>>>>>>>>
> >>> >>>>>>>> by-page
> >>> >>>>>
> >>> >>>>>> compression.
> >>> >>>>>>>>>
> >>> >>>>>>>>> 2017-05-11 11:30 GMT+03:00 Dmitriy
Setrakyan <
> >>> dsetrakyan@apache.org
> >>> >>>>>>>>>
> >>> >>>>>>>> :
> >>> >>>>>>
> >>> >>>>>>> On Thu, May 11, 2017 at 12:44 AM, Vyacheslav
Daradur <
> >>> >>>>>>>>>>
> >>> >>>>>>>>> daradurvs@gmail.com>
> >>> >>>>>>>>>
> >>> >>>>>>>>>> wrote:
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> Denis,
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> The described roadmap looks
great!
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> Additional, I vote for introducing
an ability (OOTB) to
> store
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>> objects
> >>> >>>>>
> >>> >>>>>> in
> >>> >>>>>>>>>
> >>> >>>>>>>>>> a
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>> cache in a compressed form.
> >>> >>>>>>>>>>> This will allow to store more
data at the cost of
> incriasing
> >>> of CPU
> >>> >>>>>>>>>>> utilization.
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> One of the problems with compression
is indexing and
> >>> querying. How
> >>> >>>>>>>>>>
> >>> >>>>>>>>> do
> >>> >>>>>
> >>> >>>>>> we
> >>> >>>>>>>>
> >>> >>>>>>>>> index the data if it is compressed?
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>> 2017-05-11 4:23 GMT+03:00 Denis
Magda <dmagda@apache.org>:
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> Igniters,
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> Let me start a discussion
around the scope for 2.1
> release.
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> In my vision the main direction
of our ongoing efforts
> >>> should be
> >>> >>>>>>>>>>>> implementing in life a
use case of Ignite as a
> transactional
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> distributed
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>> SQL database and HTAP platform.
The current use cases
> >>> (database
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> cache,
> >>> >>>>>>>>>
> >>> >>>>>>>>>> data
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>> grid, micro services platform,
etc.) will be supported as
> >>> usual,
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> no
> >>> >>>>>
> >>> >>>>>> changes
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>> on that frontier.
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> Keeping this in mind, the
roadmap needs to include
> essential
> >>> SQL
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> related
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>> features as well as disk based
capabilities, MVCC support,
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> advanced
> >>> >>>>>
> >>> >>>>>> DDL
> >>> >>>>>>>>>
> >>> >>>>>>>>>> implementation and so on so forth.
This is for Ignite as a
> SQL
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> database.
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>> Next, Machine Learning will
be a great addition to Ignite
> as
> >>> an
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> HTAP
> >>> >>>>>>>>
> >>> >>>>>>>>> platform offering. This is why we should
keep investing our
> time
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> and
> >>> >>>>>>>>
> >>> >>>>>>>>> resources in that recently released
component.
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> Having this said, I see
the scope for 2.1 release this
> way:
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> 1. Distributed Persistent
Store - if the donation is
> >>> accepted by
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> ASF.
> >>> >>>>>>>>
> >>> >>>>>>>>> The
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>> decision is to be done in separate
discussion. W/o the
> store
> >>> >>>>>>>>>>>>   Ignite can only be used
as In-Memory SQL database.
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> 2. SQL Grid:
> >>> >>>>>>>>>>>>    - CREATE & DROP
table commands:
> >>> https://issues.apache.org/
> >>> >>>>>>>>>>>> jira/browse/IGNITE-4651
> >>> >>>>>>>>>>>>    - Renewed JDBC driver:
https://issues.apache.org/
> >>> >>>>>>>>>>>> jira/browse/IGNITE-4922
> >>> >>>>>>>>>>>>    - Collocation based
routing of SQL queries:
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> https://issues.apache.org/
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>> jira/browse/IGNITE-4510,
> >>> >>>>>>>>>>>>       https://issues.apache.org/jira/browse/IGNITE-4509
> >>> >>>>>>>>>>>>    -
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> 3. .NET:
> >>> >>>>>>>>>>>>    - Peer-class loading:
https://issues.apache.org/
> >>> >>>>>>>>>>>> jira/browse/IGNITE-2492
> >>> >>>>>>>>>>>>    - SQLDependency: https://issues.apache.org/
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> jira/browse/IGNITE-2657
> >>> >>>>>>>>>
> >>> >>>>>>>>>> 4. C++:
> >>> >>>>>>>>>>>>    - Compute Grid: https://issues.apache.org/
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> jira/browse/IGNITE-3574
> >>> >>>>>>>>>
> >>> >>>>>>>>>> 5. ML Grid:
> >>> >>>>>>>>>>>>    - Linear regression
algorithms:
> >>> https://issues.apache.org/
> >>> >>>>>>>>>>>> jira/browse/IGNITE-5012
> >>> >>>>>>>>>>>>    - K-means clustering:
https://issues.apache.org/
> >>> >>>>>>>>>>>> jira/browse/IGNITE-5113
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> Please join the thread
and share your thoughts, ideas and
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> concerns.
> >>> >>>>>
> >>> >>>>>> —
> >>> >>>>>>>>>>>> Denis
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> --
> >>> >>>>>>>>>>> Best Regards, Vyacheslav
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>> --
> >>> >>>>>>>>> Best Regards, Vyacheslav
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>
> >>> >>> --
> >>> >>> Taras Ledkov
> >>> >>> Mail-To: tledkov@gridgain.com
> >>> >>>
> >>> >>>
> >>>
> >>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message