ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Tupitsyn <ptupit...@apache.org>
Subject Re: [DISCUSS] Ignite 3.0 development approach
Date Fri, 27 Nov 2020 12:59:56 GMT
> migrating to 'CI/CD as a Code'

Huge +1 for this.


> where should the code be stored ..
> alongside project's code (can be possible security hole)

Storing CI/CD code (yaml definitions for Travis/Azure/GH Actions, Jenkins
pipelines, etc) in the same repo is very common.
Secrets are stored separately (e.g. GitHub secrets), TeamCity probably has
a similar feature.


On Fri, Nov 27, 2020 at 3:28 PM Petr Ivanov <mr.weider@gmail.com> wrote:

> More or less, unless we specifically forbid that, I guess
>
> However there is bigger concern: JDK 15 is STS, so after half of a year it
> will be out of support and no major production team will use that JDK in
> their environment.
> I would stick to JDK 11 as it is LTS at least until JDK17, plus — a lot of
> efforts should be made to enforce Apache Ignite be built on JDK 11 alone,
> not to mention 15th version...
>
>
>
> Also, If we are going to introduce such major changes, I'd like to purpose
> maven project refactoring as well:
> 1. Full revision of all ant-calling tasks with javascript functions and
> alike — the complexity of those are overwhelming, something new should be
> at least researched.
> 2. Full revisions of profiles (for both root and modules) as there are
> some obsolete ones, and some that do ambiguous or, even worse, totally
> unknown tasks.
> 3. Introduce plugin and dependency management sections to control over and
> concrete versions of software we are relying in our project. Additionally —
> add BOM with all Ignite modules and their dependencies, which should help
> our users to better embed Ignite to their projects.
> 4. Up all versions of plugins and dependencies where possible to there
> current production versions (for plugins — it should be a must if we are
> ever going to build project under latest JDK versions).
> 5. Prepare project for parallel building, testing and assembling where
> possible for faster development process, with possible additional
> enhancement like incremental rebuild.
> 6. Possibly — research alternate builders, like Gradle (thought there are
> a lot of questions to its race condition issues during multimodule builds).
>
>
>
> And last, but not least — think of migrating to 'CI/CD as a Code' approach
> for our main instrument TeamCity.
> Whole project (both test and release build configurations) can be
> described using DSL (Kotlin in case of TC) and stored in VCS, forcing
> infrastructure changes to go through the same development processes
> including discussions, review, change history and etc.
>
> Only I am not sure for now about where should the code be stored — in
> separate repository (secure, but disables testing of code with TC settings
> both in single PR), or alongside project's code (can be possible security
> hole).
> That would require additional dev thread I think.
>
>
>
> WDYT?
>
> > On 24 Nov 2020, at 20:04, Pavel Tupitsyn <ptupitsyn@apache.org> wrote:
> >
> > If we use Java15 for development, can the resulting package be used from
> a
> > Java11 app (the latest LTS)?
> >
> > On Tue, Nov 24, 2020 at 7:51 PM Andrey Mashenkov <
> andrey.mashenkov@gmail.com>
> > wrote:
> >
> >> Jave15 looks awesome.
> >>
> >> * Hidden classes [1] can be used by codegenerators.
> >> * Records [2] can replace boilerplate code like IgniteBiTuple,
> GridTupleX.
> >>
> >> [1] https://openjdk.java.net/jeps/371
> >> [2] https://openjdk.java.net/jeps/384
> >>
> >> On Tue, Nov 24, 2020 at 3:38 PM Alexey Zinoviev <zaleslaw.sin@gmail.com
> >
> >> wrote:
> >>
> >>> Java 15 is better, VarHandles, ForeignMemory access and so on.
> >>>
> >>> In both cases, I support the Java version 11 and higher for the
> >>> development!
> >>>
> >>> вт, 24 нояб. 2020 г. в 15:21, Andrey Mashenkov <
> >> andrey.mashenkov@gmail.com
> >>>> :
> >>>
> >>>> Let's add maven plugins  for sanity checks at the early stage.
> >>>> I've created a ticket for this [1].
> >>>>
> >>>> Also, I've found initial pom.xml has a target version Java 8.
> >>>> Do we intend to move to Java 11 (or may be next LTS) and drop Java 8
> in
> >>>> Ignite 3.0?
> >>>>
> >>>> [1] https://issues.apache.org/jira/browse/IGNITE-13751
> >>>>
> >>>> On Tue, Nov 24, 2020 at 5:40 AM Valentin Kulichenko <
> >>>> valentin.kulichenko@gmail.com> wrote:
> >>>>
> >>>>> Folks,
> >>>>>
> >>>>> I went ahead and created the repository [1]. I also configured a
> >>> TeamCity
> >>>>> project [2] that runs all available JUnit tests on every PR creation
> >> or
> >>>>> update. It also sends the status update to GitHub so that it's
> >>> reflected
> >>>> in
> >>>>> the PR itself so that we can do merges directly from GitHub. Basic
> >>> steps
> >>>> to
> >>>>> make a change are described on the Wiki page [3].
> >>>>>
> >>>>> Let me know if you have any questions.
> >>>>>
> >>>>> [1] https://github.com/apache/ignite-3
> >>>>> [2] https://ci.ignite.apache.org/project/ignite3
> >>>>> [3]
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0#ApacheIgnite3.0-DevelopmentProcess
> >>>>>
> >>>>> -Val
> >>>>>
> >>>>> On Wed, Nov 18, 2020 at 4:24 PM Valentin Kulichenko <
> >>>>> valentin.kulichenko@gmail.com> wrote:
> >>>>>
> >>>>>> Thanks, guys. It looks like we are much closer to the consensus
> >> now.
> >>> I
> >>>>>> totally on board with the plan, but I would also like to address
> >> the
> >>>>>> short-term needs. As I've already mentioned earlier, there are
> >>> several
> >>>>>> active IEPs, but we still don't have even a preliminary technical
> >>>> process
> >>>>>> for working on these IEPs. I believe this might be frustrating
for
> >>> the
> >>>>>> folks who would like to commit code.
> >>>>>>
> >>>>>> The scope we agreed on is quite big, and it will surely take
> >>>> significant
> >>>>>> time to implement all the changes and stabilize them. Therefore,
> >> it's
> >>>>> clear
> >>>>>> to me that we will have to maintain 2.x and 3.x in parallel
for
> >> quite
> >>>>> some
> >>>>>> time - this needs to be addressed somehow. I'm convinced that
> >> having
> >>> a
> >>>>>> separate repo is the ONLY way to do that, and so far, I haven't
> >> heard
> >>>> any
> >>>>>> clear alternatives or reasons why we shouldn't do this.
> >>>>>>
> >>>>>> That said, I'm inclined to proceed with this in the next few
days
> >> - I
> >>>>> will
> >>>>>> create a repo and describe the process (which we, of course,
can
> >>>> discuss
> >>>>>> and modify going forward). Let's, at the very least, try and
see
> >>> where
> >>>> it
> >>>>>> leads us.
> >>>>>>
> >>>>>> If someone has any concrete alternative options on how to we
can
> >>>> maintain
> >>>>>> two major versions in parallel, let's have another voice discussion
> >>>> this
> >>>>>> Friday. If we do the meeting, we should set it up with a clear
goal
> >>> to
> >>>>> make
> >>>>>> a decision. Please let me know if there is interest in this.
> >>>>>>
> >>>>>> -Val
> >>>>>>
> >>>>>> On Mon, Nov 16, 2020 at 6:31 AM Alexey Goncharuk <
> >>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>
> >>>>>>> Good,
> >>>>>>>
> >>>>>>> I think we have an intermediate agreement on the scope and
> >>>> significance
> >>>>> of
> >>>>>>> the changes we want to make. I suggest creating separate
> >> discussion
> >>>>>>> streams
> >>>>>>> and calls for each of the suggested topics so that:
> >>>>>>>
> >>>>>>>   - It is clear for the community what is the motivation
of the
> >>>> stream
> >>>>>>>   (this includes both functional targets and technical debt
> >> issues
> >>>>>>> pointed
> >>>>>>>   out by Sergey)
> >>>>>>>   - Who is planning to take an active part in each of the
streams
> >>>> (i.e.
> >>>>>>>   the 'design committee', as Sergey suggested)
> >>>>>>>   - What are the intermediate and final goals for each of
the
> >>> streams
> >>>>>>>   - What are the cross-stream interactions and how we integrate
> >>> them
> >>>>>>>   - How each of the streams will be integrated with the
current
> >>>>> codebase
> >>>>>>>   based on the above (here is where we will see whether
drop-in
> >> or
> >>>>>>>   incremental approaches make more sense)
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Best regards,
> >>>> Andrey V. Mashenkov
> >>>>
> >>>
> >>
> >>
> >> --
> >> Best regards,
> >> Andrey V. Mashenkov
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message