metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Casey Stella <ceste...@gmail.com>
Subject Re: [DISCUSS] Next Release (0.3.1) Content
Date Fri, 03 Feb 2017 14:39:09 GMT
Thanks Mike,

I saw a +1 come in for 660 this morning and I'll give the weekend for
people to comment on 692 and add anything that is a breaking change in the
comments of that PR.  I'd like the first release candidate to go out monday
or tuesday for a vote.

On Fri, Feb 3, 2017 at 9:23 AM, Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:

> Hi Casey,
>
> My understanding from legal and our mentors is that as it currently stands,
> we would not have to wait.
>
> http://mail-archives.apache.org/mod_mbox/incubator-metron-
> dev/201701.mbox/%3CCAF1jEfCGVGd9tVDJfRwoxFjEmwHD59DuXTdMu-_OJQwcLvryog%
> 40mail.gmail.com%3E
>
> and
>
> http://mail-archives.apache.org/mod_mbox/www-legal-discuss/201702.mbox/%
> 3CCAOqetn-kB_ekCxB4z%2Bb0J%3DG0qJJKFepkfmMGa8OvBhxmPcvjvw
> %40mail.gmail.com%3E
>
>
> On Thu, Feb 2, 2017 at 3:03 PM, Casey Stella <cestella@gmail.com> wrote:
>
> > Just a quick release update.  As of now, we are waiting on
> >
> >    - METRON-660 to get reviewed and make it in
> >    - METRON-692 to get our upgrade.md completed for this release
> >
> > Mike Miklavcic, you sent an email to the legal-discuss about our kraken
> > dependency and it looked like we didn't have to change, but could you
> > comment on this thread in the dev list so I know if we need to wait for
> > METRON-650.
> >
> > Casey
> >
> > On Thu, Feb 2, 2017 at 4:58 PM, Casey Stella <cestella@gmail.com> wrote:
> >
> > > Ok, I've created the upgrading document for 0.3.0 to 0.3.1 and included
> > > the things that I know about and the things Jon mentioned here.
> Please,
> > if
> > > you have knowledge of other breaking/non-compatible changes between the
> > > 0.3.0 release and master, comment on this PR (
> https://github.com/apache/
> > > incubator-metron/pull/437) and I will incorporate them.
> > >
> > > On Fri, Jan 27, 2017 at 10:04 AM, Zeolla@GMail.com <zeolla@gmail.com>
> > > wrote:
> > >
> > >> To start I was mostly concerned with having a per-version list of
> > >> non-backwards-compatible changes, so upgrades that may skip a version
> of
> > >> two can look at what all may be impacted.  We should also probably
> > >> document
> > >> any sort of upgrade flaws as well, such as METRON-447
> > >> <https://issues.apache.org/jira/browse/METRON-447>, METRON-448
> > >> <https://issues.apache.org/jira/browse/METRON-448>, etc.  I do think
> > that
> > >> we should have a more rigorous document, but I wouldn't push that for
> > the
> > >> 0.3.1 release.  I see that (along with the Management UI, API, etc.)
> as
> > >> key
> > >> (required) components of a 1.0 release.  I'd just like to see the
> > >> foundation begin to be laid and iterated on.
> > >>
> > >> That said, this probably constitutes a mention in the development
> > >> guidelines
> > >> <https://cwiki.apache.org/confluence/pages/viewpage.action?
> > >> pageId=61332235>
> > >> once
> > >> it's in master.
> > >>
> > >> Jon
> > >>
> > >> On Fri, Jan 27, 2017 at 9:05 AM Casey Stella <cestella@gmail.com>
> > wrote:
> > >>
> > >> > I should add, you may be thinking something more rigorous and
> > >> > step-by-step.  If so, you think you might be interested in
> > volunteering
> > >> to
> > >> > do a first draft as a PR that we can adjust?
> > >> >
> > >> > On Fri, Jan 27, 2017 at 9:01 AM, Casey Stella <cestella@gmail.com>
> > >> wrote:
> > >> >
> > >> > > So, I agree with the Upgrading.md and I was going to submit a
PR
> at
> > >> least
> > >> > > to describe the the changes to indexing configurations that I
made
> > >> during
> > >> > > the 3.0.1 release.
> > >> > >
> > >> > >
> > >> > > On Thu, Jan 26, 2017 at 10:50 PM, Zeolla@GMail.com <
> > zeolla@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > >> I haven't had a chance to look through the unresolved JIRAs
but I
> > did
> > >> > want
> > >> > >> to mention a few quick things.
> > >> > >>
> > >> > >> First, when we released 0.3.0 and dropped the BETA flag,
one of
> the
> > >> > things
> > >> > >> that was discussed was putting together a method of documenting
> > >> upgrades
> > >> > >> from one version to the next.  As one of the first steps
toward
> > >> making
> > >> > >> that
> > >> > >> a reasonable process, I think we should assemble more detailed
> > >> release
> > >> > >> notes, especially outlining non-backwards compatible changes.
 In
> > the
> > >> > >> "[DISCUSS] Next Release Name" email thread Kyle Richardson
> > suggested
> > >> we
> > >> > >> use
> > >> > >> "UPGRADING.md" to do this, and I still agree with that thought,
> but
> > >> I'm
> > >> > >> open to alternatives.
> > >> > >>
> > >> > >> Separately, a *nice to have* would be *METRON-660*, which
was
> > >> discussed
> > >> > in
> > >> > >> the "[PROPOSAL] up-to-date versioned documentation" thread,
to
> give
> > >> us
> > >> > >> some
> > >> > >> cleaner documentation using the existing READMEs.  I'd be
happy
> to
> > >> help
> > >> > >> with this one, I'm just not sure what the next steps are,
aside
> > from
> > >> the
> > >> > >> start that Matt has here
> > >> > >> <https://github.com/mattf-horton/incubator-metron/tree/
> METRON-660
> > >.
> > >> > >>
> > >> > >> My last *nice to have* is *METRON-635*, which I have a *PR
open
> for
> > >> here
> > >> > >> <https://github.com/apache/incubator-metron/pull/411>*.
 If I
> > could
> > >> get
> > >> > >> someone else to reproduce the error that I'm seeing I would
be
> > happy
> > >> to
> > >> > >> pursue additional testing, troubleshooting, etc.  I've seen
> others
> > >> > report
> > >> > >> the same issue
> > >> > >> <https://community.hortonworks.com/search.html?f=&type=
> > >> > >> question&redirect=search%2Fsearch&sort=relevance&q=scp_if_ssh>
> > >> > >> on the HCC boards, so I'm fairly confident that it is not
an
> issue
> > >> with
> > >> > my
> > >> > >> local environment.
> > >> > >>
> > >> > >> Jon
> > >> > >>
> > >> > >> On Thu, Jan 26, 2017 at 6:18 PM Casey Stella <cestella@gmail.com
> >
> > >> > wrote:
> > >> > >>
> > >> > >> I took the liberty of adding the pull requests since the
new year
> > >> into
> > >> > the
> > >> > >> in-progress list in the previous email.  If you own one of
these
> > and
> > >> do
> > >> > >> not
> > >> > >> believe that you can complete the review in the next week
or so,
> > let
> > >> me
> > >> > >> know and we can remove.  The only one of these that I see
as
> > >> mandatory
> > >> > is
> > >> > >> METRON-650 because the reliance on the opensoc github repo
might
> > >> cause
> > >> > >> issues with the release being acceptable (see discussion
by Mike
> > >> > Miklavcic
> > >> > >> with the mentors a couple weeks ago).
> > >> > >>
> > >> > >> On Thu, Jan 26, 2017 at 6:06 PM, Casey Stella <
> cestella@gmail.com>
> > >> > wrote:
> > >> > >>
> > >> > >> > Hello Everyone!
> > >> > >> >
> > >> > >> > It's been almost 2 months since the last major release
and I
> > think
> > >> it
> > >> > >> > might be time to do a minor release of 0.3.1. The purpose
of
> this
> > >> > email
> > >> > >> is
> > >> > >> > multifold:
> > >> > >> >
> > >> > >> >    - to take stock of what we have already committed
> > >> > >> >    - figure out what (if anything) is missing to make
a release
> > >> that
> > >> > >> >    we're proud of
> > >> > >> >    - find volunteers for the missing items
> > >> > >> >
> > >> > >> > For those who get email alerts on the JIRA changes,
it should
> be
> > no
> > >> > >> > surprise that I have gone through and put the committed
items
> in
> > >> the
> > >> > >> 0.3.1
> > >> > >> > release bucket.  So, let's take stock of what we have.
> > >> > >> >
> > >> > >> > *What's made it so far into the next release*
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> >    - METRON-283 Migrate Geo Enrichment outside of MySQL
> > >> (justinleet)
> > >> > >> >    closes apache/incubator-metron#421
> > >> > >> >    - METRON-668: Remove the "tickUpdate" profile config
and
> make
> > >> the
> > >> > >> >    "init" phase not reset variables closes
> > >> apache/incubator-metron#420
> > >> > >> >    - METRON-672: SolrIndexingIntegrationTest fails
> intermittently
> > >> > closes
> > >> > >> >    apache/incubator-metron#424
> > >> > >> >    - METRON-664: Make the index configuration per-writer
with
> > >> > >> >    enabled/disabled closes apache/incubator-metron#419
> > >> > >> >    - METRON-600: Fix Metron Website (iraghumitra via
cestella)
> > >> closes
> > >> > >> >    apache/incubator-metron#399
> > >> > >> >    - METRON-666 Fix javadoc doclint errors closes
> > >> > >> >    apache/incubator-metron#418
> > >> > >> >    - METRON-659 Emulate Sensors in Development Environments
> > >> > (nickwallen)
> > >> > >> >    closes apache/incubator-metron#417
> > >> > >> >    - METRON-652: Extract indexing config from enrichment
config
> > >> closes
> > >> > >> >    apache/incubator-metron#415
> > >> > >> >    - METRON-654 Create RPM Installer for Profiler (nickwallen)
> > >> closes
> > >> > >> >    apache/incubator-metron#413
> > >> > >> >    - METRON-656: Make Stellar 'in' closer to functioning
like
> > >> python
> > >> > >> >    closes apache/incubator-metron#416
> > >> > >> >    - METRON-532 Define Profile Period When Calling PROFILE_GET
> > >> closes
> > >> > >> >    apache/incubator-metron#414
> > >> > >> >    - METRON-624: Updated Comparison/Equality Evaluations
in
> > Stellar
> > >> > >> >    closes apache/incubator-metron#404
> > >> > >> >    - METRON-644 RPM builds only work with Docker for
Mac
> > >> > >> (kylerichardson)
> > >> > >> >    closes apache/incubator-metron#409
> > >> > >> >    - METRON-640: Add a Stellar function to compute shannon
> > entropy
> > >> for
> > >> > >> >    strings closes apache/incubator-metron#403
> > >> > >> >    - METRON-637: Add a STATS_BIN function to Stellar.
closes
> > >> > >> >    apache/incubator-metron#401
> > >> > >> >    - METRON-622 Create a Metron Docker Compose application
> > >> > (merrimanr)
> > >> > >> >    closes apache/incubator-metron#393
> > >> > >> >    - METRON-643: Stellar function documentation needs
to be
> > updated
> > >> > >> >    closes apache/incubator-metron#407
> > >> > >> >    - METRON-645 Unable to Start Fastcapa Test Environment
> > >> (nickwallen)
> > >> > >> >    closes apache/incubator-metron#410
> > >> > >> >    - METRON-647 Parser unit test failures due to assumed
year
> > >> values
> > >> > >> >    (justinleet) closes apache/incubator-metron#412
> > >> > >> >    - METRON-639: The Network Stellar functions need
to have
> > better
> > >> > unit
> > >> > >> >    testing closes apache/incubator-metron#402
> > >> > >> >    - METRON-631 Broken link on fastcapa README (JonZeolla
via
> > >> > >> nickwallen)
> > >> > >> >    closes apache/incubator-metron#398
> > >> > >> >    - METRON-625: Parser Filters cannot be specified
from the
> > sensor
> > >> > >> >    config closes apache/incubator-metron#396
> > >> > >> >    - METRON-587 Integration tests should use common
processor
> > >> > >> >    implementations where possible (ottobackwards) closes
> > >> > >> >    apache/incubator-metron#374
> > >> > >> >    - METRON-616: Added support for float and long literals
in
> > >> Stellar
> > >> > >> >    closes apache/incubator-metron#392
> > >> > >> >    - METRON-580: Remove hard-coded Metron version from
Ambari
> > MPack
> > >> > code
> > >> > >> >    (mmiklavc) closes apache/incubator-metron#364
> > >> > >> >    - METRON-618 Eliminate Javac Warnings in metron-analytics
> > >> > >> (justinleet)
> > >> > >> >    closes apache/incubator-metron#391
> > >> > >> >    - METRON-364: Preserve Type for Arithmetic Expressions
in
> > >> Stellar
> > >> > >> >    closes apache/incubator-metron#390
> > >> > >> >    - METRON-585 Pcap Replay Fails to Stop (nickwallen)
closes
> > >> > >> >    apache/incubator-metron#369
> > >> > >> >    - METRON-586 STELLAR should have FILL_LEFT and FILL_RIGHT
> > >> functions
> > >> > >> >    (ottobackwards) closes apache/incubator-metron#370
> > >> > >> >    - METRON-612 Clean up Error Prone generated warnings
> > >> (justinleet)
> > >> > >> >    closes apache/incubator-metron#389
> > >> > >> >    - METRON-610: OnlineStatisticsProvider serialization
is
> broken
> > >> at
> > >> > >> >    random in the REPL closes apache/incubator-metron#388
> > >> > >> >    - METRON-596 Eliminate Maven warnings from build
> (justinleet)
> > >> > closes
> > >> > >> >    apache/incubator-metron#378
> > >> > >> >    - METRON-604 Mpack installs do not work on clean
machines
> due
> > to
> > >> > >> >    missing Elastic Curator repo (justinleet) closes
> > >> > >> apache/incubator-metron#385
> > >> > >> >    - METRON-607: Enrichment doc improvement and test
cleanup
> > >> > (mmiklavc)
> > >> > >> >    closes apache/incubator-metron#386
> > >> > >> >    - METRON-606 Profiler Overwriting Previously Written
Values
> > >> > >> >    (nickwallen) closes apache/incubator-metron#387
> > >> > >> >    - METRON-591: Make the website in compliance with
ASF
> > standards
> > >> > >> closes
> > >> > >> >    apache/incubator-metron#373
> > >> > >> >    - METRON-597 Sporadic Failures of Profiler Integration
Tests
> > >> > >> >    (nickwallen) closes apache/incubator-metron#383
> > >> > >> >    - METRON-595 Elasticsearch Writer only uses One IP
Address
> > >> > >> >    (JonathanRider via dlyle65535) closes
> > >> apache/incubator-metron#379
> > >> > >> >    - METRON-593 Enable an automated static analysis
tool in the
> > >> build
> > >> > >> >    (justinleet) closes apache/incubator-metron#376
> > >> > >> >    - METRON-592 Change popup to Warning message in Assign
> Master
> > >> page
> > >> > >> for
> > >> > >> >    Client check (anandsubbu via dlyle65535) closes
> > >> > >> apache/incubator-metron#375
> > >> > >> >    - METRON-598 Add Kyle Richardson to committers
> > (kylerichardson)
> > >> > >> closes
> > >> > >> >    apache/incubator-metron#382
> > >> > >> >    - METRON-565: apps/metron/enrichment/indexed directory
path
> > does
> > >> > not
> > >> > >> >    get created for metron cluster deployed via Ambari
> (mmiklavc)
> > >> > closes
> > >> > >> >    apache/incubator-metron#365
> > >> > >> >    - METRON-588 Remove ASF License Headers from directly
copied
> > >> Chef
> > >> > >> >    Bento Files closes apache/incubator-metron#372
> > >> > >> >    - METRON-576 Stellar function resolution takes too
long on
> > >> running
> > >> > >> >    cluster (nickwallen) closes apache/incubator-metron#366
> > >> > >> >    - METRON-570 Add Profiler Link to README (nickwallen)
closes
> > >> > >> >    apache/incubator-metron#371
> > >> > >> >    - METRON-578 Missing error handling bolts for enrichment
and
> > >> threat
> > >> > >> >    intel closes apache/incubator-metron#363
> > >> > >> >    - METRON-522 Need to mandate Client installation
on Metron
> > Host
> > >> > >> closes
> > >> > >> >    apache/incubator-metron#367
> > >> > >> >    - METRON-584 STELLAR should have a TO_LONG Function
> > >> (ottobackwards)
> > >> > >> >    closes apache/incubator-metron#368
> > >> > >> >    - METRON-575 State from different profiles can be
co-mingled
> > >> > >> >    incorrectly (nickwallen) closes apache/incubator-metron#362
> > >> > >> >    - METRON-538 Ensure proper shutdown, exceptions of
> integration
> > >> test
> > >> > >> >    components (ottobackwards) closes
> apache/incubator-metron#344
> > >> > >> >    - METRON-567 Usernames as numerics strings attempted
to be
> > >> parsed
> > >> > and
> > >> > >> >    compared as numbers (ottobackwards) closes
> > >> > >> apache/incubator-metron#360
> > >> > >> >    - METRON-562: Add rudimentary statistical outlier
detection
> > >> closes
> > >> > >> >    apache/incubator-metron#352
> > >> > >> >    - METRON-557 Create Stellar Functions for Kafka (nickwallen)
> > >> closes
> > >> > >> >    apache/incubator-metron#354
> > >> > >> >    - METRON-556 Profiler - Refactor 'Group By' Calculation
> > >> > (nickwallen)
> > >> > >> >    closes apache/incubator-metron#348
> > >> > >> >    - METRON-547 MaasIntegrationTests should integrate
with the
> > >> Metron
> > >> > >> >    Integration classes (ottobackwards) closes
> > >> > >> apache/incubator-metron#350
> > >> > >> >    - METRON-561 ShellEditor tests hang if vim is set
as EDITOR
> in
> > >> your
> > >> > >> >    profile (ottobackwards) closes apache/incubator-metron#351
> > >> > >> >
> > >> > >> > *What else will make it?*
> > >> > >> >
> > >> > >> > Ok, so now that we know what's in there already, I need
to ask
> > your
> > >> > help
> > >> > >> > in determining what you want in the next release that
is not
> > >> currently
> > >> > >> > being worked on.  In order to do that, please help me
by
> > >> responding to
> > >> > >> this
> > >> > >> > email and
> > >> > >> >
> > >> > >> >    - proposing and justifying a JIRA from the list of
> unresolved
> > >> JIRAs
> > >> > >> >    <
> > >> > >> https://issues.apache.org/jira/browse/METRON-469?jql=project
> > >> > >> %20%3D%20METRON%20AND%20resolution%20%3D%20Unresolved%20AND%
> > >> > >> 20fixVersion%20is%20EMPTY%20ORDER%20BY%20priority%20DESC>
> > >> > >> that
> > >> > >> >    are not currently scheduled and let me know which
ones
> should
> > be
> > >> > >> included.
> > >> > >> >    Please include whether the JIRA is "Nice to Have"
or
> "Critical
> > >> to
> > >> > >> Have" and
> > >> > >> >    if you volunteer to do it.
> > >> > >> >    - proposing and justifying that one of the in-progress
tasks
> > >> > >> >    <
> > >> > >> https://issues.apache.org/jira/browse/METRON-650?jql=project
> > >> > >> %20%3D%20METRON%20AND%20resolution%20%3D%20Unresolved%20AND%
> > >> > >> 20fixVersion%20%3D%200.3.1%20ORDER%20BY%20priority%20DESC
> > >> > >> >
> > >> > >> >     should *not* be included for release
> > >> > >> >
> > >> > >> > I'd like to aim to get release candidates out as soon
as
> > possible,
> > >> so
> > >> > >> > let's look forward a vigorous discussion and a great
release.
> > >> > >> >
> > >> > >> > Thanks for the contributions and for your continued
engagement
> in
> > >> the
> > >> > >> > Metron community.  If you see something that you don't
like in
> > this
> > >> > >> > process, please let me know.  Feedback is very welcome.
> > >> > >> >
> > >> > >> > Best,
> > >> > >> >
> > >> > >> > Casey
> > >> > >> >
> > >> > >>
> > >> > >> --
> > >> > >>
> > >> > >> Jon
> > >> > >>
> > >> > >> Sent from my mobile device
> > >> > >>
> > >> > >
> > >> > >
> > >> >
> > >> --
> > >>
> > >> Jon
> > >>
> > >> Sent from my mobile device
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message