metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zeolla@GMail.com" <zeo...@gmail.com>
Subject Re: [DISCUSS] Release?
Date Wed, 09 May 2018 17:36:39 GMT
I agree that it's probably time (more likely, overdue) for a release.
Based off of looking at all of those changes I would also suggest going to
at least 0.5.x.

It probably makes sense to take a look at Upgrading.md
<https://github.com/apache/metron/blob/master/Upgrading.md> (and related
docs) to make sure it's accurate as well.

Jon

On Wed, May 9, 2018 at 1:30 PM Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:

> Good call - I thought that made our last release, but this would be the 2nd
> follow-on from when Nick originally posed a breakdown
> 4 months ago METRON-939: Upgrade ElasticSearch and Kibana (mmiklavc via
> mmiklavc) closes apache/metron#840
>
>
> https://lists.apache.org/thread.html/01fb18dd0ee10845588c0c1a4b3f2f36d7a107c66edd2247f61756c1@%3Cdev.metron.apache.org%3E
>
> On Wed, May 9, 2018 at 11:18 AM, Zeolla@GMail.com <zeolla@gmail.com>
> wrote:
>
> > We should also mention the Upgrade of ElasticSearch and Kibana
> >
> > Jon
> >
> > On Wed, May 9, 2018 at 12:49 PM Nick Allen <nick@nickallen.org> wrote:
> >
> > > Oh, and also the Solr work that is currently in a feature branch.  We
> > would
> > > have to get the work finished up and merged though.  Sounds like we are
> > > real close on that.
> > >
> > > On Wed, May 9, 2018 at 12:47 PM, Nick Allen <nick@nickallen.org>
> wrote:
> > >
> > > > The next part of the conversation would be, what should the version
> > > number
> > > > be?  To help with that, I have tried to summarize the changes in the
> > > > release.  Of course, this is going to be heavily biased towards my
> own
> > > > interests, so please feel free to chime in if I have missed anything.
> > > >
> > > >
> > > >    - Significant performance improvements in Parsing, Enrichments,
> and
> > > >    Indexing
> > > >
> > > >
> > > >    - Support for Ubuntu installations
> > > >
> > > >
> > > >    - Support for Storm 1.1 and HDP 2.6
> > > >
> > > >
> > > >    - Event time processing in the Profiler
> > > >
> > > >
> > > >    - Complex document support in the JSONMapParser
> > > >
> > > >
> > > >    - Support for the Elasticsearch X-Pack
> > > >
> > > >
> > > >    - Run Stellar in a Zeppelin Notebook
> > > >
> > > >
> > > >    - Oodles of bug fixes and usability improvements
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, May 9, 2018 at 12:14 PM, Nick Allen <nick@nickallen.org>
> > wrote:
> > > >
> > > >> Something like this might be more digestible for these purposes.
> > > >>
> > > >> $git log --pretty="%cr %s" tags/apache-metron-0.4.2-release..HEAD
> > > >>
> > > >> 88 minutes ago METRON-1530 Default proxy config settings in
> > > >> metron-contrib need to be updated (sardell via merrimanr) closes
> > > >> apache/metron#998
> > > >> 5 days ago METRON-1545 Upgrade Spring and Spring Boot (merrimanr)
> > closes
> > > >> apache/metron#1008
> > > >> 7 days ago METRON-1543 Unable to Set Parser Output Topic in Sensor
> > > Config
> > > >> (nickwallen) closes apache/metron#1007
> > > >> 2 weeks ago METRON-1539: Specialized RENAME field transformer closes
> > > >> apache/incubator-metron#1002
> > > >> 2 weeks ago METRON-1520: Add caching for stellar field
> transformations
> > > >> closes apache/incubator-metron#990
> > > >> 2 weeks ago METRON-1529 CONFIG_GET Fails to Retrieve Latest Config
> > When
> > > >> Run in Zeppelin REPL (nickwallen) closes apache/metron#997
> > > >> 2 weeks ago METRON-1511 Unable to Serialize Profiler Configuration
> > > >> (nickwallen) closes apache/metron#982
> > > >> 3 weeks ago METRON-1528: Fix missing file in metron.spec (mmiklavc
> via
> > > >> mmiklavc) closes apache/metron#996
> > > >> 3 weeks ago METRON-1445: Update performance tuning guide with more
> > > >> explicit parameter instructions (mmiklavc via mmiklavc) closes
> > > >> apache/metron#988
> > > >> 3 weeks ago METRON-1502 Upgrade Doxia plugin to 1.8 (justinleet)
> > closes
> > > >> apache/metron#974
> > > >> 3 weeks ago METRON-1527: Remove dead test file sitting in source
> > folder
> > > >> (mmiklavc via mmiklavc) closes apache/metron#994
> > > >> 3 weeks ago METRON-1499 Enable Configuration of Unified Enrichment
> > > >> Topology via Ambari (nickwallen) closes apache/metron#984
> > > >> 3 weeks ago METRON-1515: Errors loading stellar functions currently
> > bomb
> > > >> the entire topology, they should be recoverable closes
> > > >> apache/incubator-metron#985
> > > >> 4 weeks ago METRON-1522 Fix the typo errors at profile debugger
> readme
> > > >> (MohanDV via nickwallen) closes apache/metron#992
> > > >> 4 weeks ago METRON-1519 Indexing Error Topic Property Not Displayed
> in
> > > >> MPack (nickwallen) closes apache/metron#987
> > > >> 4 weeks ago METRON-1347: Indexing Topology should fail tuples
> without
> > a
> > > >> source.type (cstella via mmiklavc) closes apache/metron#863
> > > >> 4 weeks ago Add support for Vagrant Cachier plugin if present
> > > >> (simonellistonball via mmiklavc) closes apache/metron#993
> > > >> 4 weeks ago METRON-1521: JSONMapParser is no longer serializable
> > closes
> > > >> apache/incubator-metron#991
> > > >> 4 weeks ago METRON-1516 Support for Ansible 2.5.0 (ottobackwards)
> > closes
> > > >> apache/metron#989
> > > >> 4 weeks ago METRON-1494 Profiler Emits Messages to Kafka When Not
> > Needed
> > > >> (nickwallen) closes apache/metron#967
> > > >> 4 weeks ago METRON-1510 Update Metron website to include info about
> > > >> github update subscription (anandsubbu) closes apache/metron#981
> > > >> 4 weeks ago METRON-1518 Build Failure When Using Profile HDP-2.5.0.0
> > > >> (nickwallen) closes apache/metron#986
> > > >> 4 weeks ago METRON-1465:Support for Elasticsearch X-pack (wardbekker
> > via
> > > >> mmiklavc) closes apache/metron#946
> > > >> 4 weeks ago METRON-1504: Enriching missing values does not match the
> > > >> semantics between the new enrichment topology and old closes
> > > >> apache/incubator-metron#976
> > > >> 5 weeks ago METRON-1505 Intermittent Profiler Integration Test
> Failure
> > > >> (nickwallen) closes apache/metron#977
> > > >> 5 weeks ago METRON-1449 Set Zookeeper URL for Stellar Running in
> > > Zeppelin
> > > >> Notebook (nickwallen) closes apache/metron#931
> > > >> 5 weeks ago METRON-1462: Separate ES and Kibana from Metron Mpack
> > > >> (mmiklavc via mmiklavc) closes apache/metron#943
> > > >> 5 weeks ago METRON-1501 Parser messages that fail to validate are
> > > dropped
> > > >> silently (cestella via justinleet) closes apache/metron#972
> > > >> 5 weeks ago METRON-1497 Rest endpoint '/api/v1/search/search' needs
> to
> > > >> handle null when elastic search response return null for
> > getAggregations
> > > >> (MohanDV via justinleet) closes apache/metron#968
> > > >> 7 weeks ago METRON-1500 Enhance 'prepare-commit' to Support Feature
> > > >> Branches (nickwallen) closes apache/metron#971
> > > >> 7 weeks ago METRON-1491: The indexing topology restart logic is
> wrong
> > > >> (cstella via mmiklavc) closes apache/metron#964
> > > >> 7 weeks ago METRON-590 Enable Use of Event Time in Profiler
> > (nickwallen)
> > > >> closes apache/metron#965
> > > >> 7 weeks ago METRON-1483: Create a tool to monitor performance of the
> > > >> topologies closes apache/incubator-metron#958
> > > >> 8 weeks ago METRON-1487 Define Performance Benchmarks for Enrichment
> > > >> Topology (nickwallen) closes apache/metron#961
> > > >> 8 weeks ago METRON-1493 Unhelpful Error Message When Assignment
> > > >> Expressions Fail (nickwallen) closes apache/metron#966
> > > >> 8 weeks ago METRON-1397 Support for JSON Path and complex documents
> in
> > > >> JSONMapParser closes apache/incubator-metron#914
> > > >> 8 weeks ago METRON-1299 In MetronError tests, don't test for
> HostName
> > if
> > > >> getHostName wouldn't work closes apache/incubator-metron#924
> > > >> 8 weeks ago METRON-1485 Upgrade vagrant for dev environments closes
> > > >> apache/incubator-metron#959
> > > >> 8 weeks ago METRON-1488: user_settings hbase table does not have
> acls
> > > set
> > > >> up for kerberos closes apache/metron#962
> > > >> 8 weeks ago METRON-1490: Better error message when user specifies
an
> > > >> enrichment type that doesn't exist closes apache/metron#963
> > > >> 9 weeks ago METRON-1468 Add support for
> apache/metron-bro-plugin-kafka
> > > to
> > > >> prepare-commit (JonZeolla) closes apache/metron#948
> > > >> 9 weeks ago METRON-1471: Migrate shuffle connections to local or
> > shuffle
> > > >> closes apache/metron#949
> > > >> 9 weeks ago METRON-1467: Replace guava caches in places where the
> > > >> keyspace might be large closes apache/metron#947
> > > >> 9 weeks ago METRON-1463: Adjust the groupings and shuffles in
> > enrichment
> > > >> to be more efficient closes apache/metron#944
> > > >> 9 weeks ago METRON-1460: Create a complementary non-split-join
> > > enrichment
> > > >> topology closes apache/metron#940
> > > >> 9 weeks ago METRON-1470 Update jquery to version 3+ (xyztdanid via
> > > >> merrimanr) closes apache/metron#950
> > > >> 9 weeks ago METRON-1450 Add REST endpoint docs for index topology
> > split
> > > >> (MohanDV via nickwallen) closes apache/metron#936
> > > >> 9 weeks ago METRON-1337 List of facets should not be hardcoded
> > > >> (merrimanr) closes apache/metron#853
> > > >> 9 weeks ago METRON-1452 Rebase Dev Environment on Latest CentOS 6
> > > >> (nickwallen) closes apache/metron#933
> > > >> 2 months ago METRON-1457 Move ASF links to main page in the Metron
> > > >> website (anandsubbu) closes apache/metron#938
> > > >> 3 months ago METRON-1394 Create Rest endpoint to add the ACL for
> > current
> > > >> user to kafka topics  (MohanDV via merrimanr) closes
> apache/metron#895
> > > >> 3 months ago METRON-941 native PaloAlto parser corrupts message when
> > > >> having a comma in the payload (ctramnitz via justinleet) closes
> > > >> apache/metron#579
> > > >> 3 months ago METRON-1386: Fix Metron Website Required Links closes
> > > >> apache/incubator-metron#935
> > > >> 3 months ago METRON-1455: Patch and Replace methods in the REST
> > > >> UpdateController return 400 this closes apache/incubator-metron#937
> > > >> 3 months ago METRON-1318 Update MacOS Instructions for AWS
> (wardbekker
> > > >> via nickwallen) closes apache/metron#930
> > > >> 3 months ago METRON-1451 On Centos full dev, Metron Indexing shows
> up
> > as
> > > >> stopped (anandsubbu via merrimanr) closes apache/metron#932
> > > >> 3 months ago METRON-1444: Add Ubuntu Repositories for Elasticsearch
> to
> > > >> the Mpack (mmiklavc via mmiklavc) closes apache/metron#928
> > > >> 3 months ago METRON-1273: Website documentation link should point
to
> > the
> > > >> current site-book (JonZeolla via mmiklavc) closes apache/metron#812
> > > >> 3 months ago METRON-1447 Heap Size Not Set Correctly by MPack for
ES
> > 5.x
> > > >> (nickwallen) closes apache/metron#927
> > > >> 3 months ago METRON-1446: Fix openjdk issue with Ubuntu (mmiklavc
> via
> > > >> mmiklavc) closes apache/metron#926
> > > >> 3 months ago METRON-1442: Split rest end points for indexing
> topology
> > > >> into random access indexing and batch indexing this closes
> > > >> apache/incubator-metron#923
> > > >> 3 months ago METRON-1443 Missing Critical MPack Install Instruction
> > for
> > > >> Ubuntu (nickwallen) closes apache/metron#925
> > > >> 3 months ago METRON-1438 STELLAR: Move shell functions to common
> from
> > > >> metron-management (ottobackwards) closes apache/metron#920
> > > >> 3 months ago METRON-1435 Management UI cannot save json objects in
> > > >> advanced config (merrimanr) closes apache/metron#917
> > > >> 3 months ago METRON-1439 Turn off git pager in platform-info script
> > > >> (justinleet) closes apache/metron#919
> > > >> 3 months ago METRON-1091 STELLAR Shell: Stand Alone installation
> > > >> (ottobackwards) closes apache/metron#690
> > > >> 3 months ago METRON-1427: Add support for storm 1.1 and hdp 2.6
> > (cstella
> > > >> via mmiklavc) closes apache/metron#907
> > > >> 3 months ago METRON-1391 Typos in Documentation/Examples within
> > > >> metron-management/README.md (havran via ottobackwards) closes
> > > >> apache/metron#890
> > > >> 3 months ago METRON-1389 Zeppelin notebook import does not work with
> > > >> Ambari 2.6 (anandsubbu) closes apache/metron#888
> > > >> 3 months ago METRON-1432 JDK Install Fails on Ubuntu Development
> > > >> Environment  (nickwallen) closes apache/metron#913
> > > >> 3 months ago METRON-1431 Add REGEXP_REPLACE function to Stellar
> > (havran
> > > >> via ottobackwards) closes apache/metron#912
> > > >> 3 months ago METRON-1410 Some more upgrade fallout... Can&apos;t
> > restart
> > > >> Metron Indexing. (ottobackwards) closes apache/metron#901
> > > >> 3 months ago METRON-1370 Create Full Dev Equivalent for Ubuntu
> > > >> (nickwallen via cestella) closes apache/incubator-metron#903
> > > >> 3 months ago METRON-1430: Isolate jackson from being used as
> arguments
> > > or
> > > >> returns from JSONUtils closes apache/metron#910
> > > >> 3 months ago METRON-1398 Exclude the basic-error-controller from
> being
> > > >> added to the swagger description (MohanDV via merrimanr) closes
> > > >> apache/metron#898
> > > >> 3 months ago METRON-1392 Fix a test case to expect an Exception when
> > > >> replication factor more than number of brokers while creating topic
> > > >> (MohanDV via merrimanr) closes apache/metron#892
> > > >> 3 months ago METRON-1413 Add Metron Commit Tool (nickwallen) closes
> > > >> apache/metron#902
> > > >> 3 months ago METRON-1429 SearchIntegrationTest refactor (merrimanr)
> > > >> closes apache/metron#909
> > > >> 3 months ago METRON-1426: SensorIndexingConfigController
> > IntegrationTest
> > > >> fails intermittently closes apache/metron#906
> > > >> 4 months ago METRON-1417: Disable pcap-service by default in Monit
> > > >> (mmiklavc via mmiklavc) closes apache/metron#905
> > > >> 4 months ago METRON-1400: Elasticsearch service check fails in
> Ambari
> > > >> (mmiklavc via mmiklavc) closes apache/metron#904
> > > >> 4 months ago METRON-1428: Travis build failing from metron-config
> > > >> (mmiklavc via mmiklavc) closes apache/metron#908
> > > >> 4 months ago METRON-1302: Split up Indexing Topology into batch and
> > > >> random access sections closes apache/incubator-metron#831
> > > >> 4 months ago METRON-1395 Documentation missing for Produce a message
> > to
> > > a
> > > >> Kafka topic Rest API endpoint (MohanDV via nickwallen) closes
> > > >> apache/metron#897
> > > >> 4 months ago METRON-1411 Fix sed command in Upgrading.md
> (justinleet)
> > > >> closes apache/metron#900
> > > >> 4 months ago METRON-1326: Metron deploy with Kerberos fails on
> Ambari
> > > 2.5
> > > >> during ES service stop (mmiklavc via mmiklavc) closes
> > apache/metron#894
> > > >> 4 months ago METRON-1380: Create a typosquatting use-case closes
> > > >> apache/incubator-metron#882
> > > >> 4 months ago METRON-1230: As a stopgap prior to METRON-777, add more
> > > >> simplistic sideloading of custom Parsers closes
> > > apache/incubator-metron#785
> > > >> 4 months ago METRON-1378: Create a summarizer closes
> > > >> apache/incubator-metron#879
> > > >> 4 months ago METRON-1231 Separate Sensor name and topic in the
> > > Management
> > > >> UI (merrimanr) closes apache/metron#786
> > > >> 4 months ago METRON-1382 Run Stellar in a Zeppelin Notebook
> > (nickwallen)
> > > >> closes apache/metron#884
> > > >> 4 months ago METRON-1396 Fix .gitignore files to not ignore
> themselves
> > > >> (justinleet) closes apache/metron#896
> > > >> 4 months ago METRON-1366: Add an entropy stellar function (cstella
> via
> > > >> mmiklavc) closes apache/metron#872
> > > >> 4 months ago METRON-1390: Swagger UI for "Web Security Config"
> > > Controller
> > > >> needs request method (MohanDV via mmiklavc) closes apache/metron#889
> > > >> 4 months ago METRON-1393: Fix bro Elasticsearch template (mmiklavc
> via
> > > >> mmiklavc) closes apache/metron#893
> > > >> 4 months ago METRON-1379: Add an OBJECT_GET stellar function closes
> > > >> apache/incubator-metron#880
> > > >> 4 months ago METRON-939: Upgrade ElasticSearch and Kibana (mmiklavc
> > via
> > > >> mmiklavc) closes apache/metron#840
> > > >> 4 months ago METRON-1377: Stellar function to generate typosquatted
> > > >> domains (similar to dnstwist) closes apache/incubator-metron#878
> > > >> 4 months ago METRON-1385 Missing &quot;properties&quot; in
index
> > > template
> > > >> causes ElasticsearchColumnMetadataDao.getColumnMetadata to fail
> > > >> (merrimanr) closes apache/metron#886
> > > >> 4 months ago METRON-1388 update public web site to point at 0.4.2
> new
> > > >> release (mattf-horton) closes apache/metron#887
> > > >> 4 months ago METRON-1362 Improve Metron Deployment README
> (nickwallen)
> > > >> closes apache/metron#869
> > > >> 4 months ago METRON-1384 Increment master version number to 0.4.3
> for
> > > >> on-going development (mattf-horton via nickwallen) closes
> > > apache/metron#885
> > > >> 4 months ago METRON-1381 Add Apache license to MD files and remove
> the
> > > >> Rat exclusion (justinleet) closes apache/metron#883
> > > >> 4 months ago METRON-1071 Create CONTRIBUTING.md (justinleet) closes
> > > >> apache/metron#881
> > > >> 4 months ago METRON-1373 RAT failure for
> > metron-interface/metron-alerts
> > > >> (mattf-horton) closes apache/metron#875
> > > >> 4 months ago METRON-1351 Create Installable Packages for Ubuntu
> Trusty
> > > >> (nickwallen) closes apache/metron#868
> > > >> 5 months ago METRON-1376 RC Check Script should have named
> parameters
> > > >> (ottobackwards via nickwallen) closes apache/metron#877
> > > >> 5 months ago METRON-1365: Allow PROFILE_GET to return a default
> value
> > > for
> > > >> a profile and entity that does not have a value written. closes
> > > >> apache/incubator-metron#871
> > > >> 5 months ago METRON-1348 Metron Service Checks Use Wrong Hostname
> > > >> (nickwallen) closes apache/metron#864
> > > >> 5 months ago METRON-1350: Add reservoir sampling functions to
> Stellar
> > > >> closes apache/incubator-metron#867
> > > >> 5 months ago METRON-1374 Script the RC checking process
> > (ottobackwards)
> > > >> closes apache/metron#876
> > > >> 5 months ago METRON-1372 Validate JIRA for Releases (nickwallen)
> > closes
> > > >> apache/metron#874
> > > >> 5 months ago METRON-1345: Update EC2 README for custom Ansible
> > (mmiklavc
> > > >> via mmiklavc) closes apache/metron#859
> > > >> 5 months ago METRON-1349 Full Dev Builds Metron Twice (nickwallen)
> > > closes
> > > >> apache/metron#866
> > > >> 5 months ago METRON-1343 Swagger UI for User Controller needs
> request
> > > >> method (MohanDV via ottobackwards) closes apache/metron#862
> > > >> 5 months ago METRON-1306: When index template install fails, we
> should
> > > >> fail the install closes apache/incubator-metron#834
> > > >> 5 months ago METRON-1341 Projection FieldTransformation
> > > >> (simonellistonball via ottobackwards) closes apache/metron#861
> > > >>
> > > >>
> > > >> On Wed, May 9, 2018 at 11:57 AM, Michael Miklavcic <
> > > >> michael.miklavcic@gmail.com> wrote:
> > > >>
> > > >>> Is this what you mean Otto?
> > > >>> https://github.com/apache/metron/blob/24822dddc68c264f59723f
> > > >>>
> > >
> 5e17d423cd497f6807/dev-utilities/release-utils/validate-jira-for-release
> > > >>>
> > > >>> On Wed, May 9, 2018 at 9:52 AM, Casey Stella <cestella@gmail.com>
> > > wrote:
> > > >>>
> > > >>> > I wasn't aware we had a script for that..is that in
> > > >>> > dev-utilities/release-utils?
> > > >>> >
> > > >>> > On Wed, May 9, 2018 at 11:41 AM Otto Fowler <
> > ottobackwards@gmail.com
> > > >
> > > >>> > wrote:
> > > >>> >
> > > >>> > > Can you run the issues included script and post that
for us to
> > see?
> > > >>> > >
> > > >>> > >
> > > >>> > > On May 9, 2018 at 11:14:11, Casey Stella (cestella@gmail.com)
> > > wrote:
> > > >>> > >
> > > >>> > > Is it about time for a release? I know we got some substantial
> > > >>> > performance
> > > >>> > > changes in since the last release. I think we might
have a
> > > >>> justification
> > > >>> > > for a release.
> > > >>> > >
> > > >>> > > Casey
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> > --
> >
> > Jon
> >
>
-- 

Jon

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message