ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [ambari] payert opened a new pull request #3254: AMBARI-25569 Reassess Ambari Metrics data migration - 2nd part
Date Tue, 03 Nov 2020 14:15:23 GMT

payert opened a new pull request #3254:
URL: https://github.com/apache/ambari/pull/3254


   ## What changes were proposed in this pull request?
   
   **1. Introduce an '--allmetrics' to enforce migration of all metrics regardless of other
settings.**
   Due to the suboptimal argument handling, if one wants to define an argument that comes
after the 'whitelist file'
   argument - like the 'starttime' - the 'whitelist file' argument must be defined.
   But when we don't want to use the whitelist data because we need to migrate all the metrics
the '--allmetrics' argument can be provided instead of 'whitelist file'.
   
   Example: migrate all the metrics from the last year
   `/usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf/ upgrade_start
--allmetrics "365"`
   
   **2. The start time handling has been fixed and changed**
    - The code is intended to migrate data from the "last x milliseconds" as the handling
of the default data shows where the startTime is subtracted from the current timestamp.
   `public static final long DEFAULT_START_TIME = System.currentTimeMillis() - ONE_MONTH_MILLIS;
//Last month`
   But when the user externally provided the `startTime` value it vas not subtracted from
the current timestamp, but was used as it is, which is indeed erroneous.
   - Also, I suggest using days instead of milliseconds to define the required migration time
window, because it is a more realistic and convenient granularity. Like in the above example
the command will migrate data from the last 365 days.
   - Time handling has been changed to `LocalDateTime` use in Java 8 fashion.
   
   3. **Possibly improve performance** by reducing synchronisation granularity using and BufferedWriter
when writing the "ambari-metrics-migration-state"
   
   ## How was this patch tested?
   - It was tested by manually migrating large data sets derived from real world data.
   - Running the existing unit tests.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message