hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15204) Add Configuration API for parsing storage sizes
Date Thu, 01 Feb 2018 23:32:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349492#comment-16349492

Chris Douglas commented on HADOOP-15204:

bq. As the author of HADOOP-8608, I would appreciate any perspectives you have on this JIRA.
I find the API intuitive, but that is not universal (e.g., HDFS-9847). Explaining it has taken
more cycles than I expected, and perhaps more than a good API should.
* {{TERRABYTES}} is misspelled.
* Is {{long}} insufficient as a return type for {{getStorageSize}}? I appreciate future-proofing,
but for {{Configuration}} values, that's what, ~8 petabytes? I haven't looked carefully at
the semantics of {{BigDecimal}}, but the comments imply that {{setScale}} is used to guarantee
the result will fit. An overload of {{setStorageSize}} taking {{double}} might make sense,
as would using doubles in the overload of {{String}}.
* Why [ROUND_UP|https://docs.oracle.com/javase/8/docs/api/java/math/BigDecimal.html#ROUND_UP]
of the options? Just curious.
* {{TimeUnit}} uses min/max values for Long (e.g., [TimeUnit::toNanos|https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/TimeUnit.html#toNanos-long-])
for overflow/underflow. Storage units are more likely to be exact powers of two so that may
not be appropriate.

> Add Configuration API for parsing storage sizes
> -----------------------------------------------
>                 Key: HADOOP-15204
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15204
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: conf
>    Affects Versions: 3.1.0
>            Reporter: Anu Engineer
>            Assignee: Anu Engineer
>            Priority: Minor
>             Fix For: 3.1.0
>         Attachments: HADOOP-15204.001.patch
> Hadoop has a lot of configurations that specify memory and disk size. This JIRA proposes
to add an API like {{Configuration.getStorageSize}} which will allow users
>  to specify units like KB, MB, GB etc. This is JIRA is inspired by HADOOP-8608 and Ozone.
Adding {{getTimeDuration}} support was a great improvement for ozone code base, this JIRA
hopes to do the same thing for configs that deal with disk and memory usage.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message