hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6408) Add a /conf servlet to dump running configuration
Date Mon, 11 Jan 2010 05:19:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798576#action_12798576
] 

Hemanth Yamijala commented on HADOOP-6408:
------------------------------------------

I apologize for not coming in earlier on this. But I have a concern with one particular point
that was implemented in this patch:

bq. Configuration now always tracks the "setting resource".
and
bq. Also, the number of Configuration objects floating around a daemon is reasonably small
compared to its overall RAM usage, so I don't think the memory overhead will be a problem.

In Map/Reduce, please note that every JobInProgress holds a reference to a JobConf. Since
each JobConf will cause a load of the default resources, a duplicate hash map that manages
the mapping between key and the setting resource will now be held for every Job held in memory.
I believe this will be a significant memory overload particularly if the job load is very
high. It was for this reason that we had chosen the approach we took in HADOOP-6184 to enable
this map only on demand.

Does this make sense ?

> Add a /conf servlet to dump running configuration
> -------------------------------------------------
>
>                 Key: HADOOP-6408
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6408
>             Project: Hadoop Common
>          Issue Type: New Feature
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0
>
>         Attachments: hadoop-6408.txt, hadoop-6408.txt, hadoop-6408.txt, hadoop-6408.txt,
hadoop-6408.txt, hadoop-6408.txt, hadoop-6408.txt, hadoop-6408.txt
>
>
> HADOOP-6184 added a command line flag to dump the running configuration. It would be
great for cluster troubleshooting to provide access to this as a servlet, preferably in both
JSON and XML formats. But really, any format would be better than nothing. This should/could
go into all of the daemons.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message