hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12563) Updated utility to create/modify token files
Date Thu, 31 Mar 2016 14:29:25 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219930#comment-15219930
] 

Steve Loughran commented on HADOOP-12563:
-----------------------------------------

Coming along nicely.

Someone other than me will have to look at marshalling changes.

# If a fetch fails, throw an exception rather than just return null.
# can you add to interface {{DtFetcher}} isTokenRequired(): boolean. that way, its up to the
FS, Hbase etc to decide whether this instance needs a token.
# Can you do the fetchers for the RM, ATS and KMS, to make sure they fit in with the design
too.
# CLI needs a way to specify keytab and principal to log in as. Currently it assumes caller
is kinited in.
# if you make the subcommands take an output stream, then in tests you pass in something that
writes to a stringbuffer; check the contents of that to validate. Only in main() calls do
you need to pass in System.out. I can see that you play with System.out in tests, but that
prevents the code being used in different places.
# I'd like the get subcommand (and ideally the others) to be easy to call from other code,
especially YARN apps. The inner load/get/remove/renew logic should all be isolated into something
anything can call, with command arg parsing taking place outside it.

minor
* use SLF4J and you can skip the String.format() clauses in logging
* Unsure about using {{Date.toString()}} to represent dates, especially in a CLI which may
be expected to be stable. It's deprecated for a reason.

> Updated utility to create/modify token files
> --------------------------------------------
>
>                 Key: HADOOP-12563
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12563
>             Project: Hadoop Common
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Allen Wittenauer
>            Assignee: Matthew Paduano
>         Attachments: HADOOP-12563.01.patch, HADOOP-12563.02.patch, HADOOP-12563.03.patch,
HADOOP-12563.04.patch, HADOOP-12563.05.patch, HADOOP-12563.06.patch, HADOOP-12563.07.patch,
HADOOP-12563.07.patch, dtutil-test-out, example_dtutil_commands_and_output.txt, generalized_token_case.pdf
>
>
> hdfs fetchdt is missing some critical features and is geared almost exclusively towards
HDFS operations.  Additionally, the token files that are created use Java serializations which
are hard/impossible to deal with in other languages. It should be replaced with a better utility
in common that can read/write protobuf-based token files, has enough flexibility to be used
with other services, and offers key functionality such as append and rename. The old version
file format should still be supported for backward compatibility, but will be effectively
deprecated.
> A follow-on JIRA will deprecrate fetchdt.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message