hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Íñigo Goiri (JIRA) <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15808) Harden Token service loader use
Date Tue, 30 Oct 2018 16:52:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669031#comment-16669031

Íñigo Goiri commented on HADOOP-15808:

I have to say that the pattern of iterating and then catching exceptions is much uglier than
just using the {{for(:)}} approach.
I've been trying to see if there's any way to catch those in the iterator but there's nothing
really clean.
I guess we'll have to live with this.
I don't fully understand why the {{Iterator#next()}} triggers the exception but service loading
has lazy behaviors so I guess that must be it.

Regarding the exception output, given that right now, it just crashes the whole thing, I think
that going to debug (which is pretty much swallowing it), might be a little too much.
I would log errors for those cases.

In any case, I think we can add some coverage to these cases, not sure what's the cleanest
way to trigger {{ServiceConfigurationError}}; a few options I can think:
* Create a fake type that triggers this by not having the dependencies.
* Spy some of the types to trigger the exception always.

> Harden Token service loader use
> -------------------------------
>                 Key: HADOOP-15808
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15808
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: security
>    Affects Versions: 2.9.1, 3.1.2
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Major
>         Attachments: HADOOP-15808-001.patch, HADOOP-15808-002.patch, HADOOP-15808-003.patch
> The Hadoop token service loading (identifiers, renewers...) works provided there's no
problems loading any registered implementation. If there's a classloading or classcasting
problem, the exception raised will stop all token support working; possibly the application
not starting.
> This matters for S3A/HADOOP-14556 as things may not load if aws-sdk isn't on the classpath.
It probably lurks in the wasb/abfs support too, but things have worked there because the installations
with DT support there have always had correctly set up classpaths.
> Fix: do what we did for the FS service loader. Catch failures to instantiate a service
provider impl and skip it

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message