hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13079) dfs -ls -q prints non-printable characters
Date Thu, 05 May 2016 18:36:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272802#comment-15272802

Colin Patrick McCabe commented on HADOOP-13079:

bq. It's not a security bug for the reasons you think it's a security bug. After all, wc,
find, du, ... tons of other UNIX commands will happily print out terminal escape sequences
with no option to turn them off. It is, however, problematic for traditional ftpd implementations
since it's a great way to inject buffer overflows and then get root on a remote server.

This behavior is exploitable.  That makes it a security bug, even if lots of traditional UNIX
commands have it.

Just because a behavior is traditional doesn't mean it's right.  There was a time when UNIX
programs used {{gets()}} everywhere.  When the world became a less trusting place, they had
to be fixed not to do that.  We should understand the motivations behind historical decisions
before blindly copying them.

bq. ... and my answer is the same as it was almost a decade ago, in some HDFS JIRA somewhere,
where a related topic came up before: HDFS would be better served by having a limit on what
consists of a legal file and directory name. With an unlimited namespace, it's impossible
to test against and impossible to protect every scenario in which oddball characters show
up. What's legal in one locale may not be legal in another.

That's a very good suggestion.  I think we should tackle that for Hadoop 3.

bq. Also, are you prepared to file a CVE for every single time Hadoop prints out a directory
or file name to the screen? There are probably hundreds if not thousands of places, obvious
ones like 'fs -count' and less obvious ones like 'yarn logs'. This is a 'tilting at windmills'
problem. It is MUCH better to have ls blow up than be taken by surprise by something else
later on.

The problem is, {{ls}} isn't necessarily going to "blow up," just display something odd, or
even cause your xterm to run arbitrary code by abusing escape sequences.

> dfs -ls -q prints non-printable characters
> ------------------------------------------
>                 Key: HADOOP-13079
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13079
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>            Reporter: John Zhuge
>            Assignee: John Zhuge
> Add option {{-q}} to "hdfs dfs -ls" to print non-printable characters as "?". Non-printable
characters are defined by [isprint(3)|http://linux.die.net/man/3/isprint] according to the
current locale.
> Default to {{-q}} behavior on terminal; otherwise, print raw characters. See the difference
in these 2 command lines:
> * {{hadoop fs -ls /dir}}
> * {{hadoop fs -ls /dir | od -c}}
> In C, {{isatty(STDOUT_FILENO)}} is used to find out whether the output is a terminal.
Since Java doesn't have {{isatty}}, I will use JNI to call C {{isatty()}} because the closest
test {{System.console() == null}} does not work in some cases.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message