hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13079) Add -q to fs -ls to print non-printable characters
Date Tue, 03 May 2016 20:31:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269527#comment-15269527

Colin Patrick McCabe commented on HADOOP-13079:

bq. Yup, I can't think of why -q should be the default either... but more importantly, neither
could POSIX to the point that it demanded the standard have -q be the default.

Please do not misquote what I said.  I was arguing that echoing control characters to the
terminal should not be the default behavior.  You are arguing the opposite.

bq. ... until such a point that they print the filename to the screen to show what files are
being processed. At which point this change has accomplished absolutely nothing. Changing
ls is security theater.

There are a lot of scripts that interact with HDFS via FsShell.  These scripts will never
"print the filename to the screen" or if they do, it will be a filename that they got from
{{ls}} itself which does not contain control characters.

I could come up with examples of how this is helpful all day if needed.  Here's another one:
Some sysadmin logs in and does an {{hadoop fs -ls}} of a directory created by {{\$BADGUY}}.
Should the filename be able use control characters to hijack the admin's GNU screen session
and execute arbitrary code?  I would say no, what do you say?

bq. Are we going to change cat too?

Most system administrators will not {{cat}} a file without checking what type it is.  It is
well-known that catting an unknown file could mess up the terminal.  On the other hand, most
system administrators do not think that running {{ls}} on a directory could be a security
risk.  Linux and other well known operating systems also do not protect users from this, so
there are no pre-existing expectations of protection.

bq. Then stop bringing up (traditional) UNIX if you feel it isn't relevant and especially
when you've used the term incorrectly.

There are a huge number of sysadmins who grew up with the GNU tools, which do have the behavior
we're describing here.  It's a powerful argument for implementing that behavior.  When you
add the fact that it fixes security vulnerabilities, it's an extremely compelling argument.

I think it's clear that this change does have a big positive effect in many scenarios, does
fix real-world security flaws, and does accord with the expectations of most system administrators.
 That's three powerful reasons to do it.  I can find no valid counter-argument for any of
these reasons anywhere in these comments.

> Add -q to fs -ls to print non-printable characters
> --------------------------------------------------
>                 Key: HADOOP-13079
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13079
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: John Zhuge
>            Assignee: John Zhuge
>              Labels: supportability
> Add option {{-q}} to "hdfs dfs -ls" to print non-printable characters as "?". Non-printable
characters are defined by [isprint(3)|http://linux.die.net/man/3/isprint] according to the
current locale.
> Default to {{-q}} behavior on terminal; otherwise, print raw characters. See the difference
in these 2 command lines:
> * {{hadoop fs -ls /dir}}
> * {{hadoop fs -ls /dir | od -c}}
> In C, {{isatty(STDOUT_FILENO)}} is used to find out whether the output is a terminal.
Since Java doesn't have {{isatty}}, I will use JNI to call C {{isatty()}} because the closest
test {{System.console() == null}} does not work in some cases.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message