hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13079) Add -q to fs -ls to print non-printable characters
Date Mon, 02 May 2016 17:07:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15266989#comment-15266989

Colin Patrick McCabe commented on HADOOP-13079:

bq. No way should -q be the default under any circumstances. That is extremely surprising
behavior that will definitely break stuff.

It's not surprising, because it matches the traditional UNIX / Linux behavior.  In Linux,
{{/bin/ls}} will not print control characters by default.  you must pass the {{--show-control-characters}}
option in order to see them.  From the man page:

              show non graphic characters as-is (default unless program is 'ls' and output
is a terminal)

{{ls}} blasting raw control characters into an interactive terminal is a very bad idea.  It
leads to some very serious security vulnerabilities because commonly used software like {{xterm}},
{{GNU screen}}, {{tmux}} and so forth interpret control characters.  Using control characters,
you can convince these pieces of software to execute arbitrary code.  See http://marc.info/?l=bugtraq&m=104612710031920&q=p3
and https://www.proteansec.com/linux/blast-past-executing-code-terminal-emulators-via-escape-sequences/
 There are even CVEs for some of these issues.

We should make the default opt-in for printing control characters in our next compatibility-breaking
release (Hadoop 3.x).

bq. In C, isatty(STDOUT_FILENO) is used to find out whether the output is a terminal. Since
Java doesn't have isatty, I will use JNI to call C isatty() because the closest test System.console()
== null does not work in some cases.

It would really be nice if we could determine this without using JNI, because it's often not
available.  Under what conditions does the {{System.console() == null}} check not work?  The
only case I was able to find in a quick Google search was inside an eclipse console.  That
seems like a case where the security issues would not be a concern, because it's a debugging
environment.  Are there other cases where the non-JNI check would fail?

> Add -q to fs -ls to print non-printable characters
> --------------------------------------------------
>                 Key: HADOOP-13079
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13079
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: John Zhuge
>            Assignee: John Zhuge
>              Labels: supportability
> Add option {{-q}} to "hdfs dfs -ls" to print non-printable characters as "?". Non-printable
characters are defined by [isprint(3)|http://linux.die.net/man/3/isprint] according to the
current locale.
> Default to {{-q}} behavior on terminal; otherwise, print raw characters. See the difference
in these 2 command lines:
> * {{hadoop fs -ls /dir}}
> * {{hadoop fs -ls /dir | od -c}}
> In C, {{isatty(STDOUT_FILENO)}} is used to find out whether the output is a terminal.
Since Java doesn't have {{isatty}}, I will use JNI to call C {{isatty()}} because the closest
test {{System.console() == null}} does not work in some cases.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message