hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6588) CompressionCodecFactory throws IllegalArgumentException
Date Fri, 05 Mar 2010 22:35:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842072#action_12842072
] 

Tsz Wo (Nicholas), SZE commented on HADOOP-6588:
------------------------------------------------

> That is because you specified LzoCodec class in your conf file. ...
It probably is the case.

> ... I think it is better to leave the behavior as-is instead of issuing log messages.

I am fine for keeping the current behavior, i.e. throwing an exception.  However, it is better
to change the exception since it is not an IllegalArgumentException.

Also, CompressionCodecFactory constructor calls getCodecClasses(..) which ultimately calls
Class.forName(..).  As a consequence, Class.forName(..) will be invoked for every TextInputFormat.isSplitable(..).
 It is unnecessary and expensive.

> CompressionCodecFactory throws IllegalArgumentException
> -------------------------------------------------------
>
>                 Key: HADOOP-6588
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6588
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: io
>            Reporter: Tsz Wo (Nicholas), SZE
>         Attachments: c6588_20100222.patch
>
>
> WordCount does not run. :(
> {noformat}
> java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec
not found.
>         at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:96)
>         at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:134)
>         at org.apache.hadoop.mapreduce.lib.input.TextInputFormat.isSplitable(TextInputFormat.java:46)
>         at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:247)
>         at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:886)
>         at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:780)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:444)
>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:459)
>         at org.apache.hadoop.examples.WordCount.main(WordCount.java:67)
>         ...
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message