flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From fhueske <...@git.apache.org>
Subject [GitHub] flink pull request: [FLINK-1396][FLINK-1303] Hadoop Input/Output d...
Date Wed, 04 Feb 2015 20:50:57 GMT
Github user fhueske commented on a diff in the pull request:

    https://github.com/apache/flink/pull/363#discussion_r24118164
  
    --- Diff: flink-java/src/main/java/org/apache/flink/api/java/ExecutionEnvironment.java
---
    @@ -458,6 +461,67 @@ public CsvReader readCsvFile(String filePath) {
     		
     		return new DataSource<X>(this, inputFormat, producedType, Utils.getCallLocationName());
     	}
    +
    +	// ----------------------------------- Hadoop Input Format ---------------------------------------
    +
    +	/**
    +	 * Creates a {@link DataSet} from the given {@link org.apache.hadoop.mapred.FileInputFormat}.
The
    +	 * given inputName is set on the given job.
    +	 */
    +	public <K,V> DataSource<Tuple2<K, V>> readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K,V>
mapredInputFormat, Class<K> key, Class<V> value, String inputPath, JobConf job)
{
    +		DataSource<Tuple2<K, V>> result = createHadoopInput(mapredInputFormat,
key, value, job);
    +
    +		org.apache.hadoop.mapred.FileInputFormat.addInputPath(job, new org.apache.hadoop.fs.Path(inputPath));
    +
    +		return result;
    +	}
    +
    +	/**
    +	 * Creates a {@link DataSet} from the given {@link org.apache.hadoop.mapred.FileInputFormat}.
A
    +	 * {@link org.apache.hadoop.mapred.JobConf} with the given inputPath is created.
    +	 */
    +	public <K,V> DataSource<Tuple2<K, V>> readHadoopFile(org.apache.hadoop.mapred.FileInputFormat<K,V>
mapredInputFormat, Class<K> key, Class<V> value, String inputPath) {
    +		return readHadoopFile(mapredInputFormat, key, value, inputPath, new JobConf());
    +	}
    +
    +	/**
    +	 * Creates a {@link DataSet} from the given {@link org.apache.hadoop.mapred.InputFormat}.
    +	 */
    +	public <K,V> DataSource<Tuple2<K, V>> createHadoopInput(org.apache.hadoop.mapred.InputFormat<K,V>
mapredInputFormat, Class<K> key, Class<V> value, JobConf job) {
    --- End diff --
    
    Why no "shortcut" without `JobConf`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message