flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From andralungu <...@git.apache.org>
Subject [GitHub] flink pull request: [FLINK-1520]Read edges and vertices from CSV f...
Date Mon, 13 Jul 2015 08:56:33 GMT
Github user andralungu commented on the pull request:

    I just had a closer look at this PR and it made me seriously question the utility of a
`Graph.fromCSV` method. Why? First of all because it's more limited than the regular `env.fromCsv()`
in the sense that it does not allow POJOs and it would be a bit tedious to support that. There
would be a need for methods with 2 to n fields, according to the amount of attributes present
in the POJO. 
    Second, because, and I am speaking strictly as a user here, I would rather write:
    private static DataSet<Edge<Long, Double>> getEdgesDataSet(ExecutionEnvironment
env) {
    		if(fileOutput) {
    			return env.readCsvFile(edgeInputPath)
    					.types(Long.class, Long.class, Double.class)
    					.map(new Tuple3ToEdgeMap<Long, Double>());
    		} else {
    			return CommunityDetectionData.getDefaultEdgeDataSet(env);
    private static Graph<Long, Long, Double> getGraph(ExecutionEnvironment env) {
    		Graph<Long, Long, Double> graph;
    		if(!fileOutput) {
    			DataSet<Edge<Long, Double>> edges = CommunityDetectionData.getDefaultEdgeDataSet(env);
    			graph = Graph.fromDataSet(edges,
    					new MapFunction<Long, Long>() {
    						public Long map(Long label) {
    							return label;
    					}, env);
    		} else {
    			graph = Graph.fromCsvReader(edgeInputPath,new MapFunction<Long, Long>() {
    				public Long map(Long label) {
    					return label;
    			}, env).ignoreCommentsEdges("#")
    					.typesEdges(Long.class, Double.class)
    					.typesVertices(Long.class, Long.class);
    		return graph;
    Maybe it's just a preference thing... but I believe it's at least worth a discussion.
On the other hand, the utility of such a method should have been questioned from its early
Jira days, so I guess that's my mistake.
    I would like to hear your thoughts on this. 

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message