Hi Sea,

For using Spark SQL you will need to create DataFrame from the file and then execute select * on dataframe.
In your case you will need to do something like this

        JavaRDD<String> DF = context.textFile("path");
        JavaRDD<Row> rowRDD3 = DF.map(new Function<String, Row>() {
            public Row call(String record) throws Exception {
                String[] fields = record.split("\001");
                Row createRow = createRow(fields);
                return createRow;
            }
        });
        DataFrame ResultDf3 = hiveContext.createDataFrame(rowRDD3, schema);
        ResultDf3.registerTempTable("test")
        hiveContext.sql("select * from test");

You will need to create schema for the file first just like how you have created for csv file.



 
On Friday 23 September 2016 12:26 PM, Sea wrote:
Hi, I want to run sql directly on files, I find that spark has supported sql like select * from csv.`/path/to/file`, but files may not be split by ','. Maybe it is split by '\001', how can I specify delimiter?

Thank you!