spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 孫澤恩 <>
Subject How to read LZO file in Spark?
Date Wed, 27 Sep 2017 10:36:02 GMT
Hi All,

Currently, I follow this blog
that I could use hdfs dfs -text to read the LZO file.
But I want to know how to use Spark to read lzo file?
I put the hadoop-lzo.jar to spark/jars and follow the blog

Here are my script
sc.newAPIHadoopFile(“hfs://<my_path_to_file>", classOf[com.hadoop.mapreduce.LzoTextInputFormat],classOf[],classOf[])
val lzoRDD =

The result of it is null.

Does anyone has some experience of this?

Sean Sun

View raw message