spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: Java Streaming Context - File Stream use
Date Tue, 11 Aug 2015 09:08:43 GMT
Like this: (Including the filter function)

JavaPairInputDStream<LongWritable, Text> inputStream = ssc.fileStream(
      testDir.toString(),
      LongWritable.class,
      Text.class,
      TextInputFormat.class,
      new Function<Path, Boolean>() {
        @Override
        public Boolean call(Path v1) throws Exception {
          return Boolean.TRUE;
        }
      },
      true);

Thanks
Best Regards

On Tue, Aug 11, 2015 at 1:10 AM, Ashish Soni <asoni.learn@gmail.com> wrote:

> Please help as not sure what is incorrect with below code as it gives me
> complilaton error in eclipse
>
>              SparkConf sparkConf = new
> SparkConf().setMaster("local[4]").setAppName("JavaDirectKafkaWordCount");
>             JavaStreamingContext jssc = new
> JavaStreamingContext(sparkConf, Durations.seconds(2));
>
> *            jssc.fileStream("/home/", String.class, String.class,
> TextInputFormat.class);*
>
>

Mime
View raw message