spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sonal Goyal <sonalgoy...@gmail.com>
Subject Re: How to disable input split
Date Sat, 18 Oct 2014 10:16:17 GMT
I am not sure if this works, but SparkContext seems to have hadoopFile and
hadoopRDD methods which can accept Hadoop Input formats. If you have an
InputFormat with isSplittable false, maybe it works?



Best Regards,
Sonal
Nube Technologies <http://www.nubetech.co>

<http://in.linkedin.com/in/sonalgoyal>



On Sat, Oct 18, 2014 at 11:29 AM, Davies Liu <davies@databricks.com> wrote:

> You can call coalesce() to merge the small splits into bigger ones.
>
> Davies
>
> On Fri, Oct 17, 2014 at 5:35 PM, Larry Liu <larryliu05@gmail.com> wrote:
> > Is it possible to disable input split if input is already small?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message