Get the class locally and Serialize it. http://grepcode.com/file_/repository.cloudera.com/content/repositories/releases/com.cloudera.hadoop/hadoop-core/0.20.2-737/org/apache/hadoop/io/Text.java/?v=sourcePS: Some classes may require additional classes to get serialized. Hopefully there should be some other way doing it.ThanksBest RegardsOn Fri, Sep 5, 2014 at 7:45 PM, Sarath Chandra <firstname.lastname@example.org> wrote:Hi Akhil,I've done this for the classes which are in my scope. But what to do with classes that are out of my scope?For example org.apache.hadoop.io.TextAlso I'm using several 3rd part libraries like "jeval".~SarathOn Fri, Sep 5, 2014 at 7:40 PM, Akhil Das <email@example.com> wrote:You can bring those classes out of the library and Serialize it (implements Serializable). It is not the right way of doing it though it solved few of my similar problems.ThanksBest RegardsOn Fri, Sep 5, 2014 at 7:36 PM, Sarath Chandra <firstname.lastname@example.org> wrote:Hi,
I'm trying to migrate a map-reduce program to work with spark. I migrated the program from Java to Scala. The map-reduce program basically loads a HDFS file and for each line in the file it applies several transformation functions available in various external libraries.When I execute this over spark, it is throwing me "Task not serializable" exceptions for each and every class being used from these from external libraries. I included serialization to few classes which are in my scope, but there there are several other classes which are out of my scope like org.apache.hadoop.io.Text.How to overcome these exceptions?~Sarath.