Pls checke the specific ERORR lines of the text file .
Chaces are are few Columns are not properly delimited in specific rows.

Regards
Prakash

On Fri, Sep 7, 2018, 3:41 AM dimitris plakas <dimitrispl22@gmail.com> wrote:
Hello everyone, I am new in Pyspark and i am facing an issue. Let me explain what exactly is the problem.

I have a dataframe and i apply on this a map() function (dataframe2=datframe1.rdd.map(custom_function())
dataframe = sqlContext.createDataframe(dataframe2)

when i haveĀ 

dataframe.show(30,True) it shows the result,

when i am using dataframe.show(60, True) i get the error. The Error is in the attachement Pyspark_Error.txt.

Could you please explain me what is this error and how to overpass it?


---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org