spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hxngillani <>
Subject How to load Python Pickle File in Spark Data frame
Date Mon, 26 Aug 2019 05:23:34 GMT
Hello  Dear Members 
i want to train model using Bigdl, i have data set of Medical images in the
form of pickle object files (,pck).that pickle file is a 3D image(3D array) 

i have tried 
pickleRdd = sc.pickleFilehome/student/BigDL- 
 sqlContext = SQLContext(sc)
 df = sqlContext.createDataFrame(pickleRdd)

this code throwing and error 
Caused by:
not a SequenceFile

the things i came to know is that 
The function
loads a pickle file that is created by 

I am loading a pickle file that is created by Python's "pickle" module  
My Question is that  is there any way to load that file in spark data frame 

Sent from:

To unsubscribe e-mail:

View raw message