spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From KhajaAsmath Mohammed <>
Subject Re: [EXTERNAL] Urgent Help - Py Spark submit error
Date Sat, 15 May 2021 17:31:44 GMT
Thanks everyone. I was able to resolve this. 

Here is what I did. Just passed conf file using —files option.

Mistake that I did was reading the json conf file before creating spark session . Reading
if after creating spark session helped it. Thanks once again for your valuable suggestions


> On May 15, 2021, at 8:12 AM, Sean Owen <> wrote:
> If code running on the executors need some local file like a config file, then it does
have to be passed this way. That much is normal.
>> On Sat, May 15, 2021 at 1:41 AM Gourav Sengupta <>
>> Hi,
>> once again lets start with the requirement. Why are you trying to pass xml and json
files to SPARK instead of reading them in SPARK? 
>> Generally when people pass on files they are python or jar files.
>> Regards,
>> Gourav

View raw message