Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
Thanks everyone. I was able to resolve this.Here is what I did. Just passed conf file using —files option.Mistake that I did was reading the json conf file before creating spark session . Reading if after creating spark session helped it. Thanks once again for your valuable suggestionsThanks,AsmathOn May 15, 2021, at 8:12 AM, Sean Owen <firstname.lastname@example.org> wrote:If code running on the executors need some local file like a config file, then it does have to be passed this way. That much is normal.On Sat, May 15, 2021 at 1:41 AM Gourav Sengupta <email@example.com> wrote:Hi,once again lets start with the requirement. Why are you trying to pass xml and json files to SPARK instead of reading them in SPARK?Generally when people pass on files they are python or jar files.Regards,Gourav