You may use the wrong mailing list for your question (=> spark user).
If you want to make a single string, it's :
Be careful of driver explosion !
One additional point, the following line:when set to a scala string prints nothing even when I use toString at the end. This seems to not be something that should be that out of the ordinary but I could be wrong.
I'm needing to read a json file from hdfs and turn that into a scala string, I have dug around for documentation on how to do this and found this:
The following two lines of code dont seem to do the job:
rdd = sc.wholeTextFiles("hdfs://nameservice1/user/me/test.txt") rdd.collect.foreach(t=>println(t._2))
I have tried to set the second line to a scala string but that doesn't seem to work, I would really appreciate some insights into how to do this.
Thanks in advance.