spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saurabh Adhikary <>
Subject Dataframe replace 'collect()' going in indefinite time loop
Date Wed, 03 May 2017 20:23:43 GMT
final_schema_noise_data =

for a_name in name_field_names:
#--- till here final_schema_noise_data.collect() is working---
  for t in noise_chars:
    final_schema_noise_data =,'',a_name)
    print a_name,t
#The above loop gets completed but final_schema_noise_data.collect() dos not
yield any result, cursor goes to next line & some processing goes on for
hours but no output.

#Before the inner for loop , the df.collect() gives output in secs & post
completion of the loop no output for hours. 
*Any known issue with the function ??*

View this message in context:
Sent from the Apache Spark Developers List mailing list archive at

To unsubscribe e-mail:

View raw message