spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zzzzzqf12345 <>
Subject Re: streaming on hdfs can detected all new file, but the sum of all the rdd.count() not equals which had detected
Date Tue, 13 May 2014 05:15:14 GMT
thanks for reply~~

I had solved the problem and found the reason, because I used the Master
node to upload files to hdfs, this action may take up a lot of Master's
network resources. When I changed to use another computer none of the
cluster to upload these files, it got the correct result.


View this message in context:
Sent from the Apache Spark User List mailing list archive at

View raw message