spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandeep Singh <>
Subject Crawler and Scraper with different priorities
Date Sun, 07 Sep 2014 07:15:30 GMT
Hi all,

I am Implementing a Crawler, Scraper. The It should be able to process the
request for crawling & scraping, within few seconds of submitting the
job(around 1mil/sec), for rest I can take some time(scheduled evenly all
over the day). What is the best way to implement this?


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message