spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From qinwei <>
Subject Re: IP to geo information in spark streaming application
Date Wed, 03 Dec 2014 02:00:59 GMT

1) I think using library based solution is a better idea, we used that, and it works.2) We
used broadcast variable, and it works

 From: Noam KfirDate: 2014-12-02 23:14To: user@spark.apache.orgSubject: IP to geo information
in spark streaming application


I'm new to spark streaming.

I'm currently writing spark streaming application to standardize events coming from Kinesis.

As part of the logic, I want to use IP to geo information library or service.

My questions:
1) If I would use some REST service for this task, do U think it would cause performance
penalty (over using library based solution)

2) If I would use a library based solution, I will have to use some local db file.

What mechanism should I use in order to transfer such db file? a broadcast variable?

​Tx, Noam.

View raw message