spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anil Langote <anillangote0...@gmail.com>
Subject Efficient look up in Key Pair RDD
Date Mon, 09 Jan 2017 01:03:54 GMT
Hi All,

I have a requirement where I wanted to build a distributed HashMap which
holds 10M key value pairs and provides very efficient lookups for each key.
I tried loading the file into JavaPairedRDD and tried calling lookup method
its very slow.

How can I achieve very very faster lookup by a given key?

Thank you
Anil Langote

Mime
View raw message