gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renato MarroquĂ­n Mogrovejo <renatoj.marroq...@gmail.com>
Subject Re: Data locality
Date Wed, 02 Oct 2013 19:00:29 GMT
Hi Julien,

Gora doesn't take into account data locality as this is taken care of by
each different data store. The one thing that Gora "take" into account is
the number of partitions it should use, and that number of partitions are
used to run more/less map tasks. This partition  number hasn't been
implemented by all data stores properly and AFAIK most of them return a
single partition, which means we only use a Map task to read as much data
as we have to.
Planning to work on this coming summer (SouthAmerican summer) ;)


Renato M.


2013/10/2 Julien Nioche <lists.digitalpebble@gmail.com>

> Hi guys,
>
> I can't quite remember whether Gora takes data locality into account when
> generating the input for a map reduce job. Could someone explain how its is
> currently handled and if things differ from one backend to the other then
> how?
>
> Thanks
>
> Julien
>
> --
> *
> *Open Source Solutions for Text Engineering
>
> http://digitalpebble.blogspot.com/
> http://www.digitalpebble.com
> http://twitter.com/digitalpebble
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message