flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrea Ferranti <andreaferra...@hotmail.it>
Subject Re: Information on Flink
Date Fri, 14 Nov 2014 08:54:24 GMT
More or less as you described.

I exploit jMetal to build the problem and the algorithm is it really possible to integrate
Flink in jMetal?

Best, 
Andrea

> Il giorno 13/nov/2014, alle ore 21:44, <fhueske@gmail.com> <fhueske@gmail.com>
ha scritto:
> 
> Hi Andrea,
> 
> Please correct me if I got something wrong.
> You have a population of several classifiers that you want to evolve and improve.
> In each iteration, you test each classifier with all your test data and evaluate the
fitness of each classifier. Then, you create a new population of classifiers by removing bad
classifiers and mutating (and crossing) the better ones.
> 
> I think you can do that with Flink as follows:
> you use a Map over the test data and a broadcast set as the classifiers to check the
classification of a single attribute. With a following reduce, you aggregate the fitness of
each classifier and use a reduce all to build a new population of classifiers.
> In the next iteration, the new population is broadcasted to the Map over the test data.
> 
> This is quite similar to what our KMeans example does. You should have a look at it.
> 
> Best, Fabian
> 
> From: Kostas Tzoumas <mailto:ktzoumas@apache.org>
> Sent: ‎Thursday‎, ‎13‎. ‎November‎, ‎2014 ‎17‎:‎11
> To: user@flink.incubator.apache.org <mailto:user@flink.incubator.apache.org>
> Cc: Andrea Ferranti <mailto:andreaferranti@hotmail.it>
> 
> I am forwarding this here in case someone with better knowledge of genetic algorithms
picks it up.
> 
> Kostas
> 
> ---------- Forwarded message ----------
> From: Andrea Ferranti <andreaferranti@hotmail.it <mailto:andreaferranti@hotmail.it>>
> Date: Thu, Nov 13, 2014 at 4:46 PM
> Subject: Re: Information on Flink
> To: Kostas Tzoumas <ktzoumas@apache.org <mailto:ktzoumas@apache.org>>
> 
> 
> Thanks very much for your reply.
> 
> First, can I forward this to the Flink user mailing list? Perhaps someone over there
has a better answer.
> 
> Yes, of course.
> 
> Can you describe very briefly how fitness evaluation is computed in your algorithm?
> 
> My fitness evaluation is basically an evaluation of accuracy in a classification problem,
so i must read every line of file(in which is present some attribute and a class) and verify
if my classification work well.
> So in each iteration of a genetic algorithm i change some chromosome and than evaluate
the solution.
> 
> At the moment the entire program is written in C++ but I would take it in java using
jMetal
> 
> Best regards, 
> Andrea
> 
> Il giorno 13/nov/2014, alle ore 16:25, Kostas Tzoumas <ktzoumas@apache.org <mailto:ktzoumas@apache.org>>
ha scritto:
> 
> Hey,
> 
> First, can I forward this to the Flink user mailing list? Perhaps someone over there
has a better answer.
> 
> Can you describe very briefly how fitness evaluation is computed in your algorithm?
> 
> Kostas
> 
> On Thu, Nov 13, 2014 at 4:08 PM, Andrea Ferranti <andreaferranti@hotmail.it <mailto:andreaferranti@hotmail.it>>
wrote:
> Dear Kostas Tzoumas,
> I'm Andrea Ferranti, a student of Computer Engineering at the University of Pisa.
> In my thesis I would like to exploit Flink to parallelize a Evolutionary algorithm, in
particular the fitness evaluation. My problem and algorithm are written in Java (jMetal).
> 
> 
> Do you think that flink can be a good tool for the parallelization of fitness? In my
problem the fitness is evaluate on very big datasets.
> 
> Best regards, 
> Andrea


Mime
View raw message