flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Flink and factories?
Date Wed, 19 Oct 2016 21:48:34 GMT
The functions are serialized when env.execute() is being executed. The 
thing is, as i understand it, that your singleton is simply not part of 
the serialized function, so it doesn't actually matter when the function 
is serialized.

Storing the factory instance in the function shouldn't be too much work 
actually, the following code might do the trick already (changes in bold):

    DataSet<IData> processedData = this.getEnv().fromCollection(inputData).flatMap(new
FlatMapFunction<Integer, IData>() {
    	*private final <FactoryType> factory =
    Config.getInstance().getFactory(); *     
    	@Override
    	public void flatMap(Integer integer, Collector<IData> collector) throws Exception
{
      		if (integer % 2 == 0) {
    			collector.collect(*factory.newInstance()*);
                     }
             }
    });

Regards,
Chesnay

On 19.10.2016 23:09, Sebastian Neef wrote:
> Hi Chesnay,
>
> thank you for looking into this!
>
> Is there any way to tell Flink to (re)sync the changed classes and/or
> tell it to distribute the serialized classes at a given point (e.g.
> first on a env.execute() ) or so?
>
> The thing is, that I'm working on a small framework which bases on
> flink, so passing a configuration object to all functions/classes would
> be overkill, I guess.
>
> Thanks again and kind regards,
> Sebastian
>


Mime
View raw message