gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henry Saputra (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GORA-270) IOUtils static SerializationFactory field
Date Mon, 06 Oct 2014 04:38:33 GMT

    [ https://issues.apache.org/jira/browse/GORA-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159995#comment-14159995

Henry Saputra commented on GORA-270:

[~renato2099], this is unfortunate side effect, sorry about the trouble, hopefully Giraph
community can accept your workaround for now.

As for the original patch, wouldnt call to IOUtils.getOrCreateConf will create new Configuration
if caller pass null?

> IOUtils static SerializationFactory field
> -----------------------------------------
>                 Key: GORA-270
>                 URL: https://issues.apache.org/jira/browse/GORA-270
>             Project: Apache Gora
>          Issue Type: Bug
>          Components: gora-core
>            Reporter: Damien Raude-Morvan
>            Assignee: Damien Raude-Morvan
>              Labels: mapreduce
>             Fix For: 0.4
>         Attachments: 0001-GORA-270-remove-static-reference-to-SerializationFac.patch
> (From http://mail-archives.apache.org/mod_mbox/gora-dev/201308.mbox/%3CCAG50ZE_poN4C%2B%2B8t2xLZ3MoJVDMRo6nfW_Wygd%3D%3DeteF3jyLrw%40mail.gmail.com%3E)
> Right now, IOUtils keep a *static* reference to an SerializationFactory
> which is initialized on first call to writeObject() with a Configuration
> instance. Given Configuration is also stored in a static field of same
> class for latter usage.
> But in fact each call to IOUtils.writeObject() can have a different
> Configuration instance than previous one. In my personnal use case, I've
> multiple M/R jobs which use Gora M/R feature to process Persistent object
> but each job can work with a different datastore configuration (for
> instance, name of table/collection/colum family).
> If we keep a static reference to SerializationFactory (and so its
> Configuration reference),
> QueryBase#readFields will then create a DataStore with wrong Configuration
> (ie. using first DataStore/Configuration instead of new one)
> I've started working on this issue, and come up with a possible fix :
> https://github.com/drazzib/gora/compare/apache-gora-0.2.1...ioutils_static_conf
> - remove static SerializationFactory from IOUtils (will recreate it every
> time)
> - in PartitionQueryImpl and QueryBase now send *current* configuration to
> deserialize
> One linked fix, is that gora "drivers" needs to be updated to define
> Configuration instance in PartitionQueryImpl (like this
> https://github.com/drazzib/gora/commit/395f2e2ad50d524f42ecc563104c165fa0fa6f39
> ).

This message was sent by Atlassian JIRA

View raw message