gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Damien Raude-Morvan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GORA-270) IOUtils static SerializationFactory field
Date Tue, 03 Sep 2013 19:23:51 GMT

    [ https://issues.apache.org/jira/browse/GORA-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13756959#comment-13756959
] 

Damien Raude-Morvan commented on GORA-270:
------------------------------------------

Hi Henry,

I'm in vacation right now, so need to hurry for this review :)
I can confirm you that we will need to fix all other stores to set the conf: I haven't found
a better way to fix this for now.

Regards,
                
> IOUtils static SerializationFactory field
> -----------------------------------------
>
>                 Key: GORA-270
>                 URL: https://issues.apache.org/jira/browse/GORA-270
>             Project: Apache Gora
>          Issue Type: Bug
>          Components: gora-core
>            Reporter: Damien Raude-Morvan
>            Assignee: Damien Raude-Morvan
>              Labels: mapreduce
>         Attachments: 0001-GORA-270-remove-static-reference-to-SerializationFac.patch
>
>
> (From http://mail-archives.apache.org/mod_mbox/gora-dev/201308.mbox/%3CCAG50ZE_poN4C%2B%2B8t2xLZ3MoJVDMRo6nfW_Wygd%3D%3DeteF3jyLrw%40mail.gmail.com%3E)
> Right now, IOUtils keep a *static* reference to an SerializationFactory
> which is initialized on first call to writeObject() with a Configuration
> instance. Given Configuration is also stored in a static field of same
> class for latter usage.
> But in fact each call to IOUtils.writeObject() can have a different
> Configuration instance than previous one. In my personnal use case, I've
> multiple M/R jobs which use Gora M/R feature to process Persistent object
> but each job can work with a different datastore configuration (for
> instance, name of table/collection/colum family).
> If we keep a static reference to SerializationFactory (and so its
> Configuration reference),
> QueryBase#readFields will then create a DataStore with wrong Configuration
> (ie. using first DataStore/Configuration instead of new one)
> I've started working on this issue, and come up with a possible fix :
> https://github.com/drazzib/gora/compare/apache-gora-0.2.1...ioutils_static_conf
> - remove static SerializationFactory from IOUtils (will recreate it every
> time)
> - in PartitionQueryImpl and QueryBase now send *current* configuration to
> deserialize
> One linked fix, is that gora "drivers" needs to be updated to define
> Configuration instance in PartitionQueryImpl (like this
> https://github.com/drazzib/gora/commit/395f2e2ad50d524f42ecc563104c165fa0fa6f39
> ).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message