gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renato Javier Marroquín Mogrovejo (JIRA) <j...@apache.org>
Subject [jira] [Commented] (GORA-210) thread safety
Date Tue, 26 Feb 2013 18:00:16 GMT

    [ https://issues.apache.org/jira/browse/GORA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13587314#comment-13587314
] 

Renato Javier Marroquín Mogrovejo commented on GORA-210:
--------------------------------------------------------

Ok I think I get it now (: I think what confused me was that part "iterating over any of its
collection views" and getting the keySet made me think that we needed to synchronize these
extra data structures as well. Thanks!
                
> thread safety
> -------------
>
>                 Key: GORA-210
>                 URL: https://issues.apache.org/jira/browse/GORA-210
>             Project: Apache Gora
>          Issue Type: Bug
>          Components: storage-cassandra
>    Affects Versions: 0.2
>         Environment: nutch 2.1 / cassandra 1.2.1 / gora-cassandra 0.2 / gora-core 0.2.1
> running fetch with parse=true
> fetcher.threads.per.queue>1
>            Reporter: Roland
>            Priority: Critical
>              Labels: patch
>         Attachments: GORA-210.patch
>
>
> This is the result of debugging one of my issues described in NUTCH-1534.
> I think there is a wrong assumpation about thread safety of LinkedHashMap, it is not
enough to not iterate over the buffer (which is a LinkedHashMap).
> My patch fixes this error for me:
> java.util.ConcurrentModificationException
>         at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:394)
>         at java.util.LinkedHashMap$KeyIterator.next(LinkedHashMap.java:405)
>         at java.util.AbstractCollection.toArray(AbstractCollection.java:141)
>         at org.apache.gora.cassandra.store.CassandraStore.flush(CassandraStore.java:200)
>         at org.apache.gora.mapreduce.GoraRecordWriter.write(GoraRecordWriter.java:65)
>         at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:587)
>         at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
>         at org.apache.nutch.fetcher.FetcherReducer$FetcherThread.output(FetcherReducer.java:664)
>         at org.apache.nutch.fetcher.FetcherReducer$FetcherThread.run(FetcherReducer.java:534)
> It may not be perfect from a performance point of view...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message