giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Avery Ching (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-52) There should be a scheme to limit the counter
Date Fri, 14 Oct 2011 06:46:12 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-52?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127308#comment-13127308
] 

Avery Ching commented on GIRAPH-52:
-----------------------------------

To implement sliding windows we can be a little tricky.

counter_1 = superstep of counter 1
counter_1_val = time of superstep of counter 1
counter_2 = superstep of counter 2
counter_2_val = time of superstep of counter 2

In this way, we have to use 2n counters to display the last n supersteps, but we need not
remove any counters.

Agreed on the MR2 work.

                
> There should be a scheme to limit the counter
> ---------------------------------------------
>
>                 Key: GIRAPH-52
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-52
>             Project: Giraph
>          Issue Type: Bug
>          Components: mapreduce
>    Affects Versions: 0.70.0
>            Reporter: Zhiwei Gu
>             Fix For: 0.70.0
>
>
> For hadoop version above 0.20.203.0., the cluster-wise configuration mapreduce.job.counters.limit
cannot be overrided, while the superstep iterations is not deterministic, the job might run
several hundreds or even thousand of supersteps, it will always kill the job. This will limit
the usage of Giraph and is tooooo bad.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message