beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aviem Zur (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-1629) Init metrics/aggregators accumulators before traversing pipeline
Date Thu, 09 Mar 2017 07:14:37 GMT

     [ https://issues.apache.org/jira/browse/BEAM-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Aviem Zur updated BEAM-1629:
----------------------------
    Description: 
Today aggregators and metrics accumulators are instantiated after pipeline is traversed, and
so, if code that runs on driver accesses these singletons but is not part of a closure a {{NullPointerException}}
is thrown.
This is fine for streaming branches of a pipeline as you probably did something wrong, but
for batch branches this should be allowed (In completely batch pipelines accumulators are
instantiated before traversal and everything works fine, the issue occurs in streaming+batch
combined pipelines) 

  was:
Today aggregators and metrics accumulators are instantiated after pipeline is traversed, and
so, if code that runs on driver accesses these singletons but is not part of a closure a {{NullPointerException}}
is thrown.
This is fine for streaming branches of a pipeline as you probably did something wrong, but
for batch branches this should be allowed (In completely batch pipelines accumulators are
instantiated before traversal and everything works fine, the issue occurs in streaming/batch
pipelines) 


> Init metrics/aggregators accumulators before traversing pipeline
> ----------------------------------------------------------------
>
>                 Key: BEAM-1629
>                 URL: https://issues.apache.org/jira/browse/BEAM-1629
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-spark
>            Reporter: Aviem Zur
>            Assignee: Aviem Zur
>
> Today aggregators and metrics accumulators are instantiated after pipeline is traversed,
and so, if code that runs on driver accesses these singletons but is not part of a closure
a {{NullPointerException}} is thrown.
> This is fine for streaming branches of a pipeline as you probably did something wrong,
but for batch branches this should be allowed (In completely batch pipelines accumulators
are instantiated before traversal and everything works fine, the issue occurs in streaming+batch
combined pipelines) 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message