beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmet Altay (JIRA)" <>
Subject [jira] [Created] (BEAM-527) Pickling error when pickling a nested function
Date Thu, 04 Aug 2016 01:10:20 GMT
Ahmet Altay created BEAM-527:

             Summary: Pickling error when pickling a nested function 
                 Key: BEAM-527
             Project: Beam
          Issue Type: Bug
          Components: sdk-py
            Reporter: Ahmet Altay
            Priority: Minor

There is a pickling error under the following conditions all happen:
- a function is defined inside a transforms' apply method
- then using it as MapFn
- that function references an instance variable of the outer transform.

Rewriting the nested function as an unnested DoFn appears to solve the problem.

If the limitations of pickling make it difficult to support nested functions then perhaps
there's a way to make it easier for users to detect problems caused by nested functions and
recommend appropriate fixes

This message was sent by Atlassian JIRA

View raw message