beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Weise (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (BEAM-5455) Don't info log for every bundle in the python sdk
Date Fri, 21 Sep 2018 01:31:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-5455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thomas Weise reassigned BEAM-5455:
----------------------------------

    Assignee: Micah Wylde  (was: Robert Bradshaw)

> Don't info log for every bundle in the python sdk
> -------------------------------------------------
>
>                 Key: BEAM-5455
>                 URL: https://issues.apache.org/jira/browse/BEAM-5455
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-py-harness
>            Reporter: Micah Wylde
>            Assignee: Micah Wylde
>            Priority: Minor
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently the python sdk logs a bunch of messages at INFO level for every bundle. For
the portable flink runner, which uses a bundle size of 1 in streaming mode, this produces
huge log spew:
> {code:java}
> [grpc-default-executor-1] INFO sdk_worker.run - Got work 4
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start <DataOutputOperation
>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start <DoOperation
window output_tags=['out']>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start <DoOperation
pair_with_one output_tags=['out']>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start <DoOperation
timestamp output_tags=['out']>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start <DoOperation
decode output_tags=['out']>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - start <DataInputOperation
receivers=[ConsumerSet[.out0, coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish <DataInputOperation
receivers=[ConsumerSet[.out0, coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish <DoOperation
decode output_tags=['out'], receivers=[ConsumerSet[decode.out0, coder=WindowedValueCoder[FastPrimitivesCoder],
len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish <DoOperation
timestamp output_tags=['out'], receivers=[ConsumerSet[timestamp.out0, coder=WindowedValueCoder[FastPrimitivesCoder],
len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish <DoOperation
pair_with_one output_tags=['out'], receivers=[ConsumerSet[pair_with_one.out0, coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder,
FastPrimitivesCoder]], len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish <DoOperation
window output_tags=['out'], receivers=[ConsumerSet[window.out0, coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder,
FastPrimitivesCoder]], len(consumers)=1]]>
> [grpc-default-executor-1] INFO bundle_processor.process_bundle - finish <DataOutputOperation
>
> {code}
>  
> These log messages should probably be at debug instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message