beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Halperin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-932) Findbugs doesn't pass in Spark runner
Date Mon, 07 Nov 2016 17:42:58 GMT

     [ https://issues.apache.org/jira/browse/BEAM-932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Halperin updated BEAM-932:
---------------------------------
    Description: 
{code}
[INFO] --- findbugs-maven-plugin:3.0.1:check (default) @ beam-runners-spark ---
[INFO] BugInstance size is 19
[INFO] Error size is 0
[INFO] Total bugs: 19
[INFO] instanceof will always return false in org.apache.beam.runners.spark.SparkRunner.run(Pipeline),
since a RuntimeException can't be a org.apache.spark.SparkException [org.apache.beam.runners.spark.SparkRunner]
At SparkRunner.java:[line 161]
[INFO] org.apache.beam.runners.spark.aggregators.metrics.WithNamedAggregatorsSupport$1.apply(Map$Entry)
makes inefficient use of keySet iterator instead of entrySet iterator [org.apache.beam.runners.spark.aggregators.metrics.WithNamedAggregatorsSupport$1]
At WithNamedAggregatorsSupport.java:[line 125]
[INFO] The class name org.apache.beam.runners.spark.aggregators.metrics.sink.CsvSink shadows
the simple name of the superclass org.apache.spark.metrics.sink.CsvSink [org.apache.beam.runners.spark.aggregators.metrics.sink.CsvSink]
At CsvSink.java:[lines 37-38]
[INFO] The class name org.apache.beam.runners.spark.aggregators.metrics.sink.GraphiteSink
shadows the simple name of the superclass org.apache.spark.metrics.sink.GraphiteSink [org.apache.beam.runners.spark.aggregators.metrics.sink.GraphiteSink]
At GraphiteSink.java:[lines 37-38]
[INFO] t must be non-null but is marked as nullable [org.apache.beam.runners.spark.translation.WindowingHelpers$4]
At WindowingHelpers.java:[line 88]
[INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$1.evaluate(ConsoleIO$Write$Unbound,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$1]
At StreamingTransformTranslator.java:[line 92]
[INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$10.evaluate(ParDo$Bound,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$10]
At StreamingTransformTranslator.java:[line 360]
[INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$11.evaluate(ParDo$BoundMulti,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$11]
At StreamingTransformTranslator.java:[line 395]
[INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$2.evaluate(Read$Unbounded,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$2]
At StreamingTransformTranslator.java:[line 104]
[INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$3.evaluate(CreateStream$QueuedValues,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$3]
At StreamingTransformTranslator.java:[line 115]
[INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$4.evaluate(Flatten$FlattenPCollectionList,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$4]
At StreamingTransformTranslator.java:[line 128]
[INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$5.evaluate(Window$Bound,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$5]
At StreamingTransformTranslator.java:[line 167]
[INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$6.evaluate(GroupByKey,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$6]
At StreamingTransformTranslator.java:[line 215]
[INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$7.evaluate(Combine$GroupedValues,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$7]
At StreamingTransformTranslator.java:[line 251]
[INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$8.evaluate(Combine$Globally,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$8]
At StreamingTransformTranslator.java:[line 281]
[INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$9.evaluate(Combine$PerKey,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$9]
At StreamingTransformTranslator.java:[line 321]
[INFO] Inconsistent synchronization of org.apache.beam.runners.spark.util.BroadcastHelper$CodedBroadcastHelper.value;
locked 75% of time [org.apache.beam.runners.spark.util.BroadcastHelper$CodedBroadcastHelper,
org.apache.beam.runners.spark.util.BroadcastHelper$CodedBroadcastHelper, org.apache.beam.runners.spark.util.BroadcastHelper$CodedBroadcastHelper,
org.apache.beam.runners.spark.util.BroadcastHelper$CodedBroadcastHelper] Unsynchronized access
at BroadcastHelper.java:[line 111]Synchronized access at BroadcastHelper.java:[line 103]Synchronized
access at BroadcastHelper.java:[line 106]Synchronized access at BroadcastHelper.java:[line
104]
[INFO] Inconsistent synchronization of org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper.bcast;
locked 50% of time [org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper,
org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper] Unsynchronized access
at BroadcastHelper.java:[line 81]Synchronized access at BroadcastHelper.java:[line 74]
[INFO] Inconsistent synchronization of org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper.value;
locked 75% of time [org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper,
org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper, org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper,
org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper] Unsynchronized access
at BroadcastHelper.java:[line 81]Synchronized access at BroadcastHelper.java:[line 73]Synchronized
access at BroadcastHelper.java:[line 76]Synchronized access at BroadcastHelper.java:[line
74]
[INFO]    
{code}

  was:
{code}
[INFO] --- findbugs-maven-plugin:3.0.1:check (default) @ beam-runners-flink_2.10 ---
[INFO] BugInstance size is 9
[INFO] Error size is 0
[INFO] Total bugs: 9
[INFO] Class org.apache.beam.runners.flink.FlinkRunner$StreamingViewAsMap defines non-transient
non-serializable instance field runner [org.apache.beam.runners.flink.FlinkRunner$StreamingViewAsMap]
In FlinkRunner.java
[INFO] Class org.apache.beam.runners.flink.FlinkRunner$StreamingViewAsMultimap defines non-transient
non-serializable instance field runner [org.apache.beam.runners.flink.FlinkRunner$StreamingViewAsMultimap]
In FlinkRunner.java
[INFO] Return value of org.apache.beam.sdk.io.Read$Bounded.getSource() ignored, but method
has no side effect [org.apache.beam.runners.flink.translation.FlinkStreamingTransformTranslators$BoundedReadSourceTranslator]
At FlinkStreamingTransformTranslators.java:[line 282]
[INFO] Return value of org.apache.beam.sdk.io.Read$Unbounded.getSource() ignored, but method
has no side effect [org.apache.beam.runners.flink.translation.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator]
At FlinkStreamingTransformTranslators.java:[line 252]
[INFO] org.apache.beam.runners.flink.translation.wrappers.SerializableFnAggregatorWrapper.clone()
does not call super.clone() [org.apache.beam.runners.flink.translation.wrappers.SerializableFnAggregatorWrapper]
At SerializableFnAggregatorWrapper.java:[lines 84-89]
[INFO] Class org.apache.beam.runners.flink.translation.wrappers.streaming.WindowDoFnOperator
defines non-transient non-serializable instance field stateInternals [org.apache.beam.runners.flink.translation.wrappers.streaming.WindowDoFnOperator]
In WindowDoFnOperator.java
[INFO] Class org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSocketSource$UnboundedSocketReader
defines non-transient non-serializable instance field reader [org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSocketSource$UnboundedSocketReader]
In UnboundedSocketSource.java
[INFO] Class org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSocketSource$UnboundedSocketReader
defines non-transient non-serializable instance field socket [org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSocketSource$UnboundedSocketReader]
In UnboundedSocketSource.java
[INFO] Unconditional wait in org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper.run(SourceFunction$SourceContext)
[org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper] At
UnboundedSourceWrapper.java:[line 243]
[INFO]   
{code}


> Findbugs doesn't pass in Spark runner
> -------------------------------------
>
>                 Key: BEAM-932
>                 URL: https://issues.apache.org/jira/browse/BEAM-932
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-extensions
>            Reporter: Daniel Halperin
>            Assignee: Aljoscha Krettek
>
> {code}
> [INFO] --- findbugs-maven-plugin:3.0.1:check (default) @ beam-runners-spark ---
> [INFO] BugInstance size is 19
> [INFO] Error size is 0
> [INFO] Total bugs: 19
> [INFO] instanceof will always return false in org.apache.beam.runners.spark.SparkRunner.run(Pipeline),
since a RuntimeException can't be a org.apache.spark.SparkException [org.apache.beam.runners.spark.SparkRunner]
At SparkRunner.java:[line 161]
> [INFO] org.apache.beam.runners.spark.aggregators.metrics.WithNamedAggregatorsSupport$1.apply(Map$Entry)
makes inefficient use of keySet iterator instead of entrySet iterator [org.apache.beam.runners.spark.aggregators.metrics.WithNamedAggregatorsSupport$1]
At WithNamedAggregatorsSupport.java:[line 125]
> [INFO] The class name org.apache.beam.runners.spark.aggregators.metrics.sink.CsvSink
shadows the simple name of the superclass org.apache.spark.metrics.sink.CsvSink [org.apache.beam.runners.spark.aggregators.metrics.sink.CsvSink]
At CsvSink.java:[lines 37-38]
> [INFO] The class name org.apache.beam.runners.spark.aggregators.metrics.sink.GraphiteSink
shadows the simple name of the superclass org.apache.spark.metrics.sink.GraphiteSink [org.apache.beam.runners.spark.aggregators.metrics.sink.GraphiteSink]
At GraphiteSink.java:[lines 37-38]
> [INFO] t must be non-null but is marked as nullable [org.apache.beam.runners.spark.translation.WindowingHelpers$4]
At WindowingHelpers.java:[line 88]
> [INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$1.evaluate(ConsoleIO$Write$Unbound,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$1]
At StreamingTransformTranslator.java:[line 92]
> [INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$10.evaluate(ParDo$Bound,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$10]
At StreamingTransformTranslator.java:[line 360]
> [INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$11.evaluate(ParDo$BoundMulti,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$11]
At StreamingTransformTranslator.java:[line 395]
> [INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$2.evaluate(Read$Unbounded,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$2]
At StreamingTransformTranslator.java:[line 104]
> [INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$3.evaluate(CreateStream$QueuedValues,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$3]
At StreamingTransformTranslator.java:[line 115]
> [INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$4.evaluate(Flatten$FlattenPCollectionList,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$4]
At StreamingTransformTranslator.java:[line 128]
> [INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$5.evaluate(Window$Bound,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$5]
At StreamingTransformTranslator.java:[line 167]
> [INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$6.evaluate(GroupByKey,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$6]
At StreamingTransformTranslator.java:[line 215]
> [INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$7.evaluate(Combine$GroupedValues,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$7]
At StreamingTransformTranslator.java:[line 251]
> [INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$8.evaluate(Combine$Globally,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$8]
At StreamingTransformTranslator.java:[line 281]
> [INFO] Unchecked/unconfirmed cast from org.apache.beam.runners.spark.translation.EvaluationContext
to org.apache.beam.runners.spark.translation.streaming.StreamingEvaluationContext in org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$9.evaluate(Combine$PerKey,
EvaluationContext) [org.apache.beam.runners.spark.translation.streaming.StreamingTransformTranslator$9]
At StreamingTransformTranslator.java:[line 321]
> [INFO] Inconsistent synchronization of org.apache.beam.runners.spark.util.BroadcastHelper$CodedBroadcastHelper.value;
locked 75% of time [org.apache.beam.runners.spark.util.BroadcastHelper$CodedBroadcastHelper,
org.apache.beam.runners.spark.util.BroadcastHelper$CodedBroadcastHelper, org.apache.beam.runners.spark.util.BroadcastHelper$CodedBroadcastHelper,
org.apache.beam.runners.spark.util.BroadcastHelper$CodedBroadcastHelper] Unsynchronized access
at BroadcastHelper.java:[line 111]Synchronized access at BroadcastHelper.java:[line 103]Synchronized
access at BroadcastHelper.java:[line 106]Synchronized access at BroadcastHelper.java:[line
104]
> [INFO] Inconsistent synchronization of org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper.bcast;
locked 50% of time [org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper,
org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper] Unsynchronized access
at BroadcastHelper.java:[line 81]Synchronized access at BroadcastHelper.java:[line 74]
> [INFO] Inconsistent synchronization of org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper.value;
locked 75% of time [org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper,
org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper, org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper,
org.apache.beam.runners.spark.util.BroadcastHelper$DirectBroadcastHelper] Unsynchronized access
at BroadcastHelper.java:[line 81]Synchronized access at BroadcastHelper.java:[line 73]Synchronized
access at BroadcastHelper.java:[line 76]Synchronized access at BroadcastHelper.java:[line
74]
> [INFO]    
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message