spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-22634) Update Bouncy castle dependency
Date Wed, 08 Aug 2018 04:50:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-22634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572675#comment-16572675
] 

Steve Loughran commented on SPARK-22634:
----------------------------------------

If nothing else is using it, correct. And nothing is using any of the bouncy castle APIs directly.

But: you need to be sure that nothing else is using it through the javax.crypto APIs, especially
the stuff in org.apache.spark.network.crypto, or worse: some library which uses those APIs.

The NOTICE files certainly hint that it's being used somehow

bq. This product optionally depends on 'Bouncy Castle Crypto APIs' to generate a temporary
self-signed X.509 certificate when the JVM does not provide the equivalent functionality.


There's not enough history in the git logs to line that up with any code that pops up with
a quick scan.

Safest to update to the later version, while cutting the jets3t dependency (which is provably
not used, it being incompatible with the shipping bc lib). Most due diligence: cut out bouncy
castle and see what breaks...

> Update Bouncy castle dependency
> -------------------------------
>
>                 Key: SPARK-22634
>                 URL: https://issues.apache.org/jira/browse/SPARK-22634
>             Project: Spark
>          Issue Type: Task
>          Components: Spark Core, SQL, Structured Streaming
>    Affects Versions: 2.2.0
>            Reporter: Lior Regev
>            Assignee: Sean Owen
>            Priority: Minor
>             Fix For: 2.3.0
>
>
> Spark's usage of jets3t library as well as Spark's own Flume and Kafka streaming uses
bouncy castle version 1.51
> This is an outdated version as the latest one is 1.58
> This, in turn renders packages such as [spark-hadoopcryptoledger-ds|https://github.com/ZuInnoTe/spark-hadoopcryptoledger-ds]
unusable since these require 1.58 and spark's distributions come along with 1.51
> My own attempt was to run on EMR, and since I automatically get all of spark's dependecies
(bouncy castle 1.51 being one of them) into the classpath, using the library to parse blockchain
data failed due to missing functionality.
> I have also opened an [issue|https://bitbucket.org/jmurty/jets3t/issues/242/bouncycastle-dependency]
with jets3t to update their dependecy as well, but along with that Spark would have to update
it's own or at least be packaged with a newer version



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message