spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ewan Leith <>
Subject Any way for users to help "stuck" JIRAs with pull requests for Spark 2.3 / future releases?
Date Thu, 21 Dec 2017 16:34:19 GMT
Hi all,

I was wondering with the approach of Spark 2.3 if there's any way us "regular" users can help
advance any of JIRAs that could have made it into Spark 2.3 but are likely to miss now as
the pull requests are awaiting detailed review.

For example: - Spark SQL reads unneccesary nested fields
from Parquet

Has a pull request from January 2017 with significant performance benefits for parquet reads. - Spark has exponential time complexity
to explode(array of structs)

Probably affects fewer users, but will be a real help for those users.

Both of these example tickets probably need more testing, but without them getting merged
into the master branch and included in a release with a default config setting disabling them,
the testing will be pretty limited.

Is there anything us users can do to help out with these kind of tickets, or do they need
to wait for some additional core developer time to free up (I know that's in huge demand everywhere
in the project!).


This email and any attachments to it may contain confidential information and are intended
solely for the addressee.

If you are not the intended recipient of this email or if you believe you have received this
email in error, please contact the sender and remove it from your system.Do not use, copy
or disclose the information contained in this email or in any attachment.

RealityMine Limited may monitor email traffic data including the content of email for the
purposes of security.

RealityMine Limited is a company registered in England and Wales. Registered number: 07920936
Registered office: Warren Bruce Court, Warren Bruce Road, Trafford Park, Manchester M17 1LB

View raw message