spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aniket Bhatnagar <>
Subject Data source API | sizeInBytes should be to *Scan
Date Fri, 06 Feb 2015 11:39:27 GMT
Hi Spark SQL committers

I have started experimenting with data sources API and I was wondering if
it makes sense to move the method sizeInBytes from BaseRelation to Scan
interfaces. This is because that a relation may be able to leverage filter
push down to estimate size potentially making a very large relation
broadcast-able. Thoughts?


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message