tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kuhu Shukla <kshu...@yahoo-inc.com.INVALID>
Subject DefaultSorter/OrderedWriter and Empty Partitions
Date Thu, 02 Feb 2017 23:57:35 GMT
Hello all,

In DefaultSorter, for the OrderedPartitioned case, is the inclusion of empty partition headers
and segments in the file.out.index and file.out needed?

In the Unordered case, we prune out the empty partitions in the writer and it might be worth
doing the same in the Ordered case unless this behavior was intentional. Would appreciate
some comments on this.

Thanks and Regards,

View raw message