spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koert Kuipers <>
Subject SQL analyzer breakdown
Date Tue, 15 Nov 2016 15:45:00 GMT
We see the analyzer break down almost guaranteed when programs get to a
certain size or complexity. It starts complaining with messages along the
lines of "cannot find column x#255 in list of columns that includes x#255".
The workaround is to go to rdd and back. Is there a way to achieve the same
(force the analyzer to break it up in chunks that are done independently)
without the roundtrip to rdd and back? Like a checkpoint for the analyzer
so to speak.

View raw message