tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kuhu Shukla (JIRA)" <j...@apache.org>
Subject [jira] [Created] (TEZ-3776) ShuffleHandler can theoretically NPE if all partitions in the ranged fetch are empty
Date Mon, 26 Jun 2017 21:15:00 GMT
Kuhu Shukla created TEZ-3776:
--------------------------------

             Summary: ShuffleHandler can theoretically NPE if all partitions in the ranged
fetch are empty
                 Key: TEZ-3776
                 URL: https://issues.apache.org/jira/browse/TEZ-3776
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Kuhu Shukla


{code}
if (index.getPartLength() != 0) {
          if (firstIndex == null) {
            firstIndex = index;
          }
          lastIndex = index;
        }

        ShuffleHeader header = new ShuffleHeader(mapId, index.getPartLength(), index.getRawLength(),
reduce);
        DataOutputBuffer dob = new DataOutputBuffer();
        header.write(dob);
        ch.write(wrappedBuffer(dob.getData(), 0, dob.getLength()));
      }

      final long rangeOffset = firstIndex.getStartOffset(); <<<<<<<<<<<<<<<<<
{code}
In cases where tez.runtime.empty.partitions.info-via-events.enabled is false and is respected
(say post TEZ-3287, if not handled correctly at the output side, the shuffle handler assumes
that if it has got to this point there is at least one non empty partition. The ShuffleHandler
should protect itself from this NPE.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message