phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-3600) Core MapReduce classes don't provide location info
Date Sun, 05 Feb 2017 18:12:41 GMT


Hadoop QA commented on PHOENIX-3600:

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment
  against master branch at commit 58ae6668c6097e0237a0cbf2233d3cf6aca77410.
  ATTACHMENT ID: 12851086

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 42 warning messages.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
    +    private List<InputSplit> generateSplits(final QueryPlan qplan, final List<KeyRange>
splits, Configuration config) throws IOException {
+        org.apache.hadoop.hbase.client.Connection connection = ConnectionFactory.createConnection(config);
+                    psplits.add(new PhoenixInputSplit(Collections.singletonList(aScan), regionSize,

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results:
Javadoc warnings:
Console output:

This message is automatically generated.

> Core MapReduce classes don't provide location info
> --------------------------------------------------
>                 Key: PHOENIX-3600
>                 URL:
>             Project: Phoenix
>          Issue Type: Improvement
>    Affects Versions: 4.8.0
>            Reporter: Josh Mahonin
>            Assignee: Josh Mahonin
>         Attachments: PHOENIX-3600.patch, PHOENIX-3600_v2.patch
> The core MapReduce classes {{org.apache.phoenix.mapreduce.PhoenixInputSplit}} and {{org.apache.phoenix.mapreduce.PhoenixInputFormat}}
don't provide region size or location information, leaving the execution engine (MR, Spark,
etc.) to randomly assign splits to nodes.
> Interestingly, the phoenix-hive module has reimplemented these classes, including the
node-aware functionality. We should port a subset of those changes back to the core code so
that other engines can make use of them.

This message was sent by Atlassian JIRA

View raw message