phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-10) Push projection of a single ARRAY element to the server
Date Wed, 29 Jan 2014 08:36:09 GMT


James Taylor commented on PHOENIX-10:

>From Ram:I tried out a prototype of this, just for a query that selects an indexed array
type column. I added an filter while creating a scan and implemented the filterKeyValue(kv)
in that filter.

public ReturnCode filterKeyValue(KeyValue kv) {
    inputTuple.setResult(new Result(new KeyValue[]{kv}));
    if(evaluate(inputTuple)) {
        byte[] val = new byte[tempPtr.getLength()];
        System.arraycopy(tempPtr.get(), tempPtr.getOffset(), val, 0, tempPtr.getLength());
        //KeyValue newKv = new KeyValue(kv.getRow(), kv.getFamily(), kv.getQualifier(), val);
        kv = new KeyValue(kv.getRow(), kv.getFamily(), kv.getQualifier(), val);
        return ReturnCode.INCLUDE;
    } else {
        return super.filterKeyValue(kv);

protected boolean evaluate(Tuple input) {
    try {
        if (!expression.evaluate(input, tempPtr)) {
            return false;
    } catch (IllegalDataException e) {
        return Boolean.FALSE;
    return true;
Where expression is that ArrayIndexFunction. But how can we rewrite the KeyValue itself that
is coming to the filterKeyValue.

Am not able to rewrite the kv. Is there something am missing here? 
One thing is if the query is
SELECT a_double_array[2] FROM table_with_array
it selects only that KV which has this qualifier a_double_array. So we have to get the required
indexed value from this keyvalue only.

I think the transform() in Filter can be used here? But that would be costly IMO. Also need
to see if rewriting KV would help.

> Push projection of a single ARRAY element to the server
> -------------------------------------------------------
>                 Key: PHOENIX-10
>                 URL:
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: James Taylor
> If only a single array element is selected, we'll still return the entire array back
to the client. Instead, we should push this to the server and only return the single array
element. The same goes for the reference to an ARRAY in the WHERE clause. There's a general
HBase fix for this (i.e. the ability to define a separate set of key values that will be returned
versus key values available to filters) that has a patch here, but is deemed not possible
to pull into the 0.94 branch by @lhofhansl.
> My thought is that we can add a Filter at the end our our filter chain that filters out
any KeyValues that aren't in the SELECT expressions (i.e. filter out if a column is referenced
in the WHERE clause, but not in the SELECT expressions). This same Filter could handle returning
only the elements of the array that are referenced in the SELECT expression rather than the
entire array.

This message was sent by Atlassian JIRA

View raw message