drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arina Yelchiyeva <arina.yelchiy...@gmail.com>
Subject Re: Need help with UDF/Custom Functions (DRILL-4776)
Date Sun, 20 Nov 2016 15:43:05 GMT
Hi Joseph,

I assume in drill-module.conf you indicate path to your package where
function class is in (ex: drill.classpath.scanning.packages +=
"com.drill.udf.md5"),
and after adding your jars to the classpath ($DRILL/jars), you restart your
drillbit.
Also you need to update your code:
1. do not use imports, use fully qualified class name for HashFunction,
HashCode and StandardCharsets.
2. annotate class variable HashFunction with @Workspace annotation:
    @Workspace
    com.google.common.hash.HashFunction hashFunction;

Hope this helps.

Kind regards
Arina

On Sun, Nov 20, 2016 at 2:20 AM, Charles Givre <cgivre@gmail.com> wrote:

> Hi Joseph,
> Can you share your pom.xml?  Also what command are you using to build your
> UDF?
> Thanks,
> — Charles
>
>
> > On Nov 18, 2016, at 14:44, Joseph Swingle <joeswingle@gmail.com> wrote:
> >
> > Hi Arina,
> > I apologize for not getting back to you sooner, and if this doesn't get
> > formatted properly. I just want to MD5 a string in a query.
> > I have pasted the java code for the function below.  I included
> DRILL-4776
> > in my initial question, because it pretty much summed up my frustration
> > trying to deploy this function.
> >
> > Here are the additional things I have done.
> >
> >   1. drill-module.conf.   I have tried including in my jars.  I have
> tried
> >   including it empty in my jar, I have added to $DRILL/conf folder.
> >   2. I have both the classes and sources jars in $DRILL/jars.  I have
> even
> >   tried putting them in $DRILL/jars/3rdParty
> >   3. Source hasn't changed it is below.
> >
> > A simple query like this one:   SELECT (`finish` - `start`) as
> > `taken`,`start`,finish, outcome, username, to_md5(queryText)
> > FROM drillprofiles.`default`.`drillbit_queries.json` limit 2;
> >
> > give following error:
> > Error: VALIDATION ERROR: From line 1, column 75 to line 1, column 91: No
> > match found for function signature to_md5(<ANY>)
> >
> >
> > @FunctionTemplate( name = "to_md5", scope = FunctionTemplate.
> FunctionScope.
> > SIMPLE,  nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
> >
> > public class MD5MaskFunc implements DrillSimpleFunc {
> >
> >    @Param
> >
> >    NullableVarCharHolder input;
> >
> >    @Output
> >
> >    VarCharHolder out;
> >
> >    @Inject
> >
> >    DrillBuf buffer;
> >
> >    HashFunction hashFunction;
> >
> >    public void setup() {
> >
> >    hashFunction = Hashing.md5();
> >
> >    }
> >
> >    public void eval() {
> >
> >    String stringValue =
> > org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.
> toStringFromUTF8(
> > input.start, input.end, input.buffer);
> >
> >    HashCode hashCode = hashFunction.hashString(stringValue,
> > StandardCharsets.UTF_8);//hashFunction.hashString(stringValue);
> >
> >    String outputValue = hashCode.toString();
> >
> >        // put the output value in the out buffer
> >
> >        out.buffer = buffer;
> >
> >        out.start = 0;
> >
> >        out.end = outputValue.getBytes().length;
> >
> >        buffer.setBytes(0, outputValue.getBytes());
> >
> >    }
> >
> > }
> >
> >
> >
> > On Wed, Nov 9, 2016 at 11:09 AM, Arina Yelchiyeva <
> > arina.yelchiyeva@gmail.com> wrote:
> >
> >> Hi Joseph,
> >>
> >> custom functions work should work fine in 1.8. At least I am not aware
> of
> >> any issues.
> >> Can you specify what problems with custom UDFs do you have?
> >>
> >> Kind regards
> >> Arina
> >>
> >> On Wed, Nov 9, 2016 at 6:02 PM, Joseph Swingle <joeswingle@gmail.com>
> >> wrote:
> >>
> >>> Drill team,
> >>>
> >>> Are Custom Functions simply broke in Version 1.8?
> >>>
> >>> Any movement on DRILL-4776 would be appreciated.
> >>>
> >>> Heck,  willing to even fix all the doc,  provided someone can provide
> me
> >> a
> >>> Custom Function that works in 1.8.
> >>>
> >>> https://issues.apache.org/jira/browse/DRILL-4776
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message