drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Claude Cote <jcc...@gmail.com>
Subject Re: storage plugin test case
Date Wed, 26 Sep 2018 19:17:51 GMT
I see how the cluster.makeDataDir call will setup the configuration for a
MsgpackFormatConfig. I'll update my test cases to use it.

However there is another question related to this. If I don't modify the
boostrap-storage-plugins.json then when I launch the drill-embeded it does
not know of the MsgpackFormatConfig. I need to use the web console to edit
the storage plugins.

Is there a better way?
Thanks
Jean-Claude


On Wed, Sep 26, 2018 at 1:17 PM, Arina Yelchiyeva <
arina.yelchiyeva@gmail.com> wrote:

> Taking into the account that your code is in the contrib module, modifying
> boostrap-storage-plugins.json does not make any sense.
> If you need to add your own format in unit tests, as Vitalii pointed out,
> TestCsv is a good example for this.
>
> Kind regards,
> Arina
>
> On Wed, Sep 26, 2018 at 7:07 PM Jean-Claude Cote <jccote@gmail.com> wrote:
>
> > I found the cause of the problem I had. It was not due to the fact that
> the
> > classloader did not find the resource. It is that my new FormatPlugin was
> > not registered into the
> > drill\exec\java-exec\src\main\resources\bootstrap-storage-plugins.json
> >
> >     cp: {
> >       type: "file",
> >       connection: "classpath:///",
> >       formats: {
> >         "msgpack" : {
> >           type: "msgpack",
> >           extensions: [ "mp" ]
> >         },
> >
> > So the resource was rejected. I've added this entry to the
> > boostrap-storage-plugins.json and now it works.
> >
> > Thanks for all your help.
> > Jean-Claude
> >
> >
> >
> > On Wed, Sep 26, 2018 at 7:18 AM, Arina Yelchiyeva <
> > arina.yelchiyeva@gmail.com> wrote:
> >
> > > This can also help:
> > >
> > > 1. create TEST resources folder where you'll copy your data.
> > > 2. use dirTestWatcher to copy data to the root / tmp / custom
> directory.
> > > 3. query the data:
> > >
> > > @Test
> > > public void t() {
> > >   dirTestWatcher.copyResourceToRoot(Paths.get("complex_1.parquet"));
> > >   queryBuilder().sql("select * from dfs.`root`.`complex_1.parquet`
> > > ").printCsv();
> > > }
> > >
> > >
> > > On Wed, Sep 26, 2018 at 12:37 PM Vitalii Diravka <vitalii@apache.org>
> > > wrote:
> > >
> > > > Hi Jean-Claude
> > > >
> > > > BaseTestQuery is deprecated. Please use ClusterTest instead.
> > > > See TestCsv.java for example.
> > > >
> > > > You can find more info about Drill Cluster-Fixture-Framework here:
> > > > https://github.com/paul-rogers/drill/wiki/Cluster-Fixture-Framework
> > > >
> > > > On Wed, Sep 26, 2018 at 12:00 AM Jean-Claude Cote <jccote@gmail.com>
> > > > wrote:
> > > >
> > > > > I have writing a msgpack storage plugin from drill.
> > > > > https://github.com/jcmcote/drill/tree/master/contrib/
> storage-msgpack
> > > > >
> > > > > I'm now trying to write test cases like
> > > > >
> > > > >     testBuilder()
> > > > >         .sqlQuery("select * from cp.`msgpack/testBasic.mp`")
> > > > >         .ordered()
> > > > >         .baselineColumns("a").baselineValues("1").
> baselineValues("1")
> > > > >         .baselineColumns("b").baselineValues("2").
> baselineValues("2")
> > > > >         .build().run();
> > > > >
> > > > > However when I run the test case it says it cannot find the
> > > > > msgpack/testBasic.mp file. However it is in my src/test/resources
> > > folder.
> > > > >
> > > > > Should this work? I'm I going at it the right way?
> > > > > Thanks
> > > > > jc
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message