drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Paris <nipari...@gmail.com>
Subject Re: IndexR, a new storage plugin for Drill
Date Wed, 04 Jan 2017 08:32:45 GMT
Hi Weiwan,

1)Does both drill and hive support predicat pushdown with indexR ? I mean
using the indexes and not scanning table.
2)Does it support join push down, sort etc ?
3)Can you elaborate why your team choose Drill versus equivalent (impala,
presto...)

Thanks !



2017-01-04 2:59 GMT+01:00 WeiWan <weiwan@sunteng.com>:

> Hi,
>
> It will take some time for IndexR plugin to merge into Drill. But you can
> try it out already by following those documents.
>
> Compilation:  https://github.com/shunfei/indexr/wiki/Compilation <
> https://github.com/shunfei/indexr/wiki/Compilation>
> Deployment:  https://github.com/shunfei/indexr/wiki/Deployment <
> https://github.com/shunfei/indexr/wiki/Deployment>
> User Guide:  https://github.com/shunfei/indexr/wiki/User-Guide <
> https://github.com/shunfei/indexr/wiki/User-Guide>
> Regards
> Flow Wei
>
>
>
> > On Jan 4, 2017, at 00:22, Jinfeng Ni <jni@apache.org> wrote:
> >
> > Looks like IndexR is very interesting storage plugin. Although I have
> > not looked into the detail, I'm looking forward to seeing the PR and
> > hopefully getting this into Drill!
> >
> > Thanks,
> >
> > Jinfeng
> >
> >
> > On Tue, Jan 3, 2017 at 7:30 AM, WeiWan <weiwan@sunteng.com> wrote:
> >> Hi Charles,
> >>
> >> It would be great if IndexR plugin can be merged into official Drill
> project. I will do some more tests based on latest Drill version and submit
> a PR.
> >>
> >> Regards
> >> Flow Wei
> >>
> >>
> >>
> >>> On Jan 3, 2017, at 23:18, Charles Givre <cgivre@gmail.com> wrote:
> >>>
> >>> This sounds really interesting.  Will you be submitting a PR to
> integrate this into the main Drill codebase?
> >>> — C
> >>>
> >>>> On Jan 3, 2017, at 03:35, WeiWan <weiwan@sunteng.com> wrote:
> >>>>
> >>>> IndexR is a distributed, columnar storage system based on HDFS, which
> focus on fast analyse, both for massive static(historical) data and rapidly
> ingesting realtime data. IndexR is designed for OLAP.
> >>>>
> >>>> Fast analyze on large dataset
> >>>> Realtime ingestion with zero delay for query
> >>>> Deep integration with Hadoop ecosystem
> >>>> Hardware efficiency
> >>>> Highly avaliable, scalable, manageable and simple
> >>>> Adapted with popular query engines like Apache Drill, Apache Hive,
> etc.
> >>>>
> >>>> And now it is open source.
> >>>>
> >>>> Project: https://github.com/shunfei/indexr <
> https://github.com/shunfei/indexr>
> >>>> Wiki: https://github.com/shunfei/indexr/wiki <
> https://github.com/shunfei/indexr/wiki>
> >>>>
> >>>> IndexR is original developed by Sunteng Tech. This project started a
> year ago and now has been deployed to several productions in our company.
> The whole cluster consumes over 30 billions events each day in realtime
> from Kafka. The largest table contains over 10 billions rows (after rollup)
> and rapidly increasing. Most of the statistic/analyze queries’ latency is
> less than 3 seconds in real world production environment.
> >>>>
> >>>> Currently it is mainly used as Drill and Hive storage plugin. It
> should be quite easy to master.
> >>>>
> >>>> We hope IndexR be a favor to you and make it better.
> >>>>
> >>>> Regards
> >>>> Flow Wei
> >>>>
> >>>>
> >>>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message