drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhishek Girish <agir...@apache.org>
Subject Re: distributed drill on local file system
Date Thu, 16 Aug 2018 18:47:39 GMT
I'd also like to add that with the DFS storage plugin configured with local
file system, one will have to make sure all nodes with Drillbits have the
same files (under the same directory structure). If a given query on a
dataset spawns a distributed plan (multiple fragments), then it may fail if
one of the fragments on a remote Drillbit cannot find the file being
referenced by the foreman. Also, this is not something that's been well
tried out and documented, so there may be some surprises.

-Abhishek

On Thu, Aug 16, 2018 at 9:44 AM Vitalii Diravka <vitalii.diravka@gmail.com>
wrote:

> Hi Mehran,
>
> This is a question for user mailing list.
>
> Looks like there are no issues with it, you can run Drill in distributed
> mode on Windows, Linux or MacOS based machines.
> It necessary to specify *zk.connect* for Zookeeper hostname and port number
> in *drill-override.conf* file and to run *>bin/drillbit.sh start *[1].
> But a Hadoop cluster is recommended for this purpose [2], therefore not
> sure which issues can arise with this system.
>
> [1]
>
> https://drill.apache.org/docs/starting-drill-in-distributed-mode/#drillbit.sh-command-syntax
> [2] https://drill.apache.org/docs/distributed-mode-prerequisites/
>
> Kind regards
> Vitalii
>
>
> On Thu, Aug 16, 2018 at 7:11 PM Mehran Dashti [ BR - PD ] <
> m_dashti@behinrahkar.com> wrote:
>
> > Hi,
> >
> > I wanted to know if it is possible or possible by minimal effort to have
> > distributed drills that work on local file system of their own?
> >
> > We  do not want to have HDFS as file system?
> >
> >
> >
> > Thank you in advance.
> >
> >
> >
> >
> >
> > *Best Regards,*
> >
> >
> >
> > *      [image: LOGO1]*
> >
> > *Mehran Dashti*
> >
> > *Product Leader*
> >
> > *09125902452*
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message