drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@dremio.com>
Subject Re: Drill (CTAS) Default hadoop Replication factor on HDFS ?
Date Sat, 07 May 2016 17:59:23 GMT
My suggestion would be to use Drill's capability to have config options in
the storage plugin rather than copying the hdfs-site.xml everywhere. Keeps
it in one place and allows you to tune per system you are interacting with
(instead of globally). See here for more detail:

https://issues.apache.org/jira/browse/DRILL-4383

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Fri, May 6, 2016 at 10:17 AM, Abhishek Girish <abhishek.girish@gmail.com>
wrote:

> Hello,
>
> Assuming you have defined your replication factor setting inside your
> cluster hdfs-site.xml, it might be worth a try to copy this config file
> into your Drill conf directory (on all nodes). While I haven't tried this
> myself, i'm hoping this could help.
>
> -Abhishek
>
> On Fri, May 6, 2016 at 12:50 AM, Shankar Mane <shankar.mane@games24x7.com>
> wrote:
>
> > We have hadoop cluster where default replication factor (dfs.replication)
> > is set to 1 ( this cluster is just plug and play, hence we don't need to
> > store more than 1 copies).
> >
> > When we used drill *CTAS*, it has created table on *HDFS* with their
> > own *replication
> > factor of 3. *
> >
> > *Questions are *-
> > 1. why cant it uses Hadoop default replication factor ?
> > 2. Is there any setting in Drill to change hadoop replication factor
> > realtime ?
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message