drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Rogers <prog...@maprtech.com>
Subject Re: move drill log directory to HDFS
Date Fri, 26 Aug 2016 17:32:58 GMT
I don’t believe it is possible to store Drill logs in HDFS using an HDFS URL. The log files
are set up via environment variables in the drill-config.sh script and are expected to be
normal file system files.

However, if you have the ability to NFS mount your distributed file system, then you can easily
accomplish your goal. (MapR MFS is such an example, others may support this as well.) Let’s
say that you mount your DFS at /mnt/dfs. Then, you can add the following to your drill-env.sh
script:

export DRILL_LOG_DIR=/mnt/dfs/drill/`hostname`

The bit about hostname is needed to put the logs for each Drillbit into a separate directory.
Feel free to use any other technique you like to label the directory.

As an alternative, we are making good progress in finalizing a YARN hosting solution for Drill.
Once this is available (probably Drill 1.9), then YARN will take care of collecting your Drill
log files.

- Paul

> On Aug 26, 2016, at 6:46 AM, Andries Engelbrecht <aengelbrecht@maprtech.com> wrote:
> 
> https://drill.apache.org/docs/persistent-configuration-storage/ <https://drill.apache.org/docs/persistent-configuration-storage/>
> 
> 
> --Andries
> 
>> On Aug 26, 2016, at 4:25 AM, Anup Tiwari <anup.tiwari@games24x7.com> wrote:
>> 
>> Also please note that I have tried below in all node's drill-env.sh but its
>> not working.
>> 
>> export DRILL_LOG_DIR="hdfs://namenode:9000/tmp/drilllogs/"
>> 
>> 
>> Regards,
>> *Anup Tiwari*
>> 
>> 
>> On Fri, Aug 26, 2016 at 4:06 PM, Anup Tiwari <anup.tiwari@games24x7.com>
>> wrote:
>> 
>>> Hi All,
>>> 
>>> We are trying to move drill logs directory from local file system to HDFS
>>> so that we can refer only one location rather than each node's log
>>> directory.
>>> 
>>> Can anyone help me on this?
>>> 
>>> Regards,
>>> *Anup Tiwari*
>>> 
>>> 
> 


Mime
View raw message