drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Drawater <chris.drawa...@jdsu.com>
Subject Re: Distributed SQL on Drill question
Date Fri, 24 Oct 2014 10:33:35 GMT
Jason Altekruse <altekrusejason@...> writes:


> Hi Chris,


> Welcome to the drill community!


> What are you currently using as a VM environment? Is it running on your 


> hardware or a cloud provider like AWS or Google Compute Engine? So far

> drill has required the nodes to be able to communicate using UDP multicast

> to sync some query information around the cluster for distributed queries.

> We realized that these providers do not support this level of network

> communication between nodes, so we have been trying to develop a

> workaround, but distributed queries will currently fail in these

> environments.


> If this is not the case, a look through the logs for more detailed errors

> would be the most useful for debugging your specific issue.


> Connecting to a specific node or a zk quorum should be the same in terms 


> allowing execution of distributed queries, both JDBC and ODBC connections

> will run distributed queries with locality information taken into account

> where available.


> Distributed queries can be identified by exchanges inserted in the plan,

> this is where data is sent around to various nodes to split or merge the

> data when an operation is parallelized.


> -Jason Altekruse

Thanks Jason.

We're currently using vSphere on a Windows 2010 host for dev/test purposes.

The VM network is bridged onto our internal(corporate) network and we've 

been told that multicast support is disabled by (corporate) default.... and 

that a multicast router would be required for our subnet...

Apparently multicast support has now been enabled but our distr SQL queries 

still only return data from a single Drillbit  and the logs show no 

communication between Drillbits... so maybe we still have a UDP multicast 


I guess the next step is to find some Linux UDP test utilities and/or write 

a simple Java UDP test application.


View raw message