drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Paris <nipari...@gmail.com>
Subject Re: Best architecture
Date Tue, 21 Feb 2017 22:21:55 GMT
Hi,

Join csv, json, databases.
Your needs looks like ETL processes. I am not sure drill suits well for
such goal. AFAIK, it is not able to work on disk when out of memory
occures.

Moreover those tasks usally needs some procedural code parts. I am not
sure UDFs are very flexible.

For such use case, I would use ETL tools such talend and load monetdb
direcly with it.

Le 19 févr. 2017 à 18:02, Gustavo Brian écrivait :
> Hi there,
> 
> I'm newbie to this, so i apology if I'm asking something senseless :)
> 
> Thanks for this amazing product. I'm planning to use it as main query
> engine for data analysis. My plan is to have a raw storage where to drop
> different types of documents: csv, json,... as they are produced by the
> apps. Then use Drill to query and join against sql database to produce
> enriched data to drop into a columnar storage: monetdb, druid,...
> 
> My question is: is there a preferred storage engine for this raw storage?
> Can Drill take advantage of other engines like hadoop or yarn?
> 
> Thanks in advance

Mime
View raw message