drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alfredo Serafini <ser...@gmail.com>
Subject describe query support? (catalog metadata, etc)
Date Wed, 18 Oct 2017 15:31:15 GMT
Hi I'm experimenting using Drill as a data virtualization component via
JDBC and it generally works great for my needs.

However some of the components connected via JDBC needs basic
metadata/catalog informations, and they seems to be missing for JSON / CSV
sources.

For example the simple query

DESCRIBE cp.`employee.json`;

returns no results.

Another possible example case could be when reading from an sqlite source
containing the same data on an `employees` table
DESCRIBE `emploees`

and still get no information: while this command is not directly supported
in SQLite, an equivalent one could be for instance:
PRAGMA table_info(`employees`);

but trying to execute it in Drill is not possible, as it is beyond the
supported standard SQL dialect.

Moreover using a query like:
SELECT *
FROM INFORMATION_SCHEMA.COLUMNS
WHERE (TABLE_NAME='employees_view');

on a view from the same data, seems to return the informations, so I
suppose there should be a way to pass those informations to an
internal *DatabaseMetaData
<https://docs.oracle.com/javase/8/docs/api/java/sql/DatabaseMetaData.html>*
implementation.
I wonder if there is such a component designed to manage all the catalog
informations for different sources?

In this case it could adopt different strategies for retrieving metadata,
depending on the case: for sqlite a different command / dialect could be
used, for CSV types could be guessed using simple heuristics, and so on.
Probably cases like JSON would be much more complex, anyway.
Once the metadata have been retrieved for a source, I suppose the standard
SQL dialect should work as expected.


Are there any plans to add catalog metadata support for various sources?
Does anybody have some workaround? for example using views or similar
approaches?


thanks in advance, sorry if the message is too long :-)
Alfredo

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message