drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Re: select from table with options
Date Wed, 21 Oct 2015 15:57:35 GMT
Whatever API is used to scan files from SQL, there will need to be a
corresponding way to accomplish the same thing in a user interface.
Probably a form with various fields, some of them with drop-boxes etc.

And ideally a facility that samples a few hundred rows to deduce the
probable field names and types and which fields are unique.

I think that the UI is the true "user friendly" interface. A usage
pattern might be for people to define a data source using in the UI,
save it as a view, then switch to the command line to write queries on
that view.

There are other use cases similar to reading flies. For example you
would like to read data from an HTTP URL. You might want to specify
similar parameters for formats, compression, parsing, and parameters
in the file URI that describe a family of partitioned files. A URL
might allow push-down of filters, projects and sorts. But still you
would want to specify formats, compression and parsing the same way as
reading files.

To me, this argues for decomposing the file scan syntax into pieces
that can be re-used if you get data from places other than files.


On Wed, Oct 21, 2015 at 6:15 AM, Jacques Nadeau <jacques@dremio.com> wrote:
>> This fourth is also least extensible and thus most disenfranchising for
>> those outside the inner group.
>> Table functions (hopefully) would be something that others could
> implement.
> This is a brainstorm about a user apis...
> It isn't appropriate to shoot down ideas immediately in a brainstorm. It
> has a chilling effect on other people presenting new ideas.
> User apis should be defined first with an end-user in mind. Consistency in
> different contexts is also very important (note my expansion of the
> discussion to ASCRIBE METADATA.)
> Your statement about extensibility has no technical backing. If you have a
> concern like this, ask the proposer if they think that this could be done
> in an extensible way. In this case I see a number of ways that this could
> be done. Note the mention of the grammar switch in my initial proposal or
> go review my outstanding patch for integrating JSON literals. In many ways,
> I think this approach could be considered the most extensible and
> expressive for a Drill extension developer.
> I hope others will continue to brainstorm on this thread.

View raw message