Yes, it certainly is a trade-off. Hence we want the execution engine to
support different types of connectors/channels between the operators.
In-memory is one option. On-disk (eg, what Hadoop's MapReduce uses) is
another option.
On Mon, Aug 27, 2012 at 12:05 PM, David Gruzman <david@bigdatacraft.com>wrote:
> I think that execution engine should trade fault tolerance to low latency.
> If query takes a few seconds - it
> easier to re-run it then build intra query fault tolerance.
> David
>
> On Mon, Aug 27, 2012 at 9:53 PM, Julien Le Dem <julien@twitter.com> wrote:
>
> > Hi Ted
> > As long as this plan has primitives like "group by" and "filter" it
> > would be relatively easy to wrap Pig operators into it. We did
> > something similar to run Pig on Spark. If performance is an issue,
> > separate execution engines can also coexist.
> > Julien
> >
> > On Sun, Aug 26, 2012 at 2:22 PM, Ted Dunning <ted.dunning@gmail.com>
> > wrote:
> > > Camuel,
> > >
> > > Do you have a grammar test suite that demonstrates the range of
> > expressions?
> > >
> > > Also, I believe that some have a goal to use additional languages
> besides
> > > SQL like languages. A limited version of pig, for instance, would be
> > very
> > > interesting. To do this, it will be important to have a logical plan
> > > structure that is common for different syntaxes and is not limited to
> the
> > > idiosyncracies of any particular syntax.
> > >
> > > How do you think that should be handled? Do you have an idea for a
> > logical
> > > plan structure?
> > >
> > > On Sun, Aug 26, 2012 at 4:11 PM, Camuel Gilyadov <camuel@gmail.com>
> > wrote:
> > >
> > >> I've written and attached ANTLR grammar for DrQL which I assume is
> same
> > as
> > >> BigQuery language described in Query Reference on BigQuery website.
> This
> > >> grammar includes AST production rules.
> > >>
> > >>
> > >>
> >
>
--
Tomer Shiran
Director of Product Management | MapR Technologies | 650-804-8657
|