storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ajay <ajay.ga...@gmail.com>
Subject Re: Clarifications on Storm
Date Sat, 06 Dec 2014 02:56:05 GMT
Hi Parth,

Thanks for your response.

Basically I meant to have a common query interface + computation engine on
various data stores (only read from various data stores). Thats why the
question which one to use Storm or Spark  or Drill.

Thanks
Ajay


On Fri, Dec 5, 2014 at 10:48 PM, Parth Brahmbhatt <
pbrahmbhatt@hortonworks.com> wrote:

> See inline.
>
> On Dec 5, 2014, at 3:37 AM, Ajay <ajay.garga@gmail.com> wrote:
>
> Hello,
>
> I work for an eCommerce company. Currently we are looking at building a
> Data warehouse platform as described below:
>
> DW as a Service
>     |
> REST API
>     |
> SQL On No SQL (Drill/Pig/Hive/Spark/Storm)
>     |
> No SQL databases (One or more. May be RDBMS directly too)
>     | (Bulk load)
> My SQL Database
>
> I wish to get a few clarifications on Apache Drill as follows:
>
> 1) Can we use Storm on No SQL or do we need to mix them with
> Pig/Hive/Spark or any other for any reason?
>
> Not sure what you mean here.
>
> 2) Can Storm be used a query interface for Business Intelligence,
> Analytics and Reporting
>
> The general pattern I have seen is you run storm to do
> aggregation/filtering and to load that into a data store that BI people are
> most familiar with (Hive). If you want Storm as the query interface you
> might want to look into Pig On Storm
> https://cwiki.apache.org/confluence/display/PIG/Pig+on+Storm+Proposal.
>
> 3) Do Storm supports only Hadoop, HBase?. We may use
> Cassandra/MongoDb/CouchBase as well.
>
> Storm out of the box comes with connectors for Kafka, HDFS and Hbase.
> However, there are connectors for many other systems in open source
> https://github.com/hmsonline/storm-cassandra. And you can easily add
> bolts/spouts for any storage engine that you can’t find in open source.
>
> 4) Do Storm supports RDBMS too?. We can have a single interface to pull
> out  data from multiple data sources?
>
> Not out of the box but I am sure there are open source connectors for this.
>
> 5) Any recommendations(not limited to usage of Storm) for our specific
> requirement described above.
>
> I am not entirely sure where in your pipeline you intend to use storm. If
> its on as SQL on No SQL store , storm might not be a good fit right now as
> I don’t think Pig on storm is available in open source yet.
>
>
> Thanks
> Ajay
>
> Note : I have posted a similar post on the Drill and Spark User list as
> well as I am not sure which one best fits for our usecase.
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Mime
View raw message