Hi Vartika-
Thanks for the reminder about the ingestion design document. Suggest we
open a JIRA (unless one has already been opened and I missed it) with the
specific tasks broken down (create an Epic if necessary) and attach a PDF
version of the architecture document. Would also be great to get this
document into the Spot repo.
Do we need a separate upstream branch? I don't think there has been much
development process discussion here but I would expect the work to be in
the main dev branch after the patches/PRs are accepted and the JIRAs are
resolved.
Michael
On Tue, Oct 31, 2017 at 11:31 PM, Vartika Singh <vsingh@cloudera.com> wrote:
> Hello all,
>
> The following document has been out for discussion for a while:
>
> https://docs.google.com/document/d/1yYaD50gp2HN9RHYaUG8ASDfh3f__
> GyB1Msu2LUofIU4/edit#heading=h.ptz51qlqt0t
>
> I would like to start a new branch for ingestion component. A
>
> Again, the idea of ingestion component is the following:
>
> 1. Extendible to any source.
> 2. NRT latency between ingestion and output to the HDFS/persistent
> store.
> 3. Maintain the integrity of the ODM module in order to facilitate
> seamless integration with applications.
>
>
> Thoughts?
>
> Vartika Singh
>
--
Michael Ridley <mridley@cloudera.com>
office: (650) 352-1337
mobile: (571) 438-2420
Senior Solutions Architect
Cloudera
|