spot-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Ridley <>
Subject Re: [DISCUSS] New development branch for ingestion component
Date Thu, 09 Nov 2017 03:35:47 GMT
Hi Vartika-

Thanks for the reminder about the ingestion design document.  Suggest we
open a JIRA (unless one has already been opened and I missed it) with the
specific tasks broken down (create an Epic if necessary) and attach a PDF
version of the architecture document.  Would also be great to get this
document into the Spot repo.

Do we need a separate upstream branch?  I don't think there has been much
development process discussion here but I would expect the work to be in
the main dev branch after the patches/PRs are accepted and the JIRAs are


On Tue, Oct 31, 2017 at 11:31 PM, Vartika Singh <> wrote:

> Hello all,
> The following document has been out for discussion for a while:
> GyB1Msu2LUofIU4/edit#heading=h.ptz51qlqt0t
> I would like to start a new branch for ingestion component. A
> Again, the idea of ingestion component is the following:
>    1. Extendible to any source.
>    2. NRT latency between ingestion and output to the HDFS/persistent
> store.
>    3. Maintain the integrity of the ODM module in order to facilitate
>    seamless integration with applications.
> Thoughts?
> Vartika Singh

Michael Ridley <>
office: (650) 352-1337
mobile: (571) 438-2420
Senior Solutions Architect

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message