ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanouil Gkatziouras <gkatzio...@gmail.com>
Subject Re: [DISCUSS] Pub/Sub Streamer Implementation
Date Tue, 26 Nov 2019 18:48:45 GMT
Hi all!

If someone could give me read access to the Job of Ignite Extensions on
Team City will greatly help me (username gkatzioura).
I suppose the builds get triggered automatically on a pull request.

Kind regards,
Emmanouil

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <gkatzioura@gmail.com>
wrote:

> Hi Saikat!
>
> I just rebased with the flink branch. Unfortunately I do not have read
> access to the team city link you provided and thus evaluate the tests for
> my pull request.
> I guess it has to do with ignite extensions being new.
> My username is gkatzioura and email the one I am using on this email
>
> Thank you for your time.
>
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <saikat.maitra@gmail.com>
> wrote:
>
>> Hi Emmanouil,
>>
>> The latest build on teamcity has passed on Flink pull request.
>>
>>
>> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
>>
>> you should be able to take the changes I made in my PR and run build on
>> your pull request.
>>
>> Please let me know if you have any questions.
>>
>> Regards,
>> Saikat
>>
>> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <saikat.maitra@gmail.com>
>> wrote:
>>
>> > Hi Emmanouil,
>> >
>> > I have fixed the Flink module testsuites and have setup new project in
>> > teamcity.
>> >
>> >
>> >
>> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
>> >
>> > I am looking into setting up the build.
>> >
>> > Regards,
>> > Saikat
>> >
>> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <saikat.maitra@gmail.com>
>> > wrote:
>> >
>> >> Hi Emmanouil,
>> >>
>> >> Thank you for your email. Yes, the plan is once the Flink PR is merged
>> it
>> >> will provide the Licence, parent POMs etc and you can rebase from
>> master
>> >> branch and apply the changes on top of it.
>> >>
>> >> I am getting some issues with test failures in local with
>> >> GridTestProperties as the test.properties is not present in this new
>> >> project but should be available from dependencies.
>> >>
>> >> Once I address this issue, I will go ahead and merge the changes and
>> then
>> >> we can take it from there.
>> >>
>> >> Regards,
>> >> Saikat
>> >>
>> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
>> >> gkatzioura@gmail.com> wrote:
>> >>
>> >>> Hi all,
>> >>>
>> >>> I made my first pull request [1]. Since this project is brand new (no
>> >>> parent poms, licensing), I reckoned it was best to use Saikat's
>> branch on
>> >>> Flink.
>> >>> I suppose the Flink branch will be merged first. If not please give me
>> >>> guidelines on how I should proceed next.
>> >>>
>> >>> Kind regards
>> >>> Emmanouil
>> >>>
>> >>> [1] https://github.com/apache/ignite-extensions/pull/2
>> >>>
>> >>> *Emmanouil Gkatziouras*
>> >>> https://egkatzioura.com/ |
>> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> https://github.com/gkatzioura
>> >>>
>> >>>
>> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <dmagda@apache.org> wrote:
>> >>>
>> >>> > Awesome, ping us whenever you're ready!
>> >>> >
>> >>> > -
>> >>> > Denis
>> >>> >
>> >>> >
>> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>> >>> > gkatzioura@gmail.com>
>> >>> > wrote:
>> >>> >
>> >>> > > Hi all!
>> >>> > >
>> >>> > > I am sorry for being late on that. I was trying to refactor the
>> test
>> >>> in
>> >>> > > order not to be in need of any external tools or spinning up a
>> >>> server.
>> >>> > > I did forked the new repo and indeed my changes there, so a pull
>> >>> request
>> >>> > is
>> >>> > > a matter of time!
>> >>> > >
>> >>> > > Kind regards
>> >>> > >
>> >>> > > *Emmanouil Gkatziouras*
>> >>> > > https://egkatzioura.com/ |
>> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > https://github.com/gkatzioura
>> >>> > >
>> >>> > >
>> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <dmagda@apache.org>
>> wrote:
>> >>> > >
>> >>> > > > Hi Emmanouil,
>> >>> > > >
>> >>> > > > Do you have any questions or need any support from the
>> community?
>> >>> > > >
>> >>> > > > -
>> >>> > > > Denis
>> >>> > > >
>> >>> > > >
>> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>> >>> saikat.maitra@gmail.com
>> >>> > >
>> >>> > > > wrote:
>> >>> > > >
>> >>> > > >> Hi Emmanouil,
>> >>> > > >>
>> >>> > > >> Can you please take a looks at dev utils, if this is something
>> >>> you are
>> >>> > > >> looking for
>> >>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils
>> ?
>> >>> > > >>
>> >>> > > >> IMO, if you can release Pub/Sub server in maven and then use
>> it as
>> >>> > > >> dependency, that would be great.
>> >>> > > >>
>> >>> > > >> Regards,
>> >>> > > >> Saikat
>> >>> > > >>
>> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>> >>> > saikat.maitra@gmail.com>
>> >>> > > >> wrote:
>> >>> > > >>
>> >>> > > >> > Hi Emmanouil,
>> >>> > > >> >
>> >>> > > >> > The master branch has been initialized. I have raised a PR to
>> >>> > migrate
>> >>> > > >> > Flink module to ignite-extensions.
>> >>> > > >> >
>> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
>> >>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>> >>> > > >> >
>> >>> > > >> > Regards,
>> >>> > > >> > Saikat
>> >>> > > >> >
>> >>> > > >> >
>> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>> >>> > > >> > gkatzioura@gmail.com> wrote:
>> >>> > > >> >
>> >>> > > >> >> Hi all!
>> >>> > > >> >>
>> >>> > > >> >> This is an update. The integration is tested with mocks
>> based
>> >>> on
>> >>> > the
>> >>> > > >> >> original Ignite repo [1].
>> >>> > > >> >> I am currently waiting for the first commits on the new
>> >>> integration
>> >>> > > >> >> project
>> >>> > > >> >> to take place in order to create my pull request accordingly
>> >>> to the
>> >>> > > >> >> initial
>> >>> > > >> >> commits.
>> >>> > > >> >> In order to simulate the Pub/Sub server in a test I used a
>> test
>> >>> > > server
>> >>> > > >> >> implementation [2]. Unfortunately they don't have this util
>> >>> > uploaded
>> >>> > > >> in a
>> >>> > > >> >> maven repo, thus it needs to be hosted on a maven repo
>> (Apache
>> >>> > > License
>> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
>> >>> > > >> test/build?
>> >>> > > >> >> I have also created a ticket [3]
>> >>> > > >> >>
>> >>> > > >> >> Kind regards
>> >>> > > >> >>
>> >>> > > >> >> [1]
>> >>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> >>> > > >> >> [2]
>> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> >>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>> >>> > > >> >>
>> >>> > > >> >> *Emmanouil Gkatziouras*
>> >>> > > >> >> https://egkatzioura.com/ |
>> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> https://github.com/gkatzioura
>> >>> > > >> >>
>> >>> > > >> >>
>> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
>> >>> > saikat.maitra@gmail.com
>> >>> > > >
>> >>> > > >> >> wrote:
>> >>> > > >> >>
>> >>> > > >> >> > Hello Ilya,
>> >>> > > >> >> >
>> >>> > > >> >> > Thank you for your email. Yes, before we remove any
>> >>> integration
>> >>> > we
>> >>> > > >> will
>> >>> > > >> >> > move them to another repository and release.
>> >>> > > >> >> >
>> >>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
>> >>> > implementation
>> >>> > > >> and
>> >>> > > >> >> > continue to move other integration as mentioned in the
>> >>> following
>> >>> > > doc.
>> >>> > > >> >> >
>> >>> > > >> >> >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> >
>> >>> > > >> >> > Regards,
>> >>> > > >> >> > Saikat
>> >>> > > >> >> >
>> >>> > > >> >> >
>> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> >>> > > >> >> > ilya.kasnacheev@gmail.com>
>> >>> > > >> >> > wrote:
>> >>> > > >> >> >
>> >>> > > >> >> > > Hello!
>> >>> > > >> >> > >
>> >>> > > >> >> > > My take on this: we want to offload some of
>> integrations,
>> >>> but
>> >>> > > >> please
>> >>> > > >> >> > commit
>> >>> > > >> >> > > and release them to another project first, and only
>> then we
>> >>> > will
>> >>> > > >> >> decide
>> >>> > > >> >> > to
>> >>> > > >> >> > > remove them from our repository.
>> >>> > > >> >> > >
>> >>> > > >> >> > > Please also make sure they are available with same
>> artifact
>> >>> > name
>> >>> > > >> and
>> >>> > > >> >> API.
>> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
>> >>> > > >> >> > >
>> >>> > > >> >> > > Regards,
>> >>> > > >> >> > > --
>> >>> > > >> >> > > Ilya Kasnacheev
>> >>> > > >> >> > >
>> >>> > > >> >> > >
>> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>> >>> > > >> saikat.maitra@gmail.com>:
>> >>> > > >> >> > >
>> >>> > > >> >> > > > Hello Emmanouil,
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
>> >>> community
>> >>> > > and
>> >>> > > >> >> they
>> >>> > > >> >> > > are
>> >>> > > >> >> > > > interested to have Apache Ignite extensions as part of
>> >>> Apache
>> >>> > > >> Bahir
>> >>> > > >> >> > > > project.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > Can you please share Pub/Sub streamer implementation
>> >>> details
>> >>> > > with
>> >>> > > >> >> > Apache
>> >>> > > >> >> > > > Bahir community and request for feedback.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > https://bahir.apache.org/
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > I have also requested for contributor access in Jira
>> for
>> >>> > Apache
>> >>> > > >> >> Bahir
>> >>> > > >> >> > > > project so that I can create issues and assign to
>> myself.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > I can help with code reviews as well.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > Here is the email thread for reference
>> >>> > > >> >> > > >
>> >>> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > Regards,
>> >>> > > >> >> > > > Saikat
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> >>> > > >> >> saikat.maitra@gmail.com
>> >>> > > >> >> > >
>> >>> > > >> >> > > > wrote:
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > > Hello,
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > As discussed I have created following discussion
>> >>> threads on
>> >>> > > our
>> >>> > > >> >> > > migration
>> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >>
>> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > I will share update as I hear more information.
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > Regards,
>> >>> > > >> >> > > > > Saikat
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>> >>> > > >> >> > saikat.maitra@gmail.com
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > > wrote:
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >> Hello Denis,
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
>> discussion
>> >>> > > thread
>> >>> > > >> to
>> >>> > > >> >> get
>> >>> > > >> >> > > > >> community feedback on whether we
>> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
>> Extensions"
>> >>> > > project.
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >> Regards,
>> >>> > > >> >> > > > >> Saikat
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
>> >>> > > >> dmagda@apache.org>
>> >>> > > >> >> > > wrote:
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >>> Folks,
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we
>> >>> need!
>> >>> > > >> Saikat,
>> >>> > > >> >> > thanks
>> >>> > > >> >> > > > for
>> >>> > > >> >> > > > >>> doing the research. Why I believe the * idea*
>> fits us
>> >>> > well:
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>>    - All integrations can be stored in separate
>> >>> Github
>> >>> > > >> >> repositories
>> >>> > > >> >> > > and
>> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged to
>> >>> couple
>> >>> > > all
>> >>> > > >> >> the
>> >>> > > >> >> > > > >>> integrations
>> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark can be
>> >>> located
>> >>> > in
>> >>> > > a
>> >>> > > >> >> > > dedicated
>> >>> > > >> >> > > > >>> repo
>> >>> > > >> >> > > > >>>    while streaming integrations might be in a
>> single
>> >>> one.
>> >>> > > >> It's
>> >>> > > >> >> up
>> >>> > > >> >> > to
>> >>> > > >> >> > > > us.
>> >>> > > >> >> > > > >>>    - All the repositories and integrations belong
>> to
>> >>> ASF.
>> >>> > > >> We're
>> >>> > > >> >> not
>> >>> > > >> >> > > > >>> dumping
>> >>> > > >> >> > > > >>>    them on Github but rather keep supporting and
>> >>> > developing
>> >>> > > >> in
>> >>> > > >> >> > > > >>> accordance with
>> >>> > > >> >> > > > >>>    ASF vision and practices.
>> >>> > > >> >> > > > >>>    - There is a way to reward contributors via
>> >>> > > committership
>> >>> > > >> and
>> >>> > > >> >> > PMC
>> >>> > > >> >> > > > >>>    membership. You won't get this if a project is
>> >>> just
>> >>> > one
>> >>> > > of
>> >>> > > >> >> the
>> >>> > > >> >> > > > >>> millions of
>> >>> > > >> >> > > > >>>    Github projects.
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
>> >>> kick-off a
>> >>> > > >> >> separate
>> >>> > > >> >> > > > >>> dev-list
>> >>> > > >> >> > > > >>> discussion to involve more community members and
>> >>> > referring
>> >>> > > to
>> >>> > > >> >> this
>> >>> > > >> >> > > > >>> thread?
>> >>> > > >> >> > > > >>> I think the primary question the community needs
>> to
>> >>> > answer
>> >>> > > >> >> whether
>> >>> > > >> >> > we
>> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
>> Extensions"
>> >>> > > >> project?
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> -
>> >>> > > >> >> > > > >>> Denis
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
>> >>> > > >> >> > > > >>> wrote:
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
>> Integrations
>> >>> > there,
>> >>> > > >> but
>> >>> > > >> >> > what
>> >>> > > >> >> > > is
>> >>> > > >> >> > > > >>> about
>> >>> > > >> >> > > > >>> > maintaining and committer permissions to the new
>> >>> > > >> repositories?
>> >>> > > >> >> > > > >>> > I see the list of the committers and PMC members
>> >>> there
>> >>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > Could somebody from Ignite community be added to
>> >>> this
>> >>> > > list
>> >>> > > >> as
>> >>> > > >> >> a
>> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
>> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
>> >>> collaborate
>> >>> > > with
>> >>> > > >> >> these
>> >>> > > >> >> > > > guys
>> >>> > > >> >> > > > >>> > together
>> >>> > > >> >> > > > >>> > If no, and we should start from the zero level
>> with
>> >>> > > >> external
>> >>> > > >> >> PRs
>> >>> > > >> >> > -
>> >>> > > >> >> > > > >>> hmmm,
>> >>> > > >> >> > > > >>> > it's better to have our own external repository
>> >>> with
>> >>> > > >> >> ApacheBahirr
>> >>> > > >> >> > > > >>> ideology.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could be
>> moved
>> >>> there
>> >>> > > (in
>> >>> > > >> >> > > > >>> ApacheBahirr
>> >>> > > >> >> > > > >>> > like repository), but not all modules from the
>> page
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> > > > >>> > because
>> >>> > > >> >> > > > >>> > they use different parts of core modules and
>> could
>> >>> be
>> >>> > > >> >> developed
>> >>> > > >> >> > > with
>> >>> > > >> >> > > > >>> > different velocity.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > In reality, before creation of new repositories
>> we
>> >>> need
>> >>> > > >> wide
>> >>> > > >> >> > > > discussion
>> >>> > > >> >> > > > >>> > based on the document mentioned above.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next
>> >>> release
>> >>> > > 2.8
>> >>> > > >> >> with
>> >>> > > >> >> > > one
>> >>> > > >> >> > > > >>> or two
>> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
>> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
>> >>> > > >> repositories
>> >>> > > >> >> in
>> >>> > > >> >> > > > Apache
>> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
>> >>> > > >> >> > > saikat.maitra@gmail.com
>> >>> > > >> >> > > > >:
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > Thank you for your email. I think creating a
>> >>> separate
>> >>> > > >> >> > integration
>> >>> > > >> >> > > > >>> project
>> >>> > > >> >> > > > >>> > > in github and also proposing it as an apache
>> >>> > incubator
>> >>> > > >> >> project
>> >>> > > >> >> > > will
>> >>> > > >> >> > > > >>> be a
>> >>> > > >> >> > > > >>> > > good move. The other separate integration
>> modules
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> > > > >>> > > can
>> >>> > > >> >> > > > >>> > > be moved to this new apache incubator project.
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > There are similar integration available for
>> >>> Flink and
>> >>> > > >> Spark
>> >>> > > >> >> in
>> >>> > > >> >> > > > Apache
>> >>> > > >> >> > > > >>> > Bahir
>> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > Do you think we should reach out to Apache
>> Bahir
>> >>> > > >> community
>> >>> > > >> >> > with a
>> >>> > > >> >> > > > >>> > proposal
>> >>> > > >> >> > > > >>> > > or we should plan to create a separate Apache
>> >>> > Incubator
>> >>> > > >> >> > project?
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > Regards,
>> >>> > > >> >> > > > >>> > > Saikat
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
>> >>> > Gkatziouras <
>> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
>> >>> > > >> >> > > > >>> > > wrote:
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > > Hi all!
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > Based on the discussions most probably this
>> is
>> >>> > going
>> >>> > > to
>> >>> > > >> >> be on
>> >>> > > >> >> > > > >>> another
>> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
>> >>> standalone
>> >>> > > >> project
>> >>> > > >> >> > with
>> >>> > > >> >> > > > the
>> >>> > > >> >> > > > >>> > > Pub/Sub
>> >>> > > >> >> > > > >>> > > > module and once the repository is created a
>> >>> pull
>> >>> > > >> request
>> >>> > > >> >> > shall
>> >>> > > >> >> > > be
>> >>> > > >> >> > > > >>> > issued.
>> >>> > > >> >> > > > >>> > > > If there is anything else I can do in order
>> to
>> >>> > > >> facilitate
>> >>> > > >> >> > this
>> >>> > > >> >> > > > >>> please
>> >>> > > >> >> > > > >>> > let
>> >>> > > >> >> > > > >>> > > > me know.
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I
>> shall
>> >>> be
>> >>> > > able
>> >>> > > >> to
>> >>> > > >> >> > > create
>> >>> > > >> >> > > > a
>> >>> > > >> >> > > > >>> pull
>> >>> > > >> >> > > > >>> > > > request and thus a build will take place on
>> >>> > ignite's
>> >>> > > >> CI.
>> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>> >>> > > >> >> implementation
>> >>> > > >> >> > > may I
>> >>> > > >> >> > > > >>> > create
>> >>> > > >> >> > > > >>> > > a
>> >>> > > >> >> > > > >>> > > > ticket on JIRA?
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
>> >>> configures
>> >>> > > the
>> >>> > > >> >> build
>> >>> > > >> >> > > > jobs
>> >>> > > >> >> > > > >>> in
>> >>> > > >> >> > > > >>> > > Team
>> >>> > > >> >> > > > >>> > > > City. The unit tests for my implementation
>> >>> need a
>> >>> > > local
>> >>> > > >> >> > Pub/Sub
>> >>> > > >> >> > > > >>> server
>> >>> > > >> >> > > > >>> > up
>> >>> > > >> >> > > > >>> > > > and running (they provide one for testing
>> >>> > purposes).
>> >>> > > >> Will
>> >>> > > >> >> > this
>> >>> > > >> >> > > > take
>> >>> > > >> >> > > > >>> > > effect
>> >>> > > >> >> > > > >>> > > > in a form of a build script inside my
>> >>> > implementation
>> >>> > > or
>> >>> > > >> >> is it
>> >>> > > >> >> > > > >>> something
>> >>> > > >> >> > > > >>> > > > that should be configured on Team City?
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > Kind regards,
>> >>> > > >> >> > > > >>> > > > Emmanouil
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > >
>> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
>> >>> > > >> >> dmagda@apache.org
>> >>> > > >> >> > >
>> >>> > > >> >> > > > >>> wrote:
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > After contemplating on this, let me
>> propose
>> >>> > another
>> >>> > > >> way
>> >>> > > >> >> > with
>> >>> > > >> >> > > > this
>> >>> > > >> >> > > > >>> > > > > particular integration that is aligned
>> with
>> >>> our
>> >>> > > >> >> > > modularization
>> >>> > > >> >> > > > >>> > > endeavor.
>> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the Ignite
>> >>> core,
>> >>> > > >> Ignite
>> >>> > > >> >> > > modules,
>> >>> > > >> >> > > > >>> and
>> >>> > > >> >> > > > >>> > > > > independent integrations. The first two
>> are
>> >>> to be
>> >>> > > >> >> located
>> >>> > > >> >> > in
>> >>> > > >> >> > > > >>> Ignite
>> >>> > > >> >> > > > >>> > > > > repositories and to be tested/updated by
>> the
>> >>> > > >> community
>> >>> > > >> >> for
>> >>> > > >> >> > > > every
>> >>> > > >> >> > > > >>> > > release.
>> >>> > > >> >> > > > >>> > > > > The modules are a subset of existing
>> >>> integrations
>> >>> > > >> that
>> >>> > > >> >> are
>> >>> > > >> >> > > > >>> adopted
>> >>> > > >> >> > > > >>> > > widely
>> >>> > > >> >> > > > >>> > > > > and require community attention.
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > As for the independent integrations [1],
>> >>> those
>> >>> > are
>> >>> > > >> >> supposed
>> >>> > > >> >> > > to
>> >>> > > >> >> > > > be
>> >>> > > >> >> > > > >>> > > located
>> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise
>> us
>> >>> to
>> >>> > > start
>> >>> > > >> >> > > > >>> implementing
>> >>> > > >> >> > > > >>> > this
>> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even
>> >>> though
>> >>> > > it
>> >>> > > >> >> will
>> >>> > > >> >> > be
>> >>> > > >> >> > > > >>> located
>> >>> > > >> >> > > > >>> > > in a
>> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be listed
>> among
>> >>> > > >> officially
>> >>> > > >> >> > > > >>> available
>> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
>> featuring
>> >>> > > >> Emmanouil
>> >>> > > >> >> as a
>> >>> > > >> >> > > > >>> > > contributor.
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > What do you think?
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > [1]
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > -
>> >>> > > >> >> > > > >>> > > > > Denis
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
>> >>> Maitra <
>> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > > wrote:
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted
>> to
>> >>> check
>> >>> > > if
>> >>> > > >> you
>> >>> > > >> >> > are
>> >>> > > >> >> > > > also
>> >>> > > >> >> > > > >>> > able
>> >>> > > >> >> > > > >>> > > to
>> >>> > > >> >> > > > >>> > > > > > validate that the once data is added to
>> >>> Ignite
>> >>> > > >> cluster
>> >>> > > >> >> > you
>> >>> > > >> >> > > > are
>> >>> > > >> >> > > > >>> also
>> >>> > > >> >> > > > >>> > > > able
>> >>> > > >> >> > > > >>> > > > > to
>> >>> > > >> >> > > > >>> > > > > > access it using another Ignite client or
>> >>> using
>> >>> > > rest
>> >>> > > >> >> > > > endpoints.
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the
>> >>> details are
>> >>> > > as
>> >>> > > >> >> > follows
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test suites
>> >>> that
>> >>> > have
>> >>> > > >> been
>> >>> > > >> >> > > > >>> affected by
>> >>> > > >> >> > > > >>> > > your
>> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
>> >>> > > http://ci.ignite.apache.org/>:
>> >>> > > >> <
>> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to
>> check,
>> >>> > press
>> >>> > > >> >> button
>> >>> > > >> >> > > > named
>> >>> > > >> >> > > > >>> > "..."
>> >>> > > >> >> > > > >>> > > > that
>> >>> > > >> >> > > > >>> > > > > >    is located on the left of "Run"
>> button.
>> >>> "Run
>> >>> > > >> custom
>> >>> > > >> >> > > build"
>> >>> > > >> >> > > > >>> > dialog
>> >>> > > >> >> > > > >>> > > > will
>> >>> > > >> >> > > > >>> > > > > >    appear;
>> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
>> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
>> >>> > > >> >> > > > >>> > > > in
>> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
>> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and
>> monitor
>> >>> tests
>> >>> > > >> >> results.
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also
>> >>> please
>> >>> > > >> take a
>> >>> > > >> >> > > look?
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > Regards,
>> >>> > > >> >> > > > >>> > > > > > Saikat
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
>> Emmanouil
>> >>> > > >> Gkatziouras
>> >>> > > >> >> <
>> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
>> >>> > > >> >> > > > >>> > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite
>> >>> fork on
>> >>> > > my
>> >>> > > >> >> > personal
>> >>> > > >> >> > > > >>> GitHub
>> >>> > > >> >> > > > >>> > > > > account
>> >>> > > >> >> > > > >>> > > > > > > [1]
>> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
>> implementation
>> >>> which
>> >>> > > is
>> >>> > > >> of
>> >>> > > >> >> a
>> >>> > > >> >> > > > polling
>> >>> > > >> >> > > > >>> > > nature.
>> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service like
>> >>> Kafka.
>> >>> > > >> Instead
>> >>> > > >> >> it
>> >>> > > >> >> > > > >>> provides us
>> >>> > > >> >> > > > >>> > > > with
>> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
>> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as
>> a
>> >>> > > guideline
>> >>> > > >> >> made
>> >>> > > >> >> > > more
>> >>> > > >> >> > > > >>> sense
>> >>> > > >> >> > > > >>> > > to
>> >>> > > >> >> > > > >>> > > > > me.
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation I have
>> also
>> >>> > > added a
>> >>> > > >> >> unit
>> >>> > > >> >> > > > test.
>> >>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing
>> and a
>> >>> > double
>> >>> > > >> >> check
>> >>> > > >> >> > on
>> >>> > > >> >> > > my
>> >>> > > >> >> > > > >>> side
>> >>> > > >> >> > > > >>> > to
>> >>> > > >> >> > > > >>> > > > > make
>> >>> > > >> >> > > > >>> > > > > > > the review smooth.
>> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2]
>> server
>> >>> has
>> >>> > to
>> >>> > > >> be
>> >>> > > >> >> > setup
>> >>> > > >> >> > > > >>> however
>> >>> > > >> >> > > > >>> > I
>> >>> > > >> >> > > > >>> > > am
>> >>> > > >> >> > > > >>> > > > > not
>> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to do so
>> with
>> >>> > > regards
>> >>> > > >> to
>> >>> > > >> >> > your
>> >>> > > >> >> > > > >>> CI/CD.
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > [1]
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> >
>> >>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> >>> > > >> >> > > > >>> > > > > > > [2]
>> >>> > > >> >> > > > >>>
>> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
>> >>> Maitra <
>> >>> > > >> >> > > > >>> > > saikat.maitra@gmail.com
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Can you please share more info about
>> >>> the
>> >>> > > >> >> > > implementation?
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > We have similar implementation for
>> data
>> >>> > > >> streamers
>> >>> > > >> >> > with
>> >>> > > >> >> > > > >>> Kafka
>> >>> > > >> >> > > > >>> > and
>> >>> > > >> >> > > > >>> > > > > > RocketMQ
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > >
>> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Please review and share if you have
>> any
>> >>> > > >> questions.
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Regards,
>> >>> > > >> >> > > > >>> > > > > > > > Saikat
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
>> >>> Emmanouil
>> >>> > > >> >> > Gkatziouras <
>> >>> > > >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
>> >>> > > >> >> > > > >>> > > > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean
>> Google
>> >>> > Cloud's
>> >>> > > >> >> Pub/Sub
>> >>> > > >> >> > > [1]
>> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific.
>> It
>> >>> is
>> >>> > the
>> >>> > > >> >> > messaging
>> >>> > > >> >> > > > >>> service
>> >>> > > >> >> > > > >>> > > > > provided
>> >>> > > >> >> > > > >>> > > > > > > by
>> >>> > > >> >> > > > >>> > > > > > > > > GCP.
>> >>> > > >> >> > > > >>> > > > > > > > > I have made the integration with
>> it,
>> >>> but
>> >>> > I
>> >>> > > >> >> haven't
>> >>> > > >> >> > > > >>> published
>> >>> > > >> >> > > > >>> > it
>> >>> > > >> >> > > > >>> > > > yet
>> >>> > > >> >> > > > >>> > > > > > (I
>> >>> > > >> >> > > > >>> > > > > > > am
>> >>> > > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket
>> and
>> >>> > > proceed
>> >>> > > >> >> with
>> >>> > > >> >> > > the
>> >>> > > >> >> > > > >>> > > > > contribution).
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
>> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > [1]
>> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45,
>> Denis
>> >>> > Magda <
>> >>> > > >> >> > > > >>> dmagda@apache.org
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any
>> specific
>> >>> > > >> >> implementation
>> >>> > > >> >> > > > >>> tailored
>> >>> > > >> >> > > > >>> > > for a
>> >>> > > >> >> > > > >>> > > > > > > cloud?
>> >>> > > >> >> > > > >>> > > > > > > > If
>> >>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were
>> >>> mentioning
>> >>> > > >> AWS/GCE.
>> >>> > > >> >> Is
>> >>> > > >> >> > > > there
>> >>> > > >> >> > > > >>> any
>> >>> > > >> >> > > > >>> > > > Github
>> >>> > > >> >> > > > >>> > > > > > > > source
>> >>> > > >> >> > > > >>> > > > > > > > > > code?
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
>> >>> integrations
>> >>> > > >> with
>> >>> > > >> >> > > > streaming
>> >>> > > >> >> > > > >>> and
>> >>> > > >> >> > > > >>> > > > > pub/sub
>> >>> > > >> >> > > > >>> > > > > > > > > > frameworks:
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
>> >>> master:
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to see
>> >>> what
>> >>> > are
>> >>> > > >> the
>> >>> > > >> >> > most
>> >>> > > >> >> > > > >>> suitable
>> >>> > > >> >> > > > >>> > > > > Ignite
>> >>> > > >> >> > > > >>> > > > > > > APIs
>> >>> > > >> >> > > > >>> > > > > > > > > for
>> >>> > > >> >> > > > >>> > > > > > > > > > such integrations.
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
>> >>> > saikat.maitra@gmail.com>
>> >>> > > ,
>> >>> > > >> >> > @Roman
>> >>> > > >> >> > > <
>> >>> > > >> >> > > > >>> > > > > > rshtykh@yahoo.com
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > ,
>> >>> > > >> >> > > > >>> > > > > > > > > as
>> >>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
>> >>> > > >> >> contributors/maintainers,
>> >>> > > >> >> > > > could
>> >>> > > >> >> > > > >>> you
>> >>> > > >> >> > > > >>> > > > please
>> >>> > > >> >> > > > >>> > > > > > > join
>> >>> > > >> >> > > > >>> > > > > > > > > this
>> >>> > > >> >> > > > >>> > > > > > > > > > thread?
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > -
>> >>> > > >> >> > > > >>> > > > > > > > > > Denis
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
>> >>> > Emmanouil
>> >>> > > >> >> > > > Gkatziouras <
>> >>> > > >> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
>> >>> > > >> >> > > > >>> > > > > > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately.
>> >>> Part
>> >>> > of
>> >>> > > my
>> >>> > > >> >> work
>> >>> > > >> >> > > > >>> involves
>> >>> > > >> >> > > > >>> > > > > Pub/Sub,
>> >>> > > >> >> > > > >>> > > > > > > > thus I
>> >>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
>> >>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to
>> Kafka.
>> >>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works.
>> >>> Every
>> >>> > > node
>> >>> > > >> >> > creates
>> >>> > > >> >> > > > an
>> >>> > > >> >> > > > >>> > > instance
>> >>> > > >> >> > > > >>> > > > > of
>> >>> > > >> >> > > > >>> > > > > > a
>> >>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
>> >>> > > >> >> > > > >>> > > > > > > > > > > client.
>> >>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the
>> same
>> >>> > topic
>> >>> > > >> >> using
>> >>> > > >> >> > the
>> >>> > > >> >> > > > >>> same
>> >>> > > >> >> > > > >>> > > > > > subscription
>> >>> > > >> >> > > > >>> > > > > > > > > name.
>> >>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and
>> one
>> >>> of
>> >>> > the
>> >>> > > >> nodes
>> >>> > > >> >> > will
>> >>> > > >> >> > > > >>> receive
>> >>> > > >> >> > > > >>> > > the
>> >>> > > >> >> > > > >>> > > > > > > > message,
>> >>> > > >> >> > > > >>> > > > > > > > > > > process it and then forward
>> it to
>> >>> > > Ignite
>> >>> > > >> >> using
>> >>> > > >> >> > > the
>> >>> > > >> >> > > > >>> > > extractor
>> >>> > > >> >> > > > >>> > > > > > > > provided.
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >> >
>> >>> > > >>
>> >>> > > >
>> >>> > >
>> >>> >
>> >>>
>> >>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message