ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valentin Kulichenko <valentin.kuliche...@gmail.com>
Subject Re: IP Filtering in IPFinders
Date Wed, 28 Apr 2021 18:49:10 GMT
I'm OK with the design.

Ilya, please feel free to merge if the implementation and tests look good
to you.

-Val

On Wed, Apr 28, 2021 at 1:07 AM Atri Sharma <atri@apache.org> wrote:

> Hi Ilya and Val,
>
> Thank you for taking a look and providing insights. I have updated the
> PR and raised another iteration.
>
> Val, I have moved the configuration to TcpDiscoverySpi.
>
> Please see and let me know your thoughts and comments.
>
> Regards,
>
> Atri
>
> On Wed, Apr 28, 2021 at 2:11 AM Valentin Kulichenko
> <valentin.kulichenko@gmail.com> wrote:
> >
> > Hi Atri,
> >
> > I've noticed that you added the property to the IgniteConfiguration, but
> > it's applied only within the discovery. I feel like something is wrong
> here.
> >
> > If this feature only relates to the discovery, then we should have the
> > configuration property on the TcpDiscoverySpi instead. Otherwise, the
> > filtering should be applied to all network components (although I'm not
> > necessarily sure what that would imply).
> >
> > What do you think?
> >
> > -Val
> >
> > On Tue, Apr 27, 2021 at 2:00 AM Atri Sharma <atri@apache.org> wrote:
> >
> > > Hi Val and Ilya,
> > >
> > > Thank you for taking the time to pursue this issue.
> > >
> > > I have raised a new PR for the discussed approach. Please see and let
> > > me know what you think:
> > >
> > > https://github.com/apache/ignite/pull/9048
> > >
> > > Regards,
> > >
> > > Atri
> > >
> > > On Thu, Apr 22, 2021 at 3:34 PM Ilya Kasnacheev
> > > <ilya.kasnacheev@gmail.com> wrote:
> > > >
> > > > Hello!
> > > >
> > > > I'm still not fully convinced, but Val's approach sounds rational to
> me.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > чт, 22 апр. 2021 г. в 12:45, Atri Sharma <atri@apache.org>:
> > > >
> > > > > Hello!
> > > > >
> > > > > I actually saw the shared container scenario being tried by
> somebody
> > > > > who wanted an external script to monitor all IPs being used by his
> > > > > clusters and hence thought of this idea. Another thing that came
in
> > > > > was the Firewall blocking a few IP addresses, hence the idea.
> > > > >
> > > > > I feel that the footprint of this change is small, and can be
> useful
> > > > > for esoteric use cases too without really interfering in any
> existing
> > > > > code path. Val's suggestion seems the right way to go since it
> gives
> > > > > the functionality without much change.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > On Thu, Apr 22, 2021 at 2:47 PM Ilya Kasnacheev
> > > > > <ilya.kasnacheev@gmail.com> wrote:
> > > > > >
> > > > > > Hello!
> > > > > >
> > > > > > AFAIK, a S3 container, Azure blob container, etc, is a relatively
> > > > > > lightweight entity, similar to a table in an SQL database. Why
> would
> > > > > > different clusters need to share the same discovery storage
> > > container?
> > > > > > When I tested Azure IP finder, it created several blob containers
> > > for me
> > > > > on
> > > > > > demand, based on the parameter passed to IP finder. If I wanted
> to
> > > have
> > > > > > more than one cluster it should have been seamless already.
> > > > > >
> > > > > > I can theoretically see how address filtering may be useful
to
> remove
> > > > > > public / private addresses or Docker gateway address, but it
is
> > > usually
> > > > > > handled by setting localHost setting, although requiring tuning
> it
> > > for
> > > > > each
> > > > > > instance individually. Overall benefit seems to small.
> > > > > >
> > > > > > This is why I am asking, do you have any specific scenario in
> mind
> > > where
> > > > > > this feature is an enabler? How did you arrive at the conclusion
> to
> > > go
> > > > > > forward with it?
> > > > > >
> > > > > > Regards,
> > > > > > --
> > > > > > Ilya Kasnacheev
> > > > > >
> > > > > >
> > > > > > чт, 22 апр. 2021 г. в 07:51, Atri Sharma <atri@apache.org>:
> > > > > >
> > > > > > > Hi Val,
> > > > > > >
> > > > > > > Consider a scenario where multiple Ignite clusters are
running
> and
> > > for
> > > > > > > operational ease (and also compliance, in some cases, e.g.
to
> make
> > > > > > > auditing easier), people can configure cloud based IP finders
> to
> > > share
> > > > > > > the same container (blob container in Azure, S3 container
in
> AWS
> > > etc).
> > > > > > >
> > > > > > > In such a case, IPs for all clusters will be in the same
> container.
> > > > > > > IPFinders of both the clusters will read the entire list.
In
> this
> > > > > > > case, address filtering will help ignore the irrelevant
IP
> > > addresses.
> > > > > > >
> > > > > > > Thank you for pointing me to the alternate direction. Let
me
> > > research
> > > > > > > that and revert.
> > > > > > >
> > > > > > > Atri
> > > > > > >
> > > > > > > On Wed, Apr 21, 2021 at 10:46 PM Valentin Kulichenko
> > > > > > > <valentin.kulichenko@gmail.com> wrote:
> > > > > > > >
> > > > > > > > Hi Atri,
> > > > > > > >
> > > > > > > > Can you describe the scenario in a little more detail?
What
> > > exactly
> > > > > do
> > > > > > > you
> > > > > > > > mean by a container shared by multiple clusters? What
are the
> > > > > > > consequences
> > > > > > > > of this? How does the proposed solution solve the
problem?
> > > > > > > >
> > > > > > > > Also, I would suggest revisiting the design - I'm
not sure
> such
> > > > > filtering
> > > > > > > > should be done on the IP finder level. Why not do
this on
> the SPI
> > > > > level
> > > > > > > > instead? I would simply add something like "addressFilter"
> to the
> > > > > > > > TcpDiscoverySpi. The filter can be a generic
> IgnitePredicate, so
> > > you
> > > > > will
> > > > > > > > be able to provide any implementations, including
regex or
> > > anything
> > > > > else.
> > > > > > > >
> > > > > > > > -Val
> > > > > > > >
> > > > > > > > On Wed, Apr 21, 2021 at 9:04 AM Atri Sharma <atri@apache.org
> >
> > > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > When a container is shared by multiple clusters,
then this
> can
> > > be
> > > > > > > useful
> > > > > > > > > for filtering IPs.
> > > > > > > > >
> > > > > > > > > Also, things like VPC based barriers can be circumvented
> using
> > > this
> > > > > > > > > technique.
> > > > > > > > >
> > > > > > > > > On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev, <
> > > > > ilya.kasnacheev@gmail.com
> > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hello!
> > > > > > > > > >
> > > > > > > > > > What are the expected use cases for this
feature? Can you
> > > please
> > > > > > > > > elaborate?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > --
> > > > > > > > > > Ilya Kasnacheev
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ср, 21 апр. 2021 г. в 08:23, Atri
Sharma <
> atri@apache.org>:
> > > > > > > > > >
> > > > > > > > > > > Hi All,
> > > > > > > > > > >
> > > > > > > > > > > I have opened the following JIRA for
the said topic:
> > > > > > > > > > >
> > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-14606
> > > > > > > > > > >
> > > > > > > > > > > The concept is to filter IPs based
on a pattern or a
> > > blocklist
> > > > > in
> > > > > > > > > > > IPFinders while consuming IPs. This
is more pertinent
> for
> > > cloud
> > > > > > > based
> > > > > > > > > > > IPFinders since they can have shared
containers.
> > > > > > > > > > >
> > > > > > > > > > > For the moment, I have implemented
regex based
> filtering:
> > > > > > > > > > >
> > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-14607
> > > > > > > > > > >
> > > > > > > > > > > for Azure Blob Storage IP Finder. Over
time, we can
> extend
> > > the
> > > > > > > same to
> > > > > > > > > > > other IP finders.
> > > > > > > > > > >
> > > > > > > > > > > Please see the PR:
> > > > > > > > > > >
> > > > > > > > > > > https://github.com/apache/ignite/pull/9024
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > >
> > > > > > > > > > > Atri
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Regards,
> > > > > > > > > > >
> > > > > > > > > > > Atri
> > > > > > > > > > > Apache Concerted
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Regards,
> > > > > > >
> > > > > > > Atri
> > > > > > > Apache Concerted
> > > > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Atri
> > > > > Apache Concerted
> > > > >
> > >
> > > --
> > > Regards,
> > >
> > > Atri
> > > Apache Concerted
> > >
>
> --
> Regards,
>
> Atri
> Apache Concerted
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message