cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wido den Hollander <w...@widodh.nl>
Subject Re: IPV6 in Isolated/VPC networks
Date Fri, 16 Jul 2021 12:17:04 GMT


Op 16-07-2021 om 05:54 schreef Hean Seng:
> Hi Wido,
> 
> My initial thought is not like this,  it is the /48 at ISP router, and /64
> subnet assign to AdvanceZoneVR,   AdvanceZoneVR responsible is
> distribule IPv6 ip (from the assigned /64 sunet) to VM,  and not routing
> the traffic,   in the VM that get the IPv6 IP will default route to ISP
> router as gw.   It can may be a bridge over via Advancezone-VR.
> 

How would you bridge this? That sounds like NAT?

IPv6 is meant to be routed. Not to be translated or bridged in any way.

The way a made the drawing is exactly how IPv6 should work in a VPC 
environment.

Traffic flows through the VR where it can do firewalling of the traffic.

> However, If do as the way described in the drawing, then i suppose will be
> another kind of virtual router going to introduce , to get hold the /48 in
> this virtual router right ?
> 

It can be the same VR. But keep in mind that IPv6 != IPv4.

The VR will get Frr as a new daemon which can talk BGP with the upper 
network to route traffic.

> After this,  The Advance Zone, NAT's  VR will peer with this new IPv6 VR
> for getting the IPv6 /64 prefix ?
> 

IPv4 will be behind NAT, but IPv6 will not be behind NAT.

> If do in this way, then I guess  you just only need Static route, with
> peering ip both end  as one /48 can have a lot of /64 on it.  And hardware
> budgeting for new IPv6-VR will become very important, as all traffic will
> need to pass over it .
> 

Routing or NAT is the same for the VR. You don't need a very beefy VR 
for this.

> It will be like
> 
> ISP Router  ------ >  (new IPV6-VR ) ---- > AdvanceZone-VR ----> VM
> 
> Relationship of (new IPv6 VR) and AdvanceZone-VR , may be considering on
> OSPF instead of  BGP , otherwise few thousand of AdvanceZone-VR wil have
> few thousand of BGP session. on new-IPv6-VR
> 
> Also, I suppose we cannot do ISP router. -->. Advancezone VR direct,   ,
> otherwise ISP router will be full of /64 prefix route either on BGP( Many
> BGP Session) , or  Many Static route .   If few thousand account, ti will
> be few thousand of BGP session with ISP router or few thousand static route
> which  is not possible .
> 
> 
> 
> 
> 
> 
> On Thu, Jul 15, 2021 at 10:47 PM Wido den Hollander <wido@widodh.nl> wrote:
> 
>> But you still need routing. See the attached PNG (and draw.io XML).
>>
>> You need to route the /48 subnet TO the VR which can then route it to
>> the Virtual Networks behind the VR.
>>
>> There is no other way then routing with either BGP or a Static route.
>>
>> Wido
>>
>> Op 15-07-2021 om 12:39 schreef Hean Seng:
>>> Or explain like this :
>>>
>>> 1) Cloudstack generate list of /64 subnet from /48 that Network admin
>>> assigned to Cloudstack
>>> 2) Cloudsack allocated the subnet (that generated from step1) to Virtual
>>> Router, one Virtual Router have one subniet /64
>>> 3) Virtual Router allocate single IPv6 (within the range of /64
>>> allocated to VR)  to VM
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Jul 15, 2021 at 6:25 PM Hean Seng <heanseng@gmail.com
>>> <mailto:heanseng@gmail.com>> wrote:
>>>
>>>      Hi Wido,
>>>
>>>      I think the /48 is at physical router as gateway , and subnet of /64
>>>      at VR of Cloudstack.   Cloudstack only keep which /48 prefix and
>>>      vlan information of this /48 to be later split the  /64. to VR.
>>>
>>>      And the instances is getting singe IPv6 of /64  IP.   The VR is
>>>      getting /64.  The default gateway shall goes to /48 of physical
>>>      router ip .   In this case ,does not need any BGP router .
>>>
>>>
>>>      Similar concept as IPv4 :
>>>
>>>      /48 subnet of IPv6 is equivalent to current /24 subnet of IPv4 that
>>>      created in Network.
>>>      and /64  of IPv6 is equivalent to single IP of IPv4 assign to VM.
>>>
>>>
>>>
>>>
>>>      On Thu, Jul 15, 2021 at 5:31 PM Wido den Hollander <wido@widodh.nl
>>>      <mailto:wido@widodh.nl>> wrote:
>>>
>>>
>>>
>>>          Op 14-07-2021 om 16:44 schreef Hean Seng:
>>>           > Hi
>>>           >
>>>           > I replied in another thread, i think do not need implement
>>>          BGP or OSPF,
>>>           > that would be complicated .
>>>           >
>>>           > We only need assign  IPv6 's /64 prefix to Virtual Router
>>>          (VR) in NAT
>>>           > zone, and the VR responsible to deliver single IPv6 to VM via
>>>          DHCP6.
>>>           >
>>>           > In VR, you need to have Default IPv6 route to  Physical
>>>          Router's /48. IP
>>>           > as IPv6 Gateway.  Thens should be done .
>>>           >
>>>           > Example :
>>>           > Physical Router Interface
>>>           >   IPv6 IP : 2000:aaaa::1/48
>>>           >
>>>           > Cloudstack  virtual router : 2000:aaaa:200:201::1/64 with
>>>          default ipv6
>>>           > route to router ip 2000:aaaa::1
>>>           > and Clodustack Virtual router dhcp allocate IP to VM , and
>>>          VM will have
>>>           > default route to VR. IPv6 2000:aaaa:200:201::1
>>>           >
>>>           > So in cloudstack need to allow  user to enter ,  IPv6
>>>          gwateway , and
>>>           > the  /48 Ipv6 prefix , then it will self allocate the /64 ip
>>>          to the VR ,
>>>           > and maintain make sure not ovelap allocation
>>>           >
>>>           >
>>>
>>>          But NAT is truly not the solution with IPv6. IPv6 is supposed to
>> be
>>>          routable. In addition you should avoid DHCPv6 as much as
>>>          possible as
>>>          that's not really the intended use-case for address allocation
>>>          with IPv6.
>>>
>>>          In order to route an /48 IPv6 subnet to the VR you have a few
>>>          possibilities:
>>>
>>>          - Static route from the upperlying routers which are outside of
>>>          CloudStack
>>>          - BGP
>>>          - OSPFv3 (broken in most cases!)
>>>          - DHCPv6 Prefix Delegation
>>>
>>>          BGP and/or Static routes are still the best bet here.
>>>
>>>          So what you do is that you tell CloudStack that you will route
>>>          2001:db8::/48 to the VR, the VR can then use that to split it up
>>>          into
>>>          multiple /64 subnets going towards the instances:
>>>
>>>          - 2001:db8::/64
>>>          - 2001:db8:1::/64
>>>          - 2001:db8:2::/64
>>>          ...
>>>          - 2001:db8:f::/64
>>>
>>>          And go on.
>>>
>>>          In case of BGP you indeed have to tell the VR a few things:
>>>
>>>          - It's own AS number
>>>          - The peer's address(es)
>>>
>>>          With FRR you can simply say:
>>>
>>>          neighbor 2001:db8:4fa::179 remote-as external
>>>
>>>          The /48 you need to have at the VR anyway in case of either a
>>>          static
>>>          route or BGP.
>>>
>>>          We just need to add a NullRoute on the VR for that /48 so that
>>>          traffic
>>>          will not be routed to the upper gateway in case of the VR can't
>>>          find a
>>>          route.
>>>
>>>          Wido
>>>
>>>           >
>>>           >
>>>           >
>>>           >
>>>           >
>>>           > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli
>>>           > <Alex.Mattioli@shapeblue.com
>>>          <mailto:Alex.Mattioli@shapeblue.com>
>>>          <mailto:Alex.Mattioli@shapeblue.com
>>>          <mailto:Alex.Mattioli@shapeblue.com>>> wrote:
>>>           >
>>>           >     Hi Wido,
>>>           >     That's pretty much in line with our thoughts, thanks for
>>>          the input.
>>>           >     I believe we agree on the following points then:
>>>           >
>>>           >     - FRR with BGP (no OSPF)
>>>           >     - Route /48 (or/56) down to the VR
>>>           >     - /64 per network
>>>           >     - SLACC for IP addressing
>>>           >
>>>           >     I believe the next big question is then "on which level
>>>          of ACS do we
>>>           >     manage AS numbers?".  I see two options:
>>>           >     1) Private AS number on a per-zone basis
>>>           >     2) Root Admin assigned AS number on a domain/account basis
>>>           >     3) End-user driven AS number on a per network basis (for
>>>          bring your
>>>           >     own AS and IP scenario)
>>>           >
>>>           >     Thoughts?
>>>           >
>>>           >     Cheers
>>>           >     Alex
>>>           >
>>>           >
>>>           >
>>>           >
>>>           >     -----Original Message-----
>>>           >     From: Wido den Hollander <wido@widodh.nl
>>>          <mailto:wido@widodh.nl> <mailto:wido@widodh.nl
>>>          <mailto:wido@widodh.nl>>>
>>>           >     Sent: 13 July 2021 15:08
>>>           >     To: dev@cloudstack.apache.org
>>>          <mailto:dev@cloudstack.apache.org>
>>>          <mailto:dev@cloudstack.apache.org
>>>          <mailto:dev@cloudstack.apache.org>>;
>>>           >     Alex Mattioli <Alex.Mattioli@shapeblue.com
>>>          <mailto:Alex.Mattioli@shapeblue.com>
>>>           >     <mailto:Alex.Mattioli@shapeblue.com
>>>          <mailto:Alex.Mattioli@shapeblue.com>>>
>>>           >     Cc: Wei Zhou <Wei.Zhou@shapeblue.com
>>>          <mailto:Wei.Zhou@shapeblue.com>
>>>           >     <mailto:Wei.Zhou@shapeblue.com
>>>          <mailto:Wei.Zhou@shapeblue.com>>>; Rohit Yadav
>>>           >     <rohit.yadav@shapeblue.com
>>>          <mailto:rohit.yadav@shapeblue.com>
>>>          <mailto:rohit.yadav@shapeblue.com
>>>          <mailto:rohit.yadav@shapeblue.com>>>;
>>>           >     Gabriel Beims Bräscher <gabriel@pcextreme.nl
>>>          <mailto:gabriel@pcextreme.nl>
>>>           >     <mailto:gabriel@pcextreme.nl <mailto:gabriel@pcextreme.nl
>>>>>
>>>           >     Subject: Re: IPV6 in Isolated/VPC networks
>>>           >
>>>           >
>>>           >
>>>           >     On 7/7/21 1:16 PM, Alex Mattioli wrote:
>>>           >      > Hi all,
>>>           >      > @Wei Zhou<mailto:Wei.Zhou@shapeblue.com
>>>          <mailto:Wei.Zhou@shapeblue.com>
>>>           >     <mailto:Wei.Zhou@shapeblue.com
>>>          <mailto:Wei.Zhou@shapeblue.com>>> @Rohit
>>>           >     Yadav<mailto:rohit.yadav@shapeblue.com
>>>          <mailto:rohit.yadav@shapeblue.com>
>>>           >     <mailto:rohit.yadav@shapeblue.com
>>>          <mailto:rohit.yadav@shapeblue.com>>> and myself are
>>>          investigating how
>>>           >     to enable IPV6 support on Isolated and VPC networks and
>>>          would like
>>>           >     your input on it.
>>>           >      > At the moment we are looking at implementing FRR with
>>>          BGP (and
>>>           >     possibly OSPF) on the ACS VR.
>>>           >      >
>>>           >      > We are looking for requirements, recommendations,
>>>          ideas, rants,
>>>           >     etc...etc...
>>>           >      >
>>>           >
>>>           >     Ok! Here we go.
>>>           >
>>>           >     I think that you mean that the VR will actually route the
>>>          IPv6
>>>           >     traffic and for that you need to have a way of getting a
>>>          subnet
>>>           >     routed to the VR.
>>>           >
>>>           >     BGP is probably you best bet here. Although OSPFv3
>>>          technically
>>>           >     supports this it is very badly implemented in Frr for
>>>          example.
>>>           >
>>>           >     Now FRR is a very good router and one of the fancy
>>>          features it
>>>           >     supports is BGP Unnumered. This allows for auto
>>>          configuration of BGP
>>>           >     over a L2 network when both sides are sending Router
>>>          Advertisements.
>>>           >     This is very easy for flexible BGP configurations where
>>>          both sides
>>>           >     have dynamic IPs.
>>>           >
>>>           >     What you want to do is that you get a /56, /48 or
>>>          something which is
>>>           >      >/64 bits routed to the VR.
>>>           >
>>>           >     Now you can sub-segment this into separate /64 subnets.
>>>          You don't
>>>           >     want to go smaller then a /64 is that prevents you from
>>>          using SLAAC
>>>           >     for IPv6 address configuration. This is how it works for
>>>          Shared
>>>           >     Networks now in Basic and Advanced Zones.
>>>           >
>>>           >     FRR can now also send out the Router Advertisements on
>>>          the downlinks
>>>           >     sending out:
>>>           >
>>>           >     - DNS servers
>>>           >     - DNS domain
>>>           >     - Prefix (/64) to be used
>>>           >
>>>           >     There is no need for DHCPv6. You can calculate the IPv6
>>>          address the
>>>           >     VM will obtain by using the MAC and the prefix.
>>>           >
>>>           >     So in short:
>>>           >
>>>           >     - Using BGP you routed a /48 to the VR
>>>           >     - Now you split this into /64 subnets towards the
>>>          isolated networks
>>>           >
>>>           >     Wido
>>>           >
>>>           >      > Alex Mattioli
>>>           >      >
>>>           >      >
>>>           >      >
>>>           >      >
>>>           >
>>>           >
>>>           >
>>>           > --
>>>           > Regards,
>>>           > Hean Seng
>>>
>>>
>>>
>>>      --
>>>      Regards,
>>>      Hean Seng
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Hean Seng
> 
> 
> 

Mime
View raw message