EVPN-Multihoming over MPLS w/IP Infusion’s OcNOS to netElastic’s vBNG for subscriber management

Categories
IP Infusion netElastic vBNG

EVPN-Multihoming over MPLS w/IP Infusion’s OcNOS to netElastic’s vBNG for subscriber management

Having highly available layer-2 uplinks in a service provider has been around for a long time and there have been various ways to do it over the years, multi-chassis lag, stacking, active passive pseudowires, and more. If you use traditional VPLS and dual home a customer edge router you have to account for loop prevention.

We’re going to take a look at another way to handle this. EVPN has been used for multihoming in various environments for years. You might be familiar with EVPN/VxLAN in datacenters and dual attaching servers to an MLAG pair. Well there is another option – EVPN Multihoming – which uses ethernet segment identifiers to multi-home CEs. Think of this like MLAG but in BGP instead and since we’re looking at service providers we’ll use a MPLS dataplane instead of VxLAN.

Some of the advantages:

  • Easy Multihoming
  • Active/Active forwarding
  • High Availability
  • Resilient layer-2 overlay
  • one control plane – BGP

This design is going to utilize a netElastic vBNG and IP Infusion‘s OcNOS on a mix of edgecore and ufispace platforms to run ISIS-SR w/EVPN on top for layer-2 services.

The service provider core and access network utilizes ISIS-SR for MPLS services. If you need a refresher on Segment Routing there are several other articles on this blog or at stubarea51.net.

IPI-1 and IPI-2 are aggregation routers that aggregate last mile circuits which could be fiber, fixed wireless, or various other technologies. Then EVPN VPLS is overlayed on top of the MPLS network to deliver a layer-2 service from the vBNG to the subscriber equipment, in our case a mikrotik rb4011.

The vBNG then handles subscriber management. It takes the DHCP request from the rb4011 and asks the radius server if this subscriber is authorized. If the subscriber is authorized it hands out an IP address to the client.

The ufispace-q2c, a 9600-72xc, is utilized as an EVPN route reflector. In production IPA often recommends an out of path router reflector running on FRR. Lets verify that our control plane setup.

9600-72xc-1#show bgp l2vpn evpn summary
BGP router identifier 100.127.0.0, local AS number 1016
BGP table version is 10
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd     AD  MACIP  MCAST    ESI  PREFIX-ROUTE
100.127.0.2              4  1016 38460      38496      10      0      0  01w4d09h               3      0      2      1      0      0
100.127.0.3              4  1016 38471      38490      10      0      0  01w4d09h               2      0      1      1      0      0
100.127.0.7              4  1016  312        314      10      0      0  02:11:38               2      0      1      1      0      0

Total number of neighbors 3

Total number of Established sessions 3
9600-72xc-1#show run router bgp
router bgp 1016
 neighbor 100.127.0.2 remote-as 1016
 neighbor 100.127.0.3 remote-as 1016
 neighbor 100.127.0.7 remote-as 1016
 neighbor 100.127.0.2 update-source lo
 neighbor 100.127.0.3 update-source lo
 neighbor 100.127.0.7 update-source lo
 !
 address-family l2vpn evpn
 neighbor 100.127.0.2 activate
 neighbor 100.127.0.2 route-reflector-client
 neighbor 100.127.0.3 activate
 neighbor 100.127.0.3 route-reflector-client
 neighbor 100.127.0.7 activate
 neighbor 100.127.0.7 route-reflector-client
 exit-address-family
 !

The route-reflector clients are all configured the same way.

ipi-1.lab.jan1.us.ipa.net#show bgp l2vpn evpn summary 
BGP router identifier 100.127.0.2, local AS number 1016
BGP table version is 7
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd     AD  MACIP  MCAST    ESI  PREFIX-ROUTE
100.127.0.0              4  1016 38501      38466       7      0      0  01w4d10h               4      0      2      2      0      0

Total number of neighbors 1

Total number of Established sessions 1

ipi-2.lab.jan1.us.ipa.net#show bgp l2vpn evpn summary 
BGP router identifier 100.127.0.3, local AS number 1016
BGP table version is 9
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd     AD  MACIP  MCAST    ESI  PREFIX-ROUTE
100.127.0.0              4  1016 38494      38476       9      0      0  01w4d10h               5      0      3      2      0      0

Total number of neighbors 1

Total number of Established sessions 1
5916-54XKS#show run router bgp
!
router bgp 1016
 neighbor 100.127.0.0 remote-as 1016
 neighbor 100.127.0.0 update-source lo
 !
 address-family l2vpn evpn
 neighbor 100.127.0.0 activate
 exit-address-family
!

Now that the BGP signaling is up and verified we will build the EVPN VPLS and EVPN-Multihoming (EVPN-MH) interface.

5916-54XKS#
!
hardware-profile filter evpn-mpls-mh enable
hardware-profile statistics ingress-acl enable
!
evpn mpls enable
!
evpn mpls multihoming enable
!
mac vrf ORANGE
 rd 100.127.0.7:1
 route-target both evpn-auto-rt
!
evpn mpls vtep-ip-global 100.127.0.7
!
evpn mpls id 1
 host-reachability-protocol evpn-bgp ORANGE
!
interface xe44.10 switchport
 encapsulation dot1q 10-50
 access-if-evpn
  arp-cache disable
  nd-cache disable
  map vpn-id 1
!

Some of the general setup on the OcNOS devices is the same. You first need to enable EVPN to utilize the MPLS dataplane. IMPORTANT this requires a reboot! Make sure you set the VTEP IP as the loopback and then build the mac vrf for the layer2 vpn service and tie it to an EVPN id. Finally to utilize the service you attach it to a switchport as seen on xe44.10 switchport. In this case everything with a vlan id of 10-50 is placed into EVPN vpn-id 1.

5916-54XKS#show evpn mpls 
EVPN-MPLS Information 
================= 
   Codes: NW - Network Port 
          AC - Access Port 
         (u) - Untagged 

VPN-ID   EVI-Name     EVI-Type Type Interface ESI                           VLAN      DF-Status Src-Addr         Dst-Addr        
_______________________________________________________________________________________________________________________________
1        ----         L2       NW   ----      ----                          ----      ----      100.127.0.7      100.127.0.3     
1        ----         L2       NW   ----      ----                          ----      ----      100.127.0.7      100.127.0.2     
1        ----         --       AC   xe44.10   --- Single Homed Port ---     ----      ----      ----             ----            

5916-54XKS#show evpn mpls mac-table 
=========================================================================================================================================
                                                     EVPN MPLS MAC Entries 
=========================================================================================================================================
VNID       Interface VlanId    In-VlanId Mac-Addr       VTEP-Ip/ESI                    Type            Status     MAC move AccessPortDesc
_________________________________________________________________________________________________________________________________________

1          xe44.10   ----      ----      2cc8.1b80.cebb 100.127.0.7                    Dynamic Local   -------    0        -------   
1          ----      ----      ----      2cc8.1b80.cebc 100.127.0.3                    Dynamic Remote  -------    0        -------   
1          ----      ----      ----      5254.0006.eb8f 00:00:00:5e:00:00:01:00:00:00  Dynamic Remote  -------    0        -------   
1          ----      ----      ----      c4ad.34f4.b81d 100.127.0.2                    Dynamic Remote  -------    0        -------   

Total number of entries are : 4
5916-54XKS#show bgp l2vpn evpn 
BGP table version is 8, local router ID is 100.127.0.7
Status codes: s suppressed, d damped, h history, a add-path, * valid, > best, i - internal,
              l - labeled, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

[EVPN route type]:[ESI]:[VNID]:[relevent route informantion]
1 - Ethernet Auto-discovery Route
2 - MAC/IP Route
3 - Inclusive Multicast Route
4 - Ethernet Segment Route
5 - Prefix Route

    Network          Next Hop            Metric    LocPrf       Weight     Path  Peer          Encap

RD[100.127.0.1:2]
*>i  [2]:[00:00:00:5e:00:00:01:00:00:00]:[1]:[48,5254:0006:eb8f]:[0]:[640]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      
*>i  [2]:[0]:[1]:[48,c4ad:34f4:b81d]:[0]:[640]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      
*>i  [3]:[1]:[32,100.127.0.2]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      

RD[100.127.0.3:1]
*>i  [2]:[0]:[1]:[48,2cc8:1b80:cebc]:[0]:[640]
                       100.127.0.3          0        100       0    i  100.127.0.0     MPLS      
*>i  [3]:[1]:[32,100.127.0.3]
                       100.127.0.3          0        100       0    i  100.127.0.0     MPLS      

RD[100.127.0.7:1] VRF[ORANGE]:
*>   [2]:[0]:[1]:[48,2cc8:1b80:cebb]:[0]:[640]
                       100.127.0.7          0        100       32768  i  ----------      MPLS      
* i  [2]:[0]:[1]:[48,2cc8:1b80:cebc]:[0]:[640]
                       100.127.0.3          0        100       0    i  100.127.0.0     MPLS      
* i  [2]:[00:00:00:5e:00:00:01:00:00:00]:[1]:[48,5254:0006:eb8f]:[0]:[640]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      
* i  [2]:[0]:[1]:[48,c4ad:34f4:b81d]:[0]:[640]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      
* i  [3]:[1]:[32,100.127.0.2]
                       100.127.0.2          0        100       0    i  100.127.0.0     MPLS      
* i  [3]:[1]:[32,100.127.0.3]
                       100.127.0.3          0        100       0    i  100.127.0.0     MPLS      
*>   [3]:[1]:[32,100.127.0.7]
                       100.127.0.7          0        100       32768  i  ----------      MPLS      

The multihoming side takes a little additional configuration for the access interface.

ipi-1.lab.jan1.us.ipa.net#show run int xe10
!
interface xe10
 channel-group 1 mode active
!
ipi-1.lab.jan1.us.ipa.net#show run int po1
!
interface po1
 evpn multi-homed system-mac 0000.5e00.0001
!
ipi-1.lab.jan1.us.ipa.net#show run int po1.532
!
interface po1.532 switchport
 encapsulation dot1q 532 inner-dot1q 10-50
 rewrite pop
 access-if-evpn
  arp-cache disable
  nd-cache disable
  map vpn-id 1
!
ipi-2.lab.jan1.us.ipa.net#show run int xe10
!
interface xe10
 channel-group 1 mode active
!
ipi-2.lab.jan1.us.ipa.net#show run int po1
!
interface po1
 evpn multi-homed system-mac 0000.5e00.0001
!
ipi-2.lab.jan1.us.ipa.net#show run int po1.532
!
interface po1.532 switchport
 encapsulation dot1q 532 inner-dot1q 10-50
 rewrite pop
 access-if-evpn
  arp-cache disable
  nd-cache disable
  map vpn-id 1
!

On each PE we need to build the portchannel. This port-channel needs a system-mac defined for the multihoming. Then the criteria to place traffic into the EVPN-MH VPN needs to be defined. In this case it matches doubled tagged traffic. It’s not uncommon on SP networks to utilize double tagged traffic. So we add the outer tag and remove it at the attachment point. This is done with rewrite pop which removes or adds the outer tag.

netelastic-1# show running-config interface 
interface 10gei-1/1/0
exit
interface 10gei-1/1/1
exit
interface 10gei-1/1/2
exit
interface 10gei-1/1/2.100
 ipv4 address 100.124.0.10 29
 dot1q 100
exit
interface 10gei-1/1/3
 mtu 9216
exit
interface 10gei-1/1/3.100
 ipv4 address 100.124.0.2 29
 dot1q 100
exit
interface eth-trunk1
 trunk-mode lacp-static
 trunk-port 10gei-1/1/0
  lacp port-priority 32768
  lacp timeout  long
 exit
 trunk-port 10gei-1/1/1
  lacp port-priority 32768
  lacp timeout  long
 exit
exit
interface eth-trunk1.532
 qinq internal 11 external 532
exit

The netElastic box has a stand LACP channel built and then matches the double tag on the interface defined. This interface will be mapped to BRAS for authentication prior to assigning an IP address.

First we need to build radius profiles so we can utilize them later.

netelastic-1# show running-config radius 
radius vendor-id       54268
radius accounting-on   enable
radius attribute-usermac-as mac
radius framed-route-delimiter ";"
radius username-override disable
radius authentication group RADIUS_AUTH
 server-type    ipv4-server
 timeout        3
 retry-times    3
 nas-ip-address 100.127.0.6
 master-server  1
 algorithm      master
 dead-time      5
 dead-count     10
 class-as-car   disable
 filter-id-type user-acl
 server 1 ipv4-address 192.168.0.2 port 1812 key netelastic
exit
radius accounting group RADIUS_ACCT
 server-type       ipv4-server
 timeout           3
 retry-times       3
 nas-ip-address    100.127.0.6
 algorithm         master
 dead-time         5
 dead-count        10
 flow-unit         byte
 align-authenreply disable
 server 1 ipv4-address 192.168.0.2 port 1813 key netelastic
exit
radius dmcoa group
 server-type ipv4-server
 server 1 ipv4-address 192.168.0.2 key netelastic

Then we need to build a DHCP server and IP Pool on the netElastic box.

netelastic-1# show running-config ippool
ippool group TEST
 gateway-ip 192.168.11.1 gateway-mask 255.255.255.0
 lease-time        3600
 dns-primary 8.8.8.8
 ippool-status     unlock
 warning-threshold 80
 warning-exhaust   disable
 frame-ip lease manage disable
 section start-ip 192.168.11.2 end-ip 192.168.11.254
 exit
exit
netelastic-1# show running-config dhcp  
dhcp
 dhcp enable
 relay max-user 128000
 relay option82 policy     keep
 relay option82 format     china-tel
 relay option82 user-configuration-policy interface
 interface eth-trunk1.532
  mode       server
  user-quota 32000
 exit
exit

Finally, we tie it all together with the BRAS configuration.

netelastic-1# show run bras
bras
 l2tp alive-check tcp-port 0
 access username-check disable
 access password-check disable
 domain-name-delimiter      @
 domain-location            after-delimiter
 domainname-parse-direction left-to-right
 authentication RADIUS_AUTH
  authentication-type         radius
  radius-authentication-group RADIUS_AUTH
  user-name-format            strip-domain
  nas-port-format             class1
  called-station-id-format    class1
  nas-port-id-format class1
  calling-station-id-format class1
  mac-format mac-format-type lower mac-delimiter :
  invalid-vlan-tag            0
 exit
 accounting RADIUS_ACCT
  accounting-type               radius
  accounting-update             600
  first-radius-accounting-group RADIUS_ACCT
  accounting-start-fail         online
  accounting-update-fail online
  accounting-update-immediately disable
  l2tp-accounting               vpdn-model
  user-name-format              strip-domain
  nas-port-format               class1
  called-station-id-format      class1
  nas-port-id-format class1
  calling-station-id-format class1
  mac-format mac-format-type lower mac-delimiter :
  invalid-vlan-tag              0
 exit
 authorization TEST_AUTH
  authorization-type mix-radius
  bind nat-domain-name cgnat_rules
  nat-type           inside
  radius-nat-switch  disable
 exit
 domain IPA_DOMAIN
  bind authentication-template RADIUS_AUTH
  bind accounting-template RADIUS_ACCT
  bind authorization-template TEST_AUTH
  vgi                          vgi1
  domain-status                unlock
  user-routing-distribute      disable
  user-ipv6-routing-distribute disable
  tunnel-domain                disable
  flow-statistic               enable
  radius-attribute qos-acl-profile no-exist-policy offline
  radius-attribute qos-profile no-exist-avp online
  quota-out offline
  bind-pool 1 TEST
 exit
 ipoe template IPOE_TEST
  authentication-type ipv4 dhcpv4 option
  authentication-type ipv6 dhcpv6 option
  dhcp-v4 auth-on-up password-type config config-password 123456
  dhcp-v4 auth-on-up username-type mac
  dhcp-v4 auth-on-up domain-type pre-domain
  dhcp-v6 auth-on-up password-type config config-password 123456
  dhcp-v6 auth-on-up username-type mac
  dhcp-v6 auth-on-up domain-type option
 exit
 subscriber-manage
  user-group-identify circuit
  offline-speed 1000
  session abnormal-offline-record 32000
  session normal-offline-record 32000
  session online-fail-record 32000
 exit
 vgi-configuration
  interface vgi1
  exit
 exit
 vci-configuration
  interface eth-trunk1.532
   ipoe template IPOE_TEST
   max-ipox-session  32000
   max-pppox-session 32000
   encapsulation     multi
   pre-domain        IPA_DOMAIN
   ip-access-type    ipv4
  exit
 exit
exit

This is a very simple setup where the CPE, the rb4011, is authorized by it’s mac address. There are more options. We also are not going to assign speed plans, etc…

Prior to this working end to end we also need to build the freeradius server. This setup uses daloRADIUS. I won’t go into details on the freeradius initial build here.

The rb4011 is setup to request a DHCP address on vlan 11.

When this request hits the netElastic server over the EVPN L2VPN, netElastic will ask the radius server if the user is valid.

14:43:55.817264 IP 100.127.0.6.64001 > 192.168.0.2.radius: RADIUS, Access-Request (1), id: 0x8b length: 202
14:43:55.825564 IP 192.168.0.2.radius > 100.127.0.6.64001: RADIUS, Access-Accept (2), id: 0x8b length: 20
14:43:56.077261 IP 100.127.0.6.64011 > 192.168.0.2.radius-acct: RADIUS, Accounting-Request (4), id: 0x02 length: 119
14:43:56.077270 IP 100.127.0.6.64011 > 192.168.0.2.radius-acct: RADIUS, Accounting-Request (4), id: 0x03 length: 245
14:43:56.083223 IP 192.168.0.2.radius-acct > 100.127.0.6.64011: RADIUS, Accounting-Response (5), id: 0x02 length: 20
14:43:59.437271 IP 100.127.0.6.64011 > 192.168.0.2.radius-acct: RADIUS, Accounting-Request (4), id: 0x03 length: 245

If the RADIUS server authenticates the user, which we defined with the mac address earlier, then the rb4011 will get an IP address. You can see the approval in the messages above and vlan11 has an IP address on the previous screen shot.

This lab does not have internet access but we’ll add a static route on the rb4011 to verify reachability through netElastic to the rest of the lab loopbacks.

The 3 lost packet seen in this screenshot were during the failure of IPI-1. Shutting po1 on IPI-1 forced traffic through IPI-2.

Conclusion

EVPN-MH is a great technology to uplink layer-2 services to a BNG for subscriber management and achieve high availability. This concept can continue to be built on by adding a second BNG for BNG resiliency and EVPN-MH on the remote side for high profile sites. We did not cover various other authentication methods and assigning speed packages to the subscriber but these all build off the radius server and tie back to the vBNG operation. netElastic also offers an optional CGNAT engine which can be added to this setup as well as DCHPv6.

Categories
IP Infusion

A simplified – cost effective – way to deliver IPTV with IP Infusion

Multicast used to be the defacto standard for delivering IPTV and a lot of legacy IPTV systems are still fully based on multicast or require sending local channels via multicast to a large provider to get the stream back for unicast delivery to clients. This has only been sped up by home users cancelling traditional cable and moving to things like roku, hulu live, apple tv, and then insert your streaming service of choice here.

In many networks, it is no longer advantageous or necessary to use complex mLDP and MVPN to deliver multicast traffic given the larger capacity of the links and reduced multicast traffic.

We’ve put in a lot of IPTV solutions and almost always outside of the content providers themselves, a centralized headend for delivery works well. This means building traditional VPLS tunnels back to the video source. It is simple, works well, and easy to maintain in comparison to mLDP, P2MP, and MVPN. Granted it does not scale as well as mLDP/MVPN but most tier 2 and tier 3 providers are not operating at a scale where the limits are met.

Additionally, to get these protocols it often means expensive hardware to deliver a complex solution.

Lets explore a solution put together with whitebox hardware; edgecore and ufispace running IP Infusions OcNOS SP 6.0.2.

IGP and MPLS Topology

This example utilizes LDP signaled VPLS running on top of ISIS-SR.

VPLS Topology

We’re running LDP signaled VPLS for service delivery. The latest version of OcNOS has also fixed the dependency of having to run LDP on an interface to operate LDP VPLS over SR-MPLS. This allows for easier overall configuration with less overhead in the cli.

We are going to statically join the group 239.0.0.1 on the multicast receivers; QFX-2, Mikrotik-1, and Mikrotik-2. Then we will send from the source 192.168.0.2 to group 239.0.0.1 utilizing iperf and verify receipt of traffic.

First lets look at the VPLS tunnels and verify they are up. We aren’t going to verify on every router.

9510-28DC-2#show run mpls
!
mpls vpls LDP 1
 signaling ldp
  vpls-peer 100.127.0.3
  vpls-peer 100.127.0.4
  exit-signaling
 exit-vpls
!
router ldp
 router-id 100.127.0.5
 targeted-peer ipv4 100.127.0.3
  exit-targeted-peer-mode
 targeted-peer ipv4 100.127.0.4
  exit-targeted-peer-mode
9510-28DC-2#show mpls vpls detail
Virtual Private LAN Service Instance: LDP, ID: 1
 SIG-Protocol: LDP
 Attachment-Circuit :UP
 Learning: Enabled
 Control-Word: Disabled
 Flow Label Status: Disabled, Direction: None, Static: No
 Group ID: 0, VPLS Type: Ethernet, Configured MTU: 1500
 Description: none
 service-tpid: dot1.q
 Operating mode: Raw
 Configured interfaces:
  Interface: xe27.100
 Subinterface Match Criteria(s) :
  dot1q 100
  Interface: ce2.100
 Subinterface Match Criteria(s) :
  dot1q 100
 Mesh Peers:
   100.127.0.3 (Up)
   100.127.0.4 (Up)
9510-28DC-2#show mpls ldp session
Peer IP Address           IF Name    My Role    State      KeepAlive  UpTime
100.127.0.4               ce2        Active    OPERATIONAL   30    3d23h31m
100.127.0.3               ce3        Active    OPERATIONAL   30    3d23h32m
9510-28DC-2#show mpls vpls mac-address name LDP
MAC address       Learned from   Vlan-Id  Peer address   Time-out
000c.42b2.a63c    ce2            -                100.127.0.4    300
4c5e.0c23.df4e    xe27.100
84c1.c132.5032    ce3            -                100.127.0.3    300

So now we can see that targeted sessions up, attachment circuit up, and local/remote mac addresses learned across the circuit. Lets move on to multicast verification.

First we’ll verify that routing over OSPF is up and reachability then PIM.

Here is the RP and the PIM DR for the routers connecting over the VPLS tunnels.

and for people that like the CLI.

[email protected]# run show pim neighbors
B = Bidirectional Capable, G = Generation Identifier
H = Hello Option Holdtime, L = Hello Option LAN Prune Delay,
P = Hello Option DR Priority, T = Tracking Bit,
A = Hello Option Join Attribute

Instance: PIM.master
Interface           IP V Mode        Option       Uptime Neighbor addr
xe-0/0/47.100        4 2             HPLGT       05:04:56 172.16.0.1
xe-0/0/47.100        4 2             HPLGT       05:03:43 172.16.0.2

{master:0}[edit]
[email protected]# run show ospf neighbor
Address          Interface              State           ID               Pri  Dead
172.16.0.2       xe-0/0/47.100          Full            172.16.0.2       128    35
172.16.0.1       xe-0/0/47.100          Full            192.168.0.1      128    35

Lets verify that we got the s,g entry and the multicast traffic on the receivers. We will look at the torch output on mikrotik-2 to see the inbound traffic.

[email protected]# run show multicast route
Instance: master Family: INET

Group: 239.0.0.1
    Source: 192.168.0.2/32
    Upstream interface: xe-0/0/47.100

As seen above we have the appropriate entries for the group and inbound traffic.

Conclusion

It is possible and often better to deploy a simple solution using MPLS and VPLS overlayed on top for multicast delivery. This is simple to deploy, troubleshoot, and maintain. This same principle can be extended with EVPN for a layer 2 overlay with either MPLS or VxLAN data plane. This extensibility makes the solution future proof and easy to migrate as the underlying service or transport changes.

We’ve worked with ISPs on IPTV deployments using L2VPN into the tens of thousands of receivers with the correct planning considerations.

Please look for future blog posts for scaling multicast in large networks or working with multicast in a bandwidth-constrained environment.

Categories
IP Infusion network operating systems

WEBINAR: Validated MX204 Alternatives: IP Infusion OcNOS

We recently did a webinar on alternatives to Juniper Network’s MX204 platform. While the MX204 was a fully capable router it was End Of Life’d and then recently the EOL was revoked after approximately a year. During the period before the end of life revocation operators searched for various alternatives.

IP Infusion’s OcNOS rose as one of the prime contenders as a replacement after they announced support for newer silicon, Qumran2 from broadcom, allowing them to enter the market as a capable border and peering router.

We went through some of the supported platforms, support systems, and whitebox ecosystem in this webinar.

Find a recording here: https://iparchitechs.com/presentations/2023-IPInfusion-MX204-Webinar/IPInfusion-MX204-Webinar.mp4

Find the slides here: https://iparchitechs.com/presentations/2023-IPInfusion-MX204-Webinar/IP-Infusion-MX204-Webinar.pdf

Categories
Security Field Day Tech Field Day

Cribl: a refreshing approach to product – XFD8

Cribl presented at Security Field Day 8 and while they look to have an awesome product for observability we’re going to look at something else, a refreshing way to provide training, resources, and product information.

It became immediately apparent during their presentation that their core values actually meant something:

  • customers first always
  • irreverent, but serious
  • curious
  • transparent
  • together

What does Cribl do?

Without getting into the product to much, I’ll do this after I review their free resource (more on this later), this picture gives a great overview.

The data is separated and enriched to allow for the right data to go to the right place. That is to say, the marshmallows, things people care about, are separated and put into the analytics tool. While everything is stored to meet compliance requirements.

Resources

The thing that really got me excited about Cribl was their approach to resources, training, and demos.

There are still some vendors that give free training and discounted certification, Juniper’s program comes to mind, but not to the extent here.

One of the presenters made a comment about how easy it was to get started and you didn’t need accounts/wouldn’t be pestered by sales the instant you signed up. This sounded too good to be true so I went over to their website to see for myself. I was able to get setup on their sandbox just by putting my email in for a verification code to make sure I wasn’t a bot. It’s been several days and I still haven’t gotten any sales spam.

There is also a free training and certification program. That was just as easy to get started with.

If that wasn’t enough you can download the full product suite with 1TB/day of data. How’s that for a test drive? It also was extremely easy to follow their licensing model and determine what you get.

Conclusion

It is refreshing to see a company take such a customer first approach. It was easy to get started, learn, and evaluate the software. All without having to go through a drawn out sales process we’ve become all too accustom too in technology. I didn’t have to go to a fancy dinner or jump through hoops to get a demo. I just went to the website and got all the resources I needed to start. More to come as I work through the training materials.

Categories
IP Infusion MikroTik Uncategorized

IP Infusion OcNOS 6.0: Tag Operations

One of the biggest misconceptions I had before moving into the service provider space was that all layer 2 operations had been replaced with layer 3. I quickly found out that even if you are routing everything there are still a lot of layer 2 overlays in place, carrier ethernet, or even spanning-tree (STP). Running these technologies, hopefully not STP, is especially important rural broadband, electric co-ops, and telcos to enable things such as IP conservation and subscriber management on a broadband network gateway.

The side effect of layer 2 transport technologies being so heavily used in last mile internet service providers is having to understand vlan tagging operations. The are occasions where one side of the circuit will be double tagged and the other side will be single tagged or everything will be double tagged/single tagged/untagged. Let’s take a quickly look at some simple tag operations on IP Infusions OcNOS SP 6.0.

IGP and MPLS setup

We setup a simple topology of isis and ldp to create VPWS circuits. We are going to build two circuits in this example.

ipi-1.lab.jan1.us.ipa.net#show run mpls
<<output snipped>>
mpls l2-circuit TAG-TEST 123 100.127.0.2
!
mpls l2-circuit TEST-TAG-4 1234 100.127.0.2
!
<<output snipped>>
router ldp
 router-id 100.127.0.1
 targeted-peer ipv4 100.127.0.2
  exit-targeted-peer-mode

VPWS and Attachment Circuit Topology

We are going to attach the circuits to the same physical ports and utilize tags to place the traffic into the correct VPWS circuit. Mikrotik-1 is double tagged and Mikrotik-2 is single tagged.

If you’ve built pseudowires on OcNOS before you might be familiar with the service-template model. However, when building VPWS circuits and utilizing the same interface this isn’t an option. You have to make a subinterface that is a switchport. This also changes the method for tag operations.

ipi-1.lab.jan1.us.ipa.net#show run interface

<<output snipped>>

interface xe2
 switchport
!
interface xe2.3 switchport
 encapsulation dot1q 3 inner-dot1q 400-499
 rewrite pop
 access-if-vpws
  mpls-l2-circuit TAG-TEST primary
!
interface xe2.4 switchport
 encapsulation dot1q 4 inner-dot1q 400-499
 rewrite pop
 access-if-vpws
  mpls-l2-circuit TEST-TAG-4 primary
!

The switchport subinterfaces are then assigned as an access interface matching specific tags. In this case outer tag 3 and inner tag 400-499. On IPI-2 we are expecting single tagged frames instead of double tagged. This is where the rewrite pop comes in. This will remove the outer tag on ingress to the PW and push it back on on egress towards Mikrotik-1.

ipi-2.lab.jan1.us.ipa.net#show run interface

<<output snipped>>

interface xe11
 speed 1g
 switchport
!
interface xe11.4 switchport
 encapsulation dot1q 401
 access-if-vpws
  mpls-l2-circuit TEST-TAG-4 primary
!
interface xe11.400 switchport
 encapsulation dot1q 400
 access-if-vpws
  mpls-l2-circuit TEST-TAG primary
!

Since the AC towards Mikrotik-2 is only expecting single tagged packets we just do a simple match on encapsulation.

Let’s do some verification. First we’ll verify that the circuits are up.

ipi-2.lab.jan1.us.ipa.net#show ldp targeted-peers
IP Address          Interface
100.127.0.1         xe48

ipi-2.lab.jan1.us.ipa.net#show ldp mpls-l2-circuit
Transport     Client     VC     VC            Local       Remote      Destination
VC ID         Binding    State  Type          VC Label    VC Label    Address
123           xe11.400   UP     Ethernet VLAN 25602       26240       100.127.0.1
1234          xe11.4     UP     Ethernet VLAN 25603       26241       100.127.0.1

ipi-2.lab.jan1.us.ipa.net#show ldp mpls-l2-circuit detail
PW ID: 123, VC state is up
Access IF: xe11.400,up,AC state is up
Session IF: xe48, state is up
Destination: 100.127.0.1, Peer LDP Ident: 100.127.0.1
Local vctype: vlan, remote vctype :vlan
Local groupid: 0, remote groupid: 0
Local label: 25602, remote label: 26240
Local MTU: 1500, Remote MTU: 1500
Local Control Word: disabled  Remote Control Word: Not-Applicable  Current use: disabled
Local Flow Label Direction: Disabled, Static: Disabled
Remote Flow Label Direction: Both , Static: Disabled
Local PW Status Capability : disabled
Remote PW Status Capability : disabled
Current PW Status TLV : disabled

PW ID: 1234, VC state is up
Access IF: xe11.4,up,AC state is up
Session IF: xe48, state is up
Destination: 100.127.0.1, Peer LDP Ident: 100.127.0.1
Local vctype: vlan, remote vctype :vlan
Local groupid: 0, remote groupid: 0
Local label: 25603, remote label: 26241
Local MTU: 1500, Remote MTU: 1500
Local Control Word: disabled  Remote Control Word: Not-Applicable  Current use: disabled
Local Flow Label Direction: Disabled, Static: Disabled
Remote Flow Label Direction: Disabled, Static: Disabled
Local PW Status Capability : disabled
Remote PW Status Capability : disabled
Current PW Status TLV : disabled 

Then let’s verify that we can pass traffic.

Mikrotik-1 Interfaces
Mikrotik-1 ping results
Mikrotik-2 Interfaces

We have successfully manipulated the tags and passed traffic end to end. We started with a double tagged packet, pop off the outer tag, placed it into a PW, spit out a single tagged packet, and had end to end reachability.

Conclusion

There are various methods and configurations to manipulate traffic. This is one of the more common examples of tag manipulation that occurs in broadband aggregation. I will explore tag manipulation with service-templates and VPLS in a later post.

Categories
calix Interop IP Infusion

IP Infusion OcNOS 6.0: interop VPWS and VPLS

With the current state of the supply chain lead times for networking gear can be astronomical. This led consumers to look at other options for networking equipment forcing the whitebox and disaggregated networking market to become more prevalent.

With full featured operating systems like IP Infusion‘s OcNOS 6.0 and commodity hardware from Ufispace and Edgecore companies have been about to upgrade faster and further than ever before.

We’ll be looking at the ufispace 9600-32s and 9500-30xs in this deployment. This is shaping up to be a great combination for 100g and 10g density. Since both run the same operating system moving between is easy. While the bigger Qumran2c, 9600-32s, doesn’t support breakouts/10g we can aggregate and terminate 100g services here while using a small device to delivery 10g density and breakout.

We’re going to look specifically at VPLS and VPWS delivery in this deployment. Since these deployments typically complement existing deployments we’ll look at interop with a Calix e9-2 ASM 3001 deployment.

I know Calix doesn’t normally come to mind for MPLS deployments but more for FTTX or ERPS. However, they’ve been putting a lot of effort into their MPLS stack on the e9-2 ASM platform which has helped led to this testing.

We also have an Arista 7280CR3K-32P4 acting as a p-router during link failure.

IGP/LDP Setup

We’re going to run isis as an IGP which is typical in a service provider network. This time we’re going to run straight LDP instead of SR-MPLS, however, you can still utilize SR-MPLS with a mapping server if your topology supports it.

Let’s verify IGP/LDP and routing.

ASM3001# show isis neighbors
NEIGHBOR                                HOLD  CIRCUIT
SYSTEM ID       TYPE  INTERFACE  STATE  TIME  ID
-------------------------------------------------------
0010.0127.0118  L2    la3        UP     23    3
0010.0127.0119  L2    la4        UP     26    4

 details
NEIGHBOR                                HOLD  CIRCUIT
SYSTEM ID       TYPE  INTERFACE  STATE  TIME  ID
-------------------------------------------------------
0010.0127.0118  L2    la3	 UP	23    3
Hostname:ufispace-100
SNPA:e8c5.7a77.a655
State Changed:3214
LAN Priority:0
Restart Capable:1
Peer Restart State:1
0010.0127.0119  L2    la4	 UP	26    4
Hostname:ARISTA
SNPA:c4ca.2b66.fb6d
State Changed:3152
LAN Priority:0
Restart Capable:1
Peer Restart State:1
-------------------------------------------------------
ASM3001# show mpls ldp neighbors
                                                                          LOOP                      INTERFACE
INDEX  PEER LDP ID      LOCAL LDP ID     TYPE      SESSION DISTMODE       DETECTION  TRANS ADD      NAME
---------------------------------------------------------------------------------------------------------------
1      100.127.0.118:0  100.127.0.117:0  TARGETED  DownstreamUnsolicited  Disabled   100.127.0.119  none
2      100.127.0.118:0  100.127.0.117:0  DIRECTED  DownstreamUnsolicited  Disabled   100.127.0.119  la3
3      100.127.0.119:0  100.127.0.117:0  DIRECTED  DownstreamUnsolicited  Disabled   100.127.0.119  la4
ASM3001# show ip route all
ROUTE
INDEX  PREFIX            NEXT HOP       TYPE   DISTANCE  INTERFACE  UPTIME
----------------------------------------------------------------------------
1      100.126.2.160/29  100.126.2.161  local  0/0       la3        0:9:47
2      100.126.2.161/32  0.0.0.0        local  0/0       la3        0:9:47
3      100.126.2.168/29  100.126.2.169  local  0/0       la4        0:9:42
4      100.126.2.169/32  0.0.0.0        local  0/0       la4        0:9:42
5      100.126.2.176/29  100.126.2.162  isis   115/20    la3        0:9:33
6                        100.126.2.170  isis   115/20    la4        0:9:33
7      100.126.2.184/29  100.126.2.162  isis   115/20    la3        0:9:33
8      100.127.0.117/32  0.0.0.0        local  0/0       loopback1  0:9:53
9      100.127.0.118/32  100.126.2.162  isis   115/20    la3        0:9:33
10     100.127.0.119/32  100.126.2.170  isis   115/20    la4        0:9:33
11     100.127.0.120/32  100.126.2.162  isis   115/30    la3        0:9:33
ASM3001# ping 100.127.0.120
PING 100.127.0.120 (100.127.0.120) 56(84) bytes of data.
64 bytes from 100.127.0.120: icmp_seq=1 ttl=63 time=0.778 ms
64 bytes from 100.127.0.120: icmp_seq=2 ttl=63 time=0.722 ms
OcNOS-SW#show clns neighbors

Total number of L1 adjacencies: 0
Total number of L2 adjacencies: 1
Total number of adjacencies: 1
Tag UNDERLAY:  VRF : default
System Id      Interface   SNPA                State  Holdtime  Type Protocol
ufispace-100   po1         e8c5.7a77.a657      Up     26        L2   IS-IS
OcNOS-SW#show mpls ldp neighbor
IP Address      Mode          Intf Name    Holdtime   LDP-Identifier
100.126.2.185   Interface     po1         15         100.127.0.118:0
fe80::eac5:7aff:fe77:a657Interface     po1         15         100.127.0.118:0
100.127.0.117   Targeted      po1         45         100.127.0.117:0
OcNOS-SW#ping 100.127.0.117 source-ip 100.127.0.120
Press CTRL+C to exit
PING 100.127.0.117 (100.127.0.117) from 100.127.0.120 : 56(84) bytes of data.
64 bytes from 100.127.0.117: icmp_seq=1 ttl=64 time=0.811 ms
64 bytes from 100.127.0.117: icmp_seq=2 ttl=64 time=0.746 ms

Since we have LDP neighbors and loopback to loopback reachability we can begin to build our services.

100g VPWS

First we’ll build a VPWS service between the E9-2 and ufispace-100 to verify functionality. We’ll utilize a TX300s-100GX test set to push traffic through the service.

First lets look at the ASM configuration for the xconnect.

ASM3001# show running-config l2vpn
l2vpn 1
 pw-class PWE-1
  encapsulation mpls
   cc-type        TTL
   transport-mode vlan
  !
 !
 point2point 200
  xconnect-neighbor 100.127.0.118 pw-id 200
   pw-class  PWE-1
   pw-status enable
  !
 !
!

Below you can see the config for the interface facing the test set. This will put the traffic into the VPWS service.

ASM3001# show running-config interface ethernet 1/2/q7
interface ethernet 1/2/q7
 no shutdown
 role     uni
 arp arp-announce any
 l2transport
  point-to-point 200
 !
!

Now we can see the same on the IP infusion side.

ufispace-100#show run mpls
!
service-template TEST
 match all
!
mpls l2-circuit TEST-VPWS 200 100.127.0.117
ufispace-100#show run int ce17
!
interface ce17
 switchport
 mtu 1986
 mpls-l2-circuit TEST-VPWS service-template TEST primary
!

Finally lets verify functionality. I did a verbose output of the circuit details to help see all of the details. Some important things to match are the MTU and if it’s a vlan or raw service.

IP Infusion sets the MTU on the attachment circuit while Calix is inherited from the default interface value of 2000 minus some overhead.

ufispace-100#show ldp mpls-l2-circuit detail
PW ID: 200, VC state is up
Access IF: ce17,up,AC state is up
Session IF: po1, state is up
Destination: 100.127.0.117, Peer LDP Ident: 100.127.0.117
Local vctype: vlan, remote vctype :vlan
Local groupid: 0, remote groupid: 0
Local label: 24962, remote label: 26
Local MTU: 1986, Remote MTU: 1986
Local Control Word: disabled  Remote Control Word: Not-Applicable  Current use: disabled
Local Flow Label Direction: Disabled, Static: Disabled
Remote Flow Label Direction: Disabled, Static: Disabled
Local PW Status Capability : disabled
Remote PW Status Capability : enabled
Current PW Status TLV : disabled
Local VCCV Capability:
 CC-Types: None
 CV-Types: None
Remote VCCV Capability:
 CC-Types:  Type 3
 CV-Types:
 LSP ping
ASM3001# show l2vpn xconnect pw-id 200
l2vpn xconnect pw-id 200



XCONNECT NAME                     STATE
--------------------------------- ---------------
200                               Up
-------------------------------------------------

VPWS Index : 2
VPN Key    : 131074

% 1 entries in the table.


AC Details
-------------------------------------------------

INTERFACE    VLAN      STATE           TYPE       MTU       VPWS-INDEX
------------ --------- --------------- ---------- --------- -----------
1/2/q7       NA        Active          Tagged     1986      2

% 1 entries in the table.


PW Details
-------------------------------------------------

PW-ID  PW-STATE          PW-CLASS              ENCAPSULATION  PROTOCOL  ADMIN-STATE  REDUNDANCY-STATE  VPWS-INDEX
------ ----------------- --------------------- -------------- --------- ------------ ----------------- ----------
200    Up                 PWE-1                 MPLS           LDP       Up           NA                2
-----------------------------------------------------------------------------------------------------------------

PW-INFO       LOCAL                REMOTE
------------- -------------------- --------------------
Address       100.127.0.117        100.127.0.118
PW ID         200                  uNknOwn
PW type       Tagged               uNknoWn
Label         26                   24962
MTU           1986                 1986
Control Word  Disabled             uNknOwn
Status TLV    Enabled              Disabled
CC Type       4                    0
CV Type       2                    0

Local Status (PW Status TLV): 0x6
Remote Status (PW Status TLV): 0x0
Create time: 2022-09-10 09:17:23
Last time status changed: 2022-09-10 09:30:28


% 1 entries in the table.

Finally, we can see 95g of traffic across the circuit with the test set.

10g VPLS

Next we will look at a 10g VPLS service delivered off the extension switch. We already saw end to end reachability in the IGP setup so we will start with configuration.

On the ASM you build a bridge domain and tie it to a virtual forwarding instance.

ASM3001# show running-config l2vpn
l2vpn 1
 pw-class vlan-pwe
  encapsulation mpls
   cc-type        TTL
   transport-mode vlan
  !
 !

Then we define the neighbor or neighbors in the VPLS.

ASM3001# show running-config l2vpn bridge-domain
l2vpn 1
 bridge-domain 220
  mtu 9086
  vfi 220
   neighbor 100.127.0.120 pw-id 220
    pw-class vlan-pwe
   !
  !
 !
!

Again, we tie the interface facing the test kit into the bridge-domain. This will put the traffic into the VPLS instance.

ASM3001# show running-config interface ethernet 1/1/x15
interface ethernet 1/1/x15
 no shutdown
 role     uni
 arp arp-announce any
 l2transport
  rewrite-ingress tag add dot1q 220
  !
  bridge-domain 220
  !
 !
!

Then we build the same on IP Infusion.

OcNOS-SW#show run vpls
!
mpls vpls TEST-VPLS 220
 signaling ldp
  vpls-type vlan
  vpls-peer 100.127.0.117
  exit-signaling
 exit-vpls
!

Here we also have to define the peers for targeted hellos in LDP.

OcNOS-SW#show run ldp
!
router ldp
 router-id 100.127.0.120
 graceful-restart full
 targeted-peer ipv4 100.127.0.117
  exit-targeted-peer-mode
 transport-address ipv4 100.127.0.120
!

Finally, we attached the a port to the service and plug in the test kit.

OcNOS-SW#show run int xe15
!
interface xe15
 switchport
 mtu 9086
 mpls-vpls TEST-VPLS service-template TEST
  exit-if-vpls
!

Again, we will look at the verbose output and pay attention to MTU and VPLS type, vlan in this case.

OcNOS-SW#show mpls vpls detail
Virtual Private LAN Service Instance: TEST-VPLS, ID: 220
 SIG-Protocol: LDP
 Attachment-Circuit :UP
 Learning: Enabled
 Control-Word: Disabled
 Flow Label Status: Disabled, Direction: None, Static: No
 Group ID: 0, VPLS Type: Ethernet VLAN, Configured MTU: 9086
 Description: none
 service-tpid: dot1.q
 Operating mode: Tagged
 Svlan Id: 0
 Svlan Tpid: 8100
 Configured interfaces:
  Interface: xe15
 Service-template : TEST
  Match criteria : Accept all

 Mesh Peers:
   100.127.0.117 (Up)
ASM3001# show l2vpn bridge-domain bd-name 220
l2vpn bridge-domain bd-name 220

BRIDGE DOMAIN NAME                     STATE
--------------------------------- ---------------
220                               Up
-------------------------------------------------

VPLS Index         :  3
VPN Key            :  65539
MTU                :  9086
MAC Learning       :  ENABLE
MAC Aging Time     :  300
MAC Limit Max      :  1024
MAC Action         :  FLOOD
AD Type            :  NONE
SIG type           :  NONE
Transport Mode     :  ETHERNET TAGGED
Control Word       :  DISABLE
Route Distinguisher:  0x0000000000000000(NULL)
VPLS ID            :  0x0000000000000000(DEFAULT)
VE ID              :  0
VE Range           :  8
% 1 entries in the table.
AC Details
------------ --------------- ---------- ------------ ---------------
DESCRIPTION  STATE           TYPE        VPLS INDEX   SPLIT HORIZON
------------ --------------- ---------- ------------ ---------------
1.x15        Active          Ethernet    3            Disabled
% 1 entries in the table.
PW Details
PW-ID           STATE       PW-Class              ENCAPSULATION  VPLS-INDEX  ADMIN-STATE
--------------- ----------- --------------------- -------------- ----------  ------------
220             Up          vlan-pwe              MPLS           3         Up
------------------------------------------------------------------------------------

------------- -------------------- --------------------

PW            LOCAL                REMOTE
------------- -------------------- --------------------
Address       100.127.0.117        100.127.0.120
PW ID         220                  uNknOwn
PW type       Tagged               uNknoWn
Label         34                   24961
MTU           9086                 9086
Control Word  Disabled             uNknOwn
Status TLV    Enabled              Disabled
CC Type       4                    0
CV Type       2                    0

Create time: 2022-09-10 09:55:06
Last time status changed: 2022-09-10 09:58:56


% 1 entries in the table.

Finally, we can see all of the traffic on the test set across the circuit.

Conclusion

Disaggregated networking provides an alternative to traditional vendors and these are real world examples of service deployment for service providers. A special thanks to Sorin Esanu and Race Communications for organizing this test environment as a proof of concept for their deployment.

Categories
Interop IP Infusion

IP Infusion 6.0: L3VPN and 6VPE interop with Juniper

A while back we published a blog on ip infusion‘s OcNOS and MikroTik interop with segment routing mpls and LDP. Today we are going to build on the fundamentals learned there to test VPNv4 and VPNv6 interoperability with Juniper.

We’re going to start with the same topology from the previous post but with juniper instead of mikrotik. Of course juniper could run isis end to end to eliminate the redistribution but we’re going to continue to use OSPF and LDP. OSPF and LDP are largely deployed and it is likely you will come across this scenario in your path towards SR-MPLS.

MPLS/IGP Setup

The first thing to accomplish is end to end reachability between the provider edge (PE) routers.

MPLS only requires the /32s of the loopbacks for functionality so redistribution is limited to the /32 loopbacks of the PE routers. If we ran isis end to end we would not have to perform this redistribution.

ipi-2.lab.jan1.us.ipa.net#
ip prefix-list LDP-PE-LOOPBACKS
 seq 10 permit 100.127.2.0/24 eq 32
!
ip prefix-list SR-PE-LOOPBACKS
 seq 10 permit 100.127.0.0/24 eq 32
!
route-map REDIS-OSPF-TO-ISIS permit 10
 match ip address prefix-list LDP-PE-LOOPBACKS
!
route-map REDIS-ISIS-TO-OSPF permit 10
 match ip address prefix-list SR-PE-LOOPBACKS
!
router isis UNDERLAY
 is-type level-1-2
 metric-style wide
 mpls traffic-eng router-id 100.127.0.2
 mpls traffic-eng level-1
 mpls traffic-eng level-2
 capability cspf
 dynamic-hostname
 fast-reroute ti-lfa level-1 proto ipv4
 fast-reroute ti-lfa level-2 proto ipv4
 net 49.0015.1001.2700.0002.00
 redistribute ospf level-1-2 route-map REDIS-OSPF-TO-ISIS
 segment-routing mpls
!
router ospf
 ospf router-id 100.127.0.2
 redistribute isis UNDERLAY route-map REDIS-ISIS-TO-OSPF
 passive-interface lo enable
 network 100.126.0.0/29 area 0.0.0.0
 network 100.126.0.8/29 area 0.0.0.0
 network 100.127.0.2/32 area 0.0.0.0
!
ipi-1.lab.jan1.us.ipa.net#ping 100.127.2.2 source-ip 100.127.0.1
Press CTRL+C to exit
PING 100.127.2.2 (100.127.2.2) from 100.127.0.1 : 56(84) bytes of data.
64 bytes from 100.127.2.2: icmp_seq=1 ttl=63 time=6.30 ms
64 bytes from 100.127.2.2: icmp_seq=2 ttl=63 time=2.09 ms
64 bytes from 100.127.2.2: icmp_seq=3 ttl=63 time=5.83 ms

Now that we have loopback to loopback reachability we will stitch together the LDP and segment routing (SR) domains.

IPI-1 is going to server as a segment routing mapping server. This will assign labels to the routes in the LDP label space and distribute them to through the SR domain so we can have an end to end label switched path enabling the use of MPLS services.

segment-routing
 mpls sr-prefer
 global block 16000 23999
 mapping-server
  srms preference 100
  prefix-sid-map address-family ipv4
   100.127.2.0/32 4000 range 256
  exit-ms-af
 exit-ms

This will start with prefix 100.127.2.0/32 add 4000 to the segment routing global block starting point (16000 as defined) and be able to label the next 256 routes in order. i.e. 100.127.2.1/32 gets the node sid 20001. IPI-2 shows the stitching in action.

Let’s look at the label space.

ipi-2.lab.jan1.us.ipa.net#show mpls ilm-table
Codes: > - installed ILM, * - selected ILM, p - stale ILM
        K - CLI ILM, T - MPLS-TP, s - Stitched ILM
       S - SNMP, L - LDP, R - RSVP, C - CRLDP
       B - BGP , K - CLI , V - LDP_VC, I - IGP_SHORTCUT
       O - OSPF/OSPF6 SR, i - ISIS SR, k - SR CLI
       P - SR Policy, U - unknown

Code    FEC/VRF/L2CKT    ILM-ID      In-Label    Out-Label   In-Intf    Out-Intf
/VRF       Nexthop                   LSP-Type
   i>   100.127.0.1/32     4           16101       3           N/A        xe48
           100.126.0.1               LSP_DEFAULT
   B>   evpn:1             3           17          Nolabel     N/A        N/A
           127.0.0.1                 LSP_DEFAULT
   B>   evpn:2             1           16          Nolabel     N/A        N/A
           127.0.0.1                 LSP_DEFAULT
   B>   evpn:1             2           640         Nolabel     N/A        N/A
           127.0.0.1                 LSP_DEFAULT
   L>   100.127.0.1/32     8           24961       3           N/A        xe48
           100.126.0.1               LSP_DEFAULT
 s i>   100.127.2.2/32     9           20002       3           N/A        xe47.1
           100.126.0.10              LSP_DEFAULT
   i>   100.127.0.2/32     5           16102       Nolabel     N/A        N/A
           127.0.0.1                 LSP_DEFAULT
   i>   100.126.0.1/32     6           25600       3           N/A        xe48
           100.126.0.1               LSP_DEFAULT
   L>   100.127.2.2/32     10          24962       3           N/A        xe47.1
           100.126.0.10              LSP_DEFAULT

The highlighted label above shows the stitching in action as denoted by the s. IMPORTANT mpls lsp-stitching needs to be enabled on any router in the SR and LDP domain.

IPI-1 only sees isis-sr labels and QFX-2 only sees LDP labels as shown below.

ipi-1.lab.jan1.us.ipa.net#show mpls ilm-table
Codes: > - installed ILM, * - selected ILM, p - stale ILM
        K - CLI ILM, T - MPLS-TP, s - Stitched ILM
       S - SNMP, L - LDP, R - RSVP, C - CRLDP
       B - BGP , K - CLI , V - LDP_VC, I - IGP_SHORTCUT
       O - OSPF/OSPF6 SR, i - ISIS SR, k - SR CLI
       P - SR Policy, U - unknown

Code    FEC/VRF/L2CKT    ILM-ID      In-Label    Out-Label   In-Intf    Out-Intf
/VRF       Nexthop                   LSP-Type
   i>   100.127.0.1/32     4           16101       Nolabel     N/A        N/A
           127.0.0.1                 LSP_DEFAULT
   B>   evpn:1             3           17          Nolabel     N/A        N/A
           127.0.0.1                 LSP_DEFAULT
   B>   evpn:100           1           16          Nolabel     N/A        N/A
           127.0.0.1                 LSP_DEFAULT
   B>   evpn:1             2           640         Nolabel     N/A        N/A
           127.0.0.1                 LSP_DEFAULT
   P>   100.127.0.2/32     7           20          3           N/A        xe48
           100.126.0.2               LSP_DEFAULT
   i>   100.127.2.2/32     8           20002       20002       N/A        xe48
           100.126.0.2               LSP_DEFAULT
   i>   100.127.0.2/32     6           16102       3           N/A        xe48
           100.126.0.2               LSP_DEFAULT
   i>   100.126.0.2/32     5           26240       3           N/A        xe48
           100.126.0.2               LSP_DEFAULT
   B>   VOICE              9           24960       Nolabel     N/A        VOICE
           N/A                       LSP_DEFAULT
[email protected]# run show route table inet.3

inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

100.126.0.0/29     *[LDP/9] 01:14:11, metric 1
                    >  to 100.126.0.9 via xe-0/0/47.0
100.127.0.1/32     *[LDP/9] 01:14:11, metric 1
                    >  to 100.126.0.9 via xe-0/0/47.0, Push 24961
100.127.0.2/32     *[LDP/9] 01:14:11, metric 1
                    >  to 100.126.0.9 via xe-0/0/47.0

L3VPN

It is common to place a voice services into a layer 3 vpn. So first we’ll setup a VPNv4 session. Then build the vrf and test end to end reachability.

ipi-1.lab.jan1.us.ipa.net

ip vrf VOICE
 rd 100.127.0.1:12
 route-target both 65000:12
!
interface xe46
 speed 10g
 ip vrf forwarding VOICE
 ip address 172.16.0.1/24
 ipv6 address 2001:db8::1/64
!
router bgp 65000
 neighbor 100.127.2.2 remote-as 65000
 neighbor 100.127.2.2 update-source lo
 !
 address-family vpnv4 unicast
 neighbor 100.127.2.2 activate
 exit-address-family
!
 address-family ipv4 vrf VOICE
 redistribute connected
 exit-address-family
 !
[email protected]# show routing-instances
VOICE {
    instance-type vrf;
    interface xe-0/0/47.10;
    interface xe-0/0/47.100;
    route-distinguisher 100.127.2.2:1;
    vrf-target target:65000:12;
    vrf-table-label;
}
[email protected]# show interfaces xe-0/0/47
vlan-tagging;
unit 0 {
    vlan-id 1;
    family inet {
        address 100.126.0.10/29;
    }
    family iso;
    family mpls;
}
unit 10 {
    vlan-id 10;
    family inet {
        address 172.16.2.1/24;
    }
    family inet6 {
        address 2001:db8:0:1::1/64;
    }
}
[email protected]# show protocols bgp
group VPN {
    family inet-vpn {
        unicast;
    }
    family inet6-vpn {
        unicast;
    }
    export PS-DIRECT;
    peer-as 65000;
    neighbor 100.127.0.1;
}
local-address 100.127.2.2;
local-as 65000;

We are building the vrf VOICE with route-target import and export 65000:12 for membership.

ipi-1.lab.jan1.us.ipa.net#show ip bgp vpnv4 all summary
BGP router identifier 100.127.0.1, local AS number 65000
BGP table version is 2
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd
100.127.2.2              4 65000  368        354       2      0      0  02:09:07               2


Total number of neighbors 1

Total number of Established sessions 1
[email protected]# run show bgp summary
Threading mode: BGP I/O
Groups: 1 Peers: 1 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
bgp.l3vpn.0
                       1          1          0          0          0          0
bgp.l3vpn-inet6.0
                       1          1          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
100.127.0.1           65000        319        313       0       7     2:14:50 Establ
  bgp.l3vpn.0: 1/1/1/0
  bgp.l3vpn-inet6.0: 1/1/1/0
  VOICE.inet.0: 1/1/1/0
  VOICE.inet6.0: 1/1/1/0
ipi-1.lab.jan1.us.ipa.net#show ip bgp vrf VOICE
BGP table version is 1, local router ID is 172.16.0.1
Status codes: s suppressed, d damped, h history, a add-path, * valid, > best, i - internal,
              l - labeled, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

    Network          Next Hop            Metric    LocPrf   Weight Path
*> l 172.16.0.0/24    0.0.0.0              0        100       32768  ?
*>i  172.16.2.0/24    100.127.2.2          0        100       0    i
*>i  192.168.0.0      100.127.2.2          0        100       0    i

Total number of prefixes 3
[email protected]# run show route table VOICE.inet.0

VOICE.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.0.0/24      *[BGP/170] 01:35:07, localpref 100, from 100.127.0.1
                      AS path: ?, validation-state: unverified
                    >  to 100.126.0.9 via xe-0/0/47.0, Push 24960, Push 24961(top)
172.16.2.0/24      *[Direct/0] 02:08:32
                    >  via xe-0/0/47.10
172.16.2.1/32      *[Local/0] 02:08:32
                       Local via xe-0/0/47.10
192.168.0.0/24     *[Direct/0] 1w0d 00:03:04
                    >  via xe-0/0/47.100
192.168.0.2/32     *[Local/0] 1w0d 00:03:04
                       Local via xe-0/0/47.100

We can see that we are receiving three prefixes from QFX-2 and reachability is confirmed below.

ipi-1.lab.jan1.us.ipa.net#ping 172.16.2.1 vrf VOICE
Press CTRL+C to exit
PING 172.16.2.1 (172.16.2.1) 56(84) bytes of data.
64 bytes from 172.16.2.1: icmp_seq=1 ttl=64 time=3.02 ms
64 bytes from 172.16.2.1: icmp_seq=2 ttl=64 time=6.87 ms

6VPE

6VPE is a great way to take advantage of your already deployed and operational MPLS core. Here we’re going to use the sr-mpls/LDP setup to build an IPv6 service on top, again for vrf VOICE.

Most of the configuration elements are the same except for utilizing VPNv6 and IPv6 addressing. Let’s look specifically at those portions.

ipi-1.lab.jan1.us.ipa.net#show run bgp
!
router bgp 65000
 neighbor 100.127.2.2 remote-as 65000
 neighbor 100.127.2.2 update-source lo
 !
 address-family vpnv6 unicast
 neighbor 100.127.2.2 activate
 exit-address-family
 !
 address-family ipv6 vrf VOICE
 redistribute connected
 exit-address-family
!
[email protected]# show protocols bgp
group VPN {
    family inet-vpn {
        unicast;
    }
    family inet6-vpn {
        unicast;
    }
    export PS-DIRECT;
    peer-as 65000;
    neighbor 100.127.0.1;
}
local-address 100.127.2.2;
local-as 65000;
[email protected]# show protocols mpls
ipv6-tunneling;

I will draw attention to the ipv6-tunneling command on QFX-2 above. This is required to tunnel the IPv6 packets over the MPLS core. Without this you will end up with an unusable next hop.

ipi-1.lab.jan1.us.ipa.net#show ip bgp vpnv6 all summary
BGP router identifier 100.127.0.1, local AS number 65000
BGP table version is 2
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd
100.127.2.2              4 65000  393        380       2      0      0  02:20:07               1

Total number of neighbors 1

Total number of Established sessions 1
[email protected]# run show bgp summary
Threading mode: BGP I/O
Groups: 1 Peers: 1 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
bgp.l3vpn.0
                       1          1          0          0          0          0
bgp.l3vpn-inet6.0
                       1          1          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
100.127.0.1           65000        319        313       0       7     2:14:50 Establ
  bgp.l3vpn.0: 1/1/1/0
  bgp.l3vpn-inet6.0: 1/1/1/0
  VOICE.inet.0: 1/1/1/0
  VOICE.inet6.0: 1/1/1/0
ipi-1.lab.jan1.us.ipa.net#show ip bgp vpnv6 vrf VOICE
Status codes: s suppressed, d damped, h history, a add-path, * valid, > best, i - internal, l - lab
eled
              S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

     Network          Next Hop                    Metric    LocPrf      Weight       Path
Route Distinguisher: 100.127.0.1:12 (Default for VRF VOICE)
*> l 2001:db8::/64    ::                            0        100       32768  ?
*>i  2001:db8:0:1::/64
                      ::ffff:100.127.2.2            0        100       0    i
 Announced routes count = 1
 Accepted routes count = 1
[email protected]# run show route table VOICE.inet6.0

VOICE.inet6.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2001:db8::/64      *[BGP/170] 01:43:15, localpref 100, from 100.127.0.1
                      AS path: ?, validation-state: unverified
                    >  to 100.126.0.9 via xe-0/0/47.0, Push 24960, Push 24961(top)
2001:db8:0:1::/64  *[Direct/0] 01:49:56
                    >  via xe-0/0/47.10
2001:db8:0:1::1/128*[Local/0] 01:49:56
                       Local via xe-0/0/47.10
fe80::86c1:c100:a32:5032/128
                   *[Local/0] 01:49:56
                       Local via xe-0/0/47.10
ff02::2/128        *[INET6/0] 1w2d 20:22:10
                       MultiRecv

{master:0}[edit]

Now that we have all over our routes and signaling setup lets verify reachability.

ipi-1.lab.jan1.us.ipa.net#ping ipv6 2001:db8:0:1::1 vrf VOICE
Press CTRL+C to exit
PING 2001:db8:0:1::1(2001:db8:0:1::1) 56 data bytes
64 bytes from 2001:db8:0:1::1: icmp_seq=1 ttl=64 time=2.80 ms
64 bytes from 2001:db8:0:1::1: icmp_seq=2 ttl=64 time=3.84 ms

Conclusion

With the new features and support in IP Infusion GA 6.0 stability and interop between vendors is becoming better. This paves the way for deployments of segment-routing while migrating the LDP segments all while maintaining current VPNs.

The 6VPE support allows for rapid adoption of IPv6 utilizing the core already in place.

We’ll be working on more interop testing to include L2VPN with both LDP signaled VPLS and BGP signaled VPLS. Be sure to come back for more.

Categories
Uncategorized

IP Infusion: EVPN-MPLS first look on GA 6.0

IP Infusion just released OcNOS version 6.0 and the release notes, as well as press release, show a focus on EVPN with an MPLS data plane. Don’t forget EVPN and VxLAN aren’t mutually exclusive, EVPN runs on and was originally designed for a MPLS data plane. I recently discussed this on a podcast EVPN doesn’t need VxLAN if you want to know more on that topic.

Lets take a look at basic EVPN-VPWS and EVPN-VPLS deployment. Since we’re looking at an MPLS data plane we will utilize ISIS-SR for MPLS. We’re utilizing ISIS-SR as it is increasingly replacing LDP and RSVP-TE for label distribution.

IGP and Label Distribution

First let’s look at the IGP setup and label distribution as everything else will be built on top of this.

ipi-1.lab.jan1.us.ipa.net#show run int lo
interface lo
 ip address 127.0.0.1/8
 ip address 100.127.0.1/32 secondary
 ipv6 address ::1/128
 ipv6 address 2001:db8::1/128
 prefix-sid index 101
 ip router isis UNDERLAY
 ipv6 router isis UNDERLAY
!

We have to set an index to create the node-sid for this device. In this case we use 101.

ipi-1.lab.jan1.us.ipa.net#show run segment-routing
segment-routing
 mpls sr-prefer
 global block 16000 23999

Since our segment routing global block starts at 16000 the node-sid becomes 16101 as the index + the start of the SRGB defines the sid. Additionally, we run mpls sr-prefer as this will prefer SR labels over LDP or RSVP-TE labels.

ipi-1.lab.jan1.us.ipa.net#show run isis
router isis UNDERLAY
 is-type level-1-2
 metric-style wide
 mpls traffic-eng router-id 100.127.0.1
 mpls traffic-eng level-1
 mpls traffic-eng level-2
 capability cspf
 dynamic-hostname
 fast-reroute ti-lfa level-1 proto ipv4
 fast-reroute ti-lfa level-2 proto ipv4
 net 49.0015.1001.2700.0001.00
 segment-routing mpls
!

Finally, we have to enable ISIS for segment routing.

ipi-1.lab.jan1.us.ipa.net#show clns neighbors

Total number of L1 adjacencies: 1
Total number of L2 adjacencies: 1
Total number of adjacencies: 2
Tag UNDERLAY:  VRF : default
System Id      Interface   SNPA                State  Holdtime  Type Protocol
ipi-2.lab.jan1.us.ipa.net xe48        3c2c.99c0.00aa      Up     26        L1L2 IS-IS
ipi-1.lab.jan1.us.ipa.net#show mpls ilm-table
Codes: > - installed ILM, * - selected ILM, p - stale ILM
        K - CLI ILM, T - MPLS-TP, s - Stitched ILM
       S - SNMP, L - LDP, R - RSVP, C - CRLDP
       B - BGP , K - CLI , V - LDP_VC, I - IGP_SHORTCUT
       O - OSPF/OSPF6 SR, i - ISIS SR, k - SR CLI
       P - SR Policy, U - unknown

Code    FEC/VRF/L2CKT    ILM-ID      In-Label    Out-Label   In-Intf    Out-Intf/VRF       Nexthop
     LSP-Type
   i>   100.127.0.1/32     4           16101       Nolabel     N/A        N/A              127.0.0.1
     LSP_DEFAULT
   B>   evpn:1             3           17          Nolabel     N/A        N/A              127.0.0.1
     LSP_DEFAULT
   B>   evpn:100           1           16          Nolabel     N/A        N/A              127.0.0.1
     LSP_DEFAULT
   B>   evpn:1             2           640         Nolabel     N/A        N/A              127.0.0.1
     LSP_DEFAULT
   P>   100.127.0.2/32     7           20          3           N/A        xe48             100.126.0.2
     LSP_DEFAULT
   i>   100.126.0.2/32     5           26240       3           N/A        xe48             100.126.0.2
     LSP_DEFAULT
   i>   100.127.0.2/32     6           16102       3           N/A        xe48             100.126.0.2
     LSP_DEFAULT

Now we can see that we have a clns/isis neighbor with ipi-2 as well as learned labels. We can see both device’s node-sids in the label table on ipi-1.

This image has an empty alt attribute; its file name is IPA-Blog-ad-template-network.jpg
iparchitechs.com/contact

BGP EVPN Setup

Next we can build EVPN on top of the underlay to begin delivering services. First we have to build an EVPN BGP session between the two routers.

ipi-1.lab.jan1.us.ipa.net#show run bgp
!
router bgp 65000
 neighbor 100.127.0.2 remote-as 65000
 neighbor 100.127.0.2 update-source lo
 !
 address-family l2vpn evpn
 neighbor 100.127.0.2 activate
 exit-address-family
 !
ipi-1.lab.jan1.us.ipa.net#show bgp l2vpn evpn summary
BGP router identifier 100.127.0.1, local AS number 65000
BGP table version is 32
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd     AD  MACIP
MCAST    ESI  PREFIX-ROUTE
100.127.0.2              4 65000 22856      22856      32      0      0  6d18h34m               2      1      0
     1      0      0

EVPN-VPWS

Next we can start build services on top. First we’ll build an EVPN-VPWS service.

ipi-1.lab.jan1.us.ipa.net:
!
evpn mpls enable
!
evpn mpls vtep-ip-global 100.127.0.1
!
mac vrf BLUE
 rd 100.127.0.1:1
 route-target both evpn-auto-rt
!
evpn mpls id 100 xconnect target-mpls-id 2
 host-reachability-protocol evpn-bgp BLUE
!
interface xe46.10 switchport
 encapsulation dot1q 10
 access-if-evpn
  map vpn-id 100
!

EVPN MPLS has to be enabled. *IMPORTANT* This requires a reboot. Next the vtep id needs to be set. These are global settings for the environment.

For the creation of the service we’ll start by making a mac vrf to generate the information needed to create a EVPN type 2 route (mac-ip).

Since this is VPWS it is considered a cross connect xconnect and a target is defined. This is the remote PE vpn-id, in this case 2.

Finally it is assigned to a switchport. It has to be a switchport with a type of access-if-evpn. This maps back to the EVPN mac-vrf via the xconnect. Anything arriving on xe46.10 with a dot1q tag of 10 is placed into this tunnel.

ipi-1.lab.jan1.us.ipa.net#show evpn mpls xconnect
EVPN Xconnect Info
========================
AC-AC: Local-Cross-connect
AC-NW: Cross-connect to Network
AC-UP: Access-port is up
AC-DN: Access-port is down
NW-UP: Network is up
NW-DN: Network is down
NW-SET: Network and AC both are up

Local                            Remote       Connection-Details

================================ ============ ==========================================================================
=========
VPN-ID       EVI-Name      MTU   VPN-ID       Source       Destination                   PE-IP           MTU   Type   NW
-Status
================================ ============ ==========================================================================
=========
100          ----          1500  2            xe46.10      --- Single Homed Port ---     100.127.0.2     1500  AC-NW  NW
-SET

Total number of entries are 1
ipi-1.lab.jan1.us.ipa.net#show evpn mpls xconnect tunnel
EVPN-MPLS Network tunnel Entries
Source           Destination      Status        Up/Down       Update        local-evpn-id remote-evpn-id
========================================================================================================
100.127.0.1      100.127.0.2      Installed     01:31:06      01:31:06      100           2

Total number of entries are 1

The tunnels are up, installed, and ready for forwarding. We can see the CE macs as mac-ip routes in evpn.

ipi-1.lab.jan1.us.ipa.net#show bgp l2vpn evpn vrf BLUE
BGP table version is 1, local router ID is 100.127.0.1
Status codes: s suppressed, d damped, h history, a add-path, * valid, > best, i - internal,
              l - labeled, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

[EVPN route type]:[ESI]:[VNID]:[relevent route informantion]
1 - Ethernet Auto-discovery Route
2 - MAC/IP Route
3 - Inclusive Multicast Route
4 - Ethernet Segment Route
5 - Prefix Route

    Network          Next Hop            Metric    LocPrf	Weight     Path  Peer          Encap
* i  [1]:[0]:[2]:[16]  100.127.0.2          0        100       0    i  100.127.0.2     MPLS
*>   [1]:[0]:[100]:[16]
                       100.127.0.1          0        100       32768  i  ----------      MPLS

Total number of prefixes 2

The mac addresses are sent via an EVPN type-2 route between PEs.

ipi-1.lab.jan1.us.ipa.net#show evpn mpls mac-table
========================================================================================================================
=================
                                                     EVPN MPLS MAC Entries
========================================================================================================================
=================
VNID       Interface VlanId    In-VlanId Mac-Addr       VTEP-Ip/ESI                    Type            Status     MAC mo
ve AccessPortDesc
________________________________________________________________________________________________________________________
_________________


Total number of entries are : 0

Since this is VPWS there are no macs learned on the device.

[email protected]# run ping 172.16.0.2
PING 172.16.0.2 (172.16.0.2): 56 data bytes
64 bytes from 172.16.0.2: icmp_seq=0 ttl=64 time=21.531 ms
64 bytes from 172.16.0.2: icmp_seq=1 ttl=64 time=22.124 ms

Success! The CEs can reach each other over the EVPN-VPWS circuit.

EVPN-VPLS

Now we’ll build an EVPN-VPLS service. The BGP setup is the same so we’ll focus solely on the differences. The first one being the vpn-id creation.

mac vrf ORANGE
 rd 100.127.0.1:2
 route-target both evpn-auto-rt
!
evpn mpls id 1
 host-reachability-protocol evpn-bgp ORANGE
!

There is no end point defined as a xconnect. All that is necessary is to bind the mac vrf to the evpn vpn id.

interface xe46.100 switchport
 encapsulation dot1q 100
 access-if-evpn
  map vpn-id 1
!

Again, a switchport defined as an access-if-evpn is necessary. This is then mapped to the vpn-id for the VPLS service. In this case anything coming in with a dot1q tag of 100 will be placed into vpn-id 1.

ipi-1.lab.jan1.us.ipa.net#show evpn mpls mac-table
========================================================================================================================
=================
                                                     EVPN MPLS MAC Entries
========================================================================================================================
=================
VNID       Interface VlanId    In-VlanId Mac-Addr       VTEP-Ip/ESI                    Type            Status     MAC mo
ve AccessPortDesc
________________________________________________________________________________________________________________________
_________________

1          xe46.100  ----      ----      84c1.c132.5031 100.127.0.1                    Dynamic Local   -------    0
   -------
1          ----      ----      ----      84c1.c132.5032 100.127.0.2                    Dynamic Remote  -------    0
   -------

Total number of entries are : 2

Since this is a VPLS service MACs are learned both locally and remotely. The remote MAC is the MAC of the remote CE. This was learned via EVPN and from the VTEP 100.127.0.2.

ipi-1.lab.jan1.us.ipa.net#show bgp l2vpn evpn mac-ip vrf ORANGE
ESI                            Eth-Tag     Mac-Address    IP-Address                              VNID/LABEL     L3VNID
   Nexthop         GW-Type         Encap
0                              1           84c1:c132:5031 --                                      17             0
   100.127.0.1     --              MPLS
0                              1           84c1:c132:5032 --                                      17             0
   100.127.0.2     --              MPLS

The type-2 routes are populated in the BGP table.

[email protected]# run ping 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: icmp_seq=0 ttl=64 time=21.894 ms
64 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=22.159 ms

Success! We have reachability across the service.

Conclusion

IP Infusion is continuing to build their evpn/mpls deployment as well as segment routing. It is exciting to see these feature sets continue to mature as traditional LDP/VPLS deployments move to EVPN/MPLS. If you need assistance on the transition from LDP to segment routing or VPLS to EVPN reach out to IP Architechs.


Categories
network operating systems

Networking CLI Rosetta Stone

Changing between network operating systems is one of the most challenging things for new engineers. Most people learned cisco cli due to their extensive training system or got on the job training for whatever their company runs.

We are hoping to make moving back and forth between network operating systems a little easier with some useful show and operational commands for Mikrotik, Juniper, Cisco, and IP Infusion. There are in detail usages of these commands on stubarea51.net.

OSPF Commands

MikroTikJuniperCiscoIP Infusion
routing ospf neighbor printshow ospf neighborshow ip ospf neighborshow ip ospf neighbor
routing ospf interface printshow ospf interfaceshow ip ospf interfaceshow ip ospf interface
routing ospf instance print detailshow ospf overview briefshow ip ospf 1show ip ospf 1
routing ospf lsa printshow ospf databaseshow ip ospf databaseshow ip ospf database
ip route print where ospf=yesshow route protocol ospfshow ip route ospfshow ip route ospf

routing ospf area-border-router print
show ospf route abrshow ip ospf border-routersshow ip ospf border-routers

routing ospf as-border-router print
show ospf route asbrshow ip ospf border-routersshow ip ospf border-routers

MPLS – LDP Commands

MikrotikJuniperCiscoIP Infusion
mpls ldp neighbor printshow ldp neighborshow mpls ldp neighborshow mpls ldp neighbor
mpls ldp interface printshow ldp interfaceshow mpls interfacesshow ldp interface
mpls forwarding-table printshow route forwarding-table family mplsshow mpls forwarding-tableshow mpls forwarding-table
mpls remote-bindings printshow ldp databaseshow mpls bindingshow mpls ilm-table
mpls local-bindings printshow ldp databasesh mpls ip binding localshow mpls ilm-table
mpls printshow mpls label usagesh mpls ldp parametersshow mpls label-space 0

BGP Commands

MikroTikJuniperCiscoIP Infusion
routing bgp peer print briefshow bgp summaryshow ip bgp summaryshow ip bgp summary
routing bgp peer print statusshow bgp neighborshow ip bgp neighborshow ip bgp neighbors
routing bgp advertisements print peer=peer_nameshow route advertising-protocol bgp 172.31.254.2show ip bgp neighbor 172.31.254.2 advertised-routesshow ip bgp neighbors 172.31.254.2 advertised-routes
ip route print where received-from=peer_nameshow route receive-protocol bgp 172.31.254.2show ip bgp neighbor 172.31.254.2 received-routesshow ip bgp neighbors 172.31.254.2 received-routes
ip route print where bgp=yesshow route protocol bgpshow ip route bgpshow ip route bgp
routing bgp peer refresh peer1clear bgp neighbor 172.31.254.2 soft-inboundclear ip bgp 172.31.254.2 soft inclear ip bgp 172.31.254.2 soft in
routing bgp peer resend peer1clear bgp neighbor 172.31.254.2 softclear ip bgp 172.31.254.2 soft outclear ip bgp 172.31.254.2 soft out

Let us know what other commands you would like to see in our rosetta stone to make switching network operating systems a breeze.

Categories
Uncategorized

Situational Awareness for Network Migrations

At IP Architechs we perform a lot of network migrations and it is no secret network migrations/ maintenance windows can be one of the most nerve-racking things for engineers, managers, and business leaders for a variety of reasons.

For the engineers the uncertainty might be caused by fear of failure, not being able to predict the outcome due to complexity, rushed on preparation to meet a deadline, or a litany of other reasons.

For managers and business leaders it might be more along the lines of; what happens if this goes wrong, how will this effect my bottom line, are there going to be 1000s of trouble tickets come 8/9am when everyone hits the office, and so on.

The Preparation

We’re going to look at this at the perspective of the engineer throughout. The prep work is probably one of the most important pieces of success. This is where you do many things including but not limited to:

  • building and testing the configuration to be implemented
  • making a rollback plan — this might be something as simple as move a cable and shut an interface or a multistep/multi-device plan
  • know the situation surrounding the window

Lets explore understanding the situation surrounding the window a some more. I’ll use some real examples here to help.

We were getting ready to change the internet edge deployment at an enterprise. We did all the prep and rollback planning. However, we were given a few constraints on downtime by the business. Additionally, all of the product teams had to join the call for verification due to the impact of the, relatively small, routing change. The next opportunity was going to be a few months out due to change freezes and the coordination of resources necessary.

So what did we learn by engaging outside of the technical realm?

  • We had tight timeframes which placed an increased emphasis on planning
  • We needed to have plans for things that could go wrong and resolution paths based on downtime constraints
  • although a low impact routing change it was a high impact business change
  • We needed to have clearly defined decision points on what would be cause for a rollback
Image

The Execution

All the prep is done and it’s time to execute the change. We put in the first couple lines of the script and everything is going well. We get to the point where we need to clean up the old configuration. Then every engineers nightmare happens – everything starts to go down.

Okay what do we do now, we know based on the situation we don’t have a lot of time to work through the problem. We need to stay calm and start working through our decision trees made during the planning process.

Some quick troubleshooting revealed when we removed the no longer used virtual routing and forwarding (VRF) instance it shutdown the ports that we now in the global table. We put the VRF back, still unused, everything began to work as expected again.

Next the debate began, should we get TAC on the line to assist. There were still a few items to knock out in the change window to avoid a complete rollback. A majority of people wanted to “chase the rabbit” of what caused the VRF deletion to bring down the interface. However, this would not be a good use of our time. If we got TAC on the line and began to go down that rabbit hole there is no telling where it would have gone or how long it would have taken. The facts were leaving the unused VRF, although annoying to have extra config, didn’t effect performance as far as we could tell and we needed to get through the rest of the migration.

After a short debate we all agreed based on the circumstances of the migration, coordination efforts, business drivers, and still needing to get some more work done we would continue down the migration path. We also took the necessary logs for an initial case with TAC and opened a ticket in the morning. Would we get the same level of info/t-shooting on that problem? No, but we were able to complete the migration and follow up on the weird behavior at a safer time.

Conclusion

Sometimes, based on different circumstance, the right decision would be to get TAC on the line and work through the issue. The owners might decide everything can be down until it’s working as planned or anywhere in between. Often, things like physical access or travel will allow for longer down time/troubleshooting.

It is important to know the situation around the migration, why it’s happening, who’s involved, and keep awareness of those during the migration to make informed decisions with the owner to make everyone successful.

If you need help planning your migrations reach out to us.

This image has an empty alt attribute; its file name is IPA-Blog-ad-template-network.jpg