To me it seems as everything is as it should be (config-wise). Maybe one thing to add is "soft-reconfiguration inbound" option in order to not be dependent on clearing the BGP session everytime you change policies/config.
If the issue still persists, I would now start doing some debugging, making sure the BGP session is stable (e.g. no constant UPDATE messages arriving from the peer revoking and advertising the default route). BGP debugging may lead you to the reason why the route is constantly installed and revoked again.
Did you try to disable BFD for a second to check whether the issue persists? And am I correct that the two 6400 are the only routers in AS 64921 (--> iBGP requires a full-mesh or a route-reflector design to work properly)?
What software version are you using? Did you check the release notes whether there is any bugs fixed in newer versions?
Best,
Thomas
------------------------------
Thomas Siegenthaler
------------------------------
Original Message:
Sent: Jan 10, 2022 04:49 AM
From: Jukka Aaltonen
Subject: BGP default route flapping in VSX pair
Yes 10.100.54.18/31 is configured on VLAN interface and that VLAN is allowed between CX switches. I did clear the BGP session, and now CX1 can see the next hop for 0.0.0.0/0 as 10.100.54.19 (I mean I did it before sending those configs to this discussion, not now)
Original Message:
Sent: Jan 10, 2022 04:45 AM
From: Vincent Giles
Subject: BGP default route flapping in VSX pair
Is 10.100.54.18 a "connected" IP on CX-2 ?
Did you perform a clear bgp * after adding next-hop-self ?
------------------------------
Vincent Giles
Original Message:
Sent: Jan 10, 2022 02:45 AM
From: Jukka Aaltonen
Subject: BGP default route flapping in VSX pair
Hi,
this is after I've shut down the CX-1 - core link. After shutting it down and doing show ip ro vrf vrf1 I can see default there, then it's gone and back again etc. I did forget to add next-hop self, but even after adding it I'm getting the same behaviour
CX-1:
router bgp 64921
bgp router-id 10.100.6.102
vrf vrf1
neighbor 10.100.29.192 remote-as 64775
neighbor 10.100.29.192 description external
neighbor 10.100.29.192 timers 3 9
neighbor 10.100.54.19 remote-as 64921
neighbor 10.100.54.19 description cx2
neighbor 10.100.54.19 fall-over bfd
address-family ipv4 unicast
neighbor 10.100.29.192 activate
neighbor 10.100.29.192 advertisement-interval 0
neighbor 10.100.54.19 activate
neighbor 10.100.54.19 next-hop-self
redistribute connected
CX-1# show ip ro vrf vrf1
0.0.0.0/0, vrf vrf1
via 10.100.54.19, [200/0], bgp
Address-family : IPv4 Unicast
-----------------------------
Neighbor Remote-AS MsgRcvd MsgSent Up/Down Time State AdminStatus
10.100.29.192 64775 263346 318284 00h:04m:41s Idle Up
10.100.54.19 64921 40692 47551 00h:07m:09s Established Up
interface vlan3962
vrf attach vrf1
ip address 10.100.54.18/31
t3-agg1# show bgp vrf vrf1 ipv4 unicast neighbors 10.100.54.19 routes
Network Nexthop Metric LocPrf Weight Path
*>i 0.0.0.0/0 10.100.54.19 0 100 0 64775 65300 65301 65442 ?
CX-2:
router bgp 64921
bgp router-id 10.100.6.104
vrf vrf1
neighbor 10.100.54.16 remote-as 64775
neighbor 10.100.54.16 description external
neighbor 10.100.54.16 timers 3 9
neighbor 10.100.54.18 remote-as 64921
neighbor 10.100.54.18 description internal
neighbor 10.100.54.18 fall-over bfd
address-family ipv4 unicast
neighbor 10.100.54.16 activate
neighbor 10.100.54.16 advertisement-interval 0
neighbor 10.100.54.16 route-map ADD-COST-100 out
neighbor 10.100.54.18 activate
neighbor 10.100.54.18 next-hop-self
neighbor 10.100.54.18 route-map LOCAL-PREF-150 in
redistribute connected
CX-2# show ip ro vrf vrf1
0.0.0.0/0, vrf vrf1
via 10.100.54.16, [20/0], bgp
CX-2# show bgp vrf vrf1 ipv4 unicast neighbors 10.100.54.18 adv
Network Nexthop Metric LocPrf Weight Path
*>i 0.0.0.0/0 10.100.54.19 0 100 0 64775 65300 65301 65442 ?
Original Message:
Sent: Jan 10, 2022 02:21 AM
From: Thomas Siegenthaler
Subject: BGP default route flapping in VSX pair
Hi
can you please share your BGP config of the two 6400 router and some output like "show ip route", "show bgp all summary" and "show bgp all neighbor x.x.x.x advertised/received" ? One typical source of such a problem is the missing "next-hop-self" option on the iBGP peering between the two 6400. A BGP route does only get installed to the FIB of the nexthop is reachable to the router. So if you are relying on the other router's default prefix from the eBGP peering the nexthop would be the external router's address which may not be reachable by the not-directly-connected router. For this reason one would typically use the own address to advertise the external route internally using iBGP.
Best,
Thomas
------------------------------
Thomas Siegenthaler
Original Message:
Sent: Jan 08, 2022 05:58 AM
From: Jukka Aaltonen
Subject: BGP default route flapping in VSX pair
We have 2x 6400 connected with iBGP routing, peerin withing VRFs (vrf lite type of thing). If I shutdown the first switches uplink VLAN I get default route from switch 2, then it goes away, comes back to routing table, goes away etc. After a while flapping it settles and is left in the routing table.
Anyone seen anything similar or any ideas what might cause this?
Both 6400's are connected to different core routers with eBGP. External routes advertise only 0.0.0.0/0 to this VRF. While it is flapping the route is in BGP routing table just fine