Wired Intelligent Edge

Hello all,

I need to create  GRE+IPsec tunnels via Internet between a Mobility access switch (MAS) at each remote office and two 7200 controllers (located at different locations joined via a L3 MPLS) so as to allow running ospf.

Is it correct to assume that just like in Cisco, I can use MAS inner interface (management) ip address as source of GRE tunnel, while the controller end external / DMZ mapped public routable address as tunnel destination address at the MAS ends, with controller ends using public ip as tunnel source and private inside address of each MAS as tunnel destination?

IPsec VPN is real easy part between MAS and Controller to establish underlying routing for GRE to take place on top. I may even try to set up a L2 GRE between two controllers over L3 MPLS, if that makes more sense to setup VRRP, but for now, simple LMS / backup LMS arrangement will work for downstream APs at remote offices.

Another question I have is if two tunnels from MAS to controllers can be setup as Primary and backup for simpler applications that may not need dynamic routing.

Any advice will be much appreciated.

Thank you

Further to my post, I do find that there is an option of Route Monitoring (just like IP SLA in Cisco) and use of the same for primary / secondary VPN tunnels, but it is for redundant Internet connections at branch. In my case, there is a single Internet, but two tunnels, one each to two separate controllers. If One controller fails, then in absence of OSPF, there does not seem to be a good solution.

Hi 'networkengineer'

When you mention 'Management interface'  Are ou refering to "out of band" ==> (config) #interface mgmt ?
 :If yes MAS don’t do any functionality on MGMT interface, It is just used for box login via ssh etc.. not traffic switching/routing
 : Instead use 'in-band management' (interface vlan <x>' ) -  which is pointed towards the extrenal network.

 With that, you can use public IP address of the contoller to establish GRE over IPsec.
  Two points:
   1. Peer IP should be reachable via internet
   2. 'tunnel source' would be your interesting traffic via OSPF

Regarding your second queston:
  You can try terminating IPsec on VRRP IP address, so that If primary goes down  your GREoverIPsec will be still up with new elected primary.

Rgds,

-Vinay

Hi Vinay,

Thanks for your advice. 

I did not mean out of band management on any equipment as I have not configured it any where. it simply was the inside vlan interface of the switch. Say this is 192.168.10.0/24 subnet on the MAS end. 

To make it simple, let us just consider a MAS to a single controller link over Internet.

The controller side IP is static public (say 30.40.50.60) and MAS side is dynamic public IP, so yes peer addresses are reachable over the internet, it is only that MAS as tunnel initiator will make a call out to the controller, while controller will simply be responder and has peer address to be 0.0.0.0.

For the controller side the inside private subnet is 172.16.10.0/24. There is a DMZ vlan 1 of  10.1.0.0/24 on the Controller with ip address say 10.1.0.10, that is then mapped thru firewall to 30.40.50.60 for outside world.

For IPsec tunnel, src-net at MAS end is 192.168.10.0 255.255.255.0 and dst-net is 172.16.10.0 255.255.255.0, with peer-ip of 30.40.50.60.

After IPsec tunnel forms, GRE tunnel can form using the private end to end reachability provided by IPsec.

The tunnel source-ip (for L3 GRE)  in this case is 192.168.10.1 (the gateway address for the inside hosts at MAS) and tunnel destination-ip is 172.16.10.10 (the ip address of controller inside vlan, also termed as management vlan by me).

If this works, then ospf can be run over this GRE+IPsec tunnel. My confusion is around the tunnel source and destination interfaces requirements, but you clarified that this needs to be the "Interesting traffic" and that should confirm my assumption as well. There is mention in the ArubaOS7.4 book but does not have this clarified anywhere and I would expect some diagram in guides with IP addresses and vlan IDs labeled, but there are none.

Let me know if this sounds correct and if this works, then L2GRE on the L3 MPLS between two controller sites may not be necessary as ospf will do the magic of failover and reachability.

Thanks again,

Yes, This should work.

This has become bit more complicated than I had thought. I missed the point that the link between the two controller sites has a L3 MPLS circuit and I need to run OSPF ( area 0) across the MPLS so that both core switches and the two controllers are in backbone area and then downstream MASs will be in different stub areas. And Aruba only permit one instance of OSPF on switches and controllers. 

And MPLS service provider is not ready (yet and that may be procedural and money matters) to run ospf on their interface and then distribute the ospf learned routes via BGP and make those available at other end. In absence of the same, I have no idea as to what can be done to achieve some kind of site resiliency with each controller at different site. At this time, current single controller is at one site and there are static routes between two sites. If these were just the APs terminating on to the controllers, then bridge mode / local switching would generally be acceptable, but issue is the MAS acting as the switch and firewall at each remote site.

Any ideas to bring some resiliency in this situation will be greatly appreciated.

The issue of carrying ospf over L3 MPLS is resolved. So that part is good. I also have ospf now running over IPsec+GRE from both WLCs to the MAS. The issue is that somehow, only one of the tunnel stays up if both WLC are terminating their respective tunnels from the MAS. On the MAS side both tunnels show up / up and hence routes pointed to the ipsec tunnel stay on, but GRE tunnel goes down, so that cause traffic blackhole. I have spent lots of time, and not able to understand as to why only one tunnel stays up. All devices have almost latest code. The two sites core switches and the WLCs sides towards switches are in area 0, while MAS branch is in area 1.Same situation even if I use area 0 all over.

Open a TAC case so that specific config details etc... can be reviewed in detail for further assistance.

This got resolved. It was potentially a bug in 7.4.0.1 in MAS OS. After reading teh release notes of 7.4.0.2 indicating some issues resolution for GRE and IPsec, I did firmware upgrade on the MAS and I can now have both tunnels up with ospf running end to end.

Just have a question though. Since on the controller(s) side, we will have a single crypto local ipsec map, with dst-net of 0.0.0.0 and peer-ip of 0.0.0.0, any idea as to how many ipsec tunnels a single map can handle? Although specs will indicate a large number of tunnels, I dont find data that will say all those can be tied to a single ipsec map. 

Thanks

Yes, There shouldn't be no limitation of number of SAs coming on one crypto-map with server kind of configuration(peer-ip being 0.0.0.0) as such.

What is the maximum 'Concurrent IPSec Sessions'  your deployment is likely to hit ?

Thanks,

-Vinay

To add to previous response:

There could system level limits defined on each of the platforms (7210/7220/7240) depending the  processing power built in.

 The expected max cuncurrent IPSec sessions for your deployment would help choose the right platform based on the published specs.(And I suppose those numbers should be w.r.t for single crypto map unless stated explcilty as with multiple...)

Thanks so much Vinay. I think ultimately there will be less than 200 sites, so we should be good. A few older RAPs could generate more tunnels than these switches will. Only feature request I will like to make is to permit distribution of static routes, just like any switch or router will do. Any low end router or switch that I come across supporting ospf, does support most functionality of ospf and I would expect the same from controller and switches, if they cannot support multiple instances of the ospf. 

Hello Vinay,

So another issue I encuntered today was that with two Ipsec tunnels configured, and on one of them going down, the routing table will still keep sending the traffic that way thru the downed tunnel. My understanding is that with tunnel down, the ipsec route shoudl be removed. This is now causing some issues in testing as though I am learning specific routes via ospf, the ipsec static route takes precedence and hence the ospf learned routes do not get installed. The ipsec tunnel routes gets installed as soon as the dst-net statement is configured udner ipsec map and they seem to not go away. I tried to give them higher metric of 120 ( ospf being 110) udner ip-profile on the switch, but they still show up as connected with a metric of 1.

Thanks

I resolved this issue by creating a loopback address on each controller and then using that as ipsec map source and tunnel source address. Works great now. only thing that I cannot ping when an internet circuit goes down at controller end is the supposed to be always up loopback on the controller from the switch.