Wireless Access

Hi

In our Mobility Master environment running 8.4.0.3 we have 10 controllers.

One of the controllers daily drops to a down status on the MM and will not come back online on the MM - if you log on locally to the controller or over the site to site to the MD it is still up as expected however the tunnel to the MM seems to drop for a slight period and then the only way to get it to come back up on the MM is to reboot the controller.  Any ideas of a fix for this?

Thanks

Scott

If its the tunnel causing the issue, have you tried restarting the isakmpd process on the controller rather than rebooting the controller on the MD?

Its an IPSEC tunnel so what is the status of the security associations during the time it goes down? What do the security logs on the MM show for this specific tunnel?

Enable IKE logging to find out why exactly the tunnel is being torn down.

What do the heartbeat stats between the MM and MD indicate?

--Give Kudos: found something helpful, important, or cool? Click Kudos Star in a post.
--Problem Solved? Click "Accepted Solution" in a post.

How do I restart the isakmpd process?

Issue the command "show process monitor statistics"

check for the total restarts for this specific process. What does the restart knob indicate? Has this already been restarted by the inhouse process manager(Its called Nanny)?

To restart this you need to issue the command " process restart <name of the process>".

Could you share the output of the command " show log security 50 | include ike"

Also enable ike logging, I am only seeing Fpapps logs.

When the controller( MD ) is up and running with an established tunnel,

Check if the IPSEC map on the controller is showing the correct peer address.

To find the name ipsecmap just issue " show ip route ", the maps should be at the bottom of the route table.

To view the map, "show crypto-local ipsecmap <name of the map>" 

check the output of the peer SA's, can be done using the command " show crypto isakmp peer <peer ID> ".

To Clean up the existing SA's use the command "crypto-local ipsec sa-cleanup" if you feel there are unwanted SA's.

--Give Kudos: found something helpful, important, or cool? Click Kudos Star in a post.
--Problem Solved? Click "Accepted Solution" in a post.

The heartbeat stats clearly indicate that the MD cannot heartbeat with its master.

Is UDP 4500 allowed on the firewall ?What does the datapath session show on the MM?

Show datapath session-table <MD IP> | include 4500

Which Phase of IPSEC is failing?

1. ISAKMP - show crypto isakmp sa - if there is a Security Association Entry here then phase 1 is good to go.

2. IPSEC - show crypto ipsec sa - check to see if there is an SA here.

What is the output of the " show switches ". More exactly what is the configuration state and Configuration ID of the MD in question? Is there a mismatch between the config ID's ?

( The config ID is the last column of the output )

--Give Kudos: found something helpful, important, or cool? Click Kudos Star in a post.
--Problem Solved? Click "Accepted Solution" in a post.

Do you restart on the MM or on the MD?

Never on the MM (I believe its in production).

On the MD as an alternative to rebooting the controller.

Delete the SA using the cleanup command, then wait for the ISAKMP SA to form again.

I see exactly same issue at only one of my customers. The issue focus only on one of two controllers , its always the same one. The only differents is that the "bad one" is connected the IPSEC over the internet. 

So probably this issue is an unstabl (IPSEC) connection. But i never get the clue on this.