Understanding and Troubleshooting Master-Local controller issues
Master-Local communication is not UP/ Master-Local IPSec VPN is down/Master-Local Heart-beat is missing
Step 1 : Confirm whether Master-Local connectivity is established
On Master controller, use “ show switches” command and it should show both Master and Local controllers in the output. In the following sample output there is not Local controller information.
Step 2 : Verify whether the master is defined in Local or not
Use “show switchinfo” command on Local and identify the following
Use “Show localip” on the master and identify the following
Step 3: Debug security logs and identify the issue.
Enable “ logging level debugging security process crypto”
Use “show log security 50 | include ike” on the local (can be used on Master)
From the above output it can be identified that preshared key is mismatched.
Step 4: Ensure that the Key used on both master and local is same.
Use “encrypt disable” on both Master and Local and use the same commands as above
In the above sample output, keys are different, if so we have to change the key either of the controllers (preferably on master because any changes on local cases rebooting)
Once the key is matched, you should be able to see the following output. We can see a tunnel is formed between Master and Local.
Now we should be able to see Local controller in the Master when you run “ show switches” command
The following output is still not showing any Local controller.
Step 5 : Check whether Heart Beats are missing ?
Use “show master-local stats” to verify the HB req and resp status on bot Master and Local. The following sample output is showing that, the Master is not able to send HB response.
Now you have to get into system logs to verify the reason. The following sample output saying that, Master is receiving HB from Local (10.20.25.66) but, due to some reasons Master is not able to respond the HB req.
Enable “logging level debugging system process cfgm”
Use “show log system 50” on the master and Local
System Log output on Master :
System log output on Local :
The above output (Highlighted in Green Colour) indicating that master is failed to upgrade the Image of the Local. The suspected area is IPSec tunnel. It seems IPSec is up but there can be some sync issues.
The following output “show datapath session | include 4500” showing flag status as “FY”, this indicates IPSec negotiation is not completed successfully, Flag status “FY” indicates that there is no sync.
From “show log security 40 | include ike” we can identify the issue with 4500 traffic.
Possible cause will be Image mismatch, after updating the Image, Master-Local will come up. The same can be seen from the following sample output.
The following sample output showing that the Master-Local is up and synced successfully.