Wireless Access

last person joined: 23 hours ago 

Access network design for branch, remote, outdoor, and campus locations with HPE Aruba Networking access points and mobility controllers.
Back to discussions
Expand all | Collapse all

Should tunnel going down between MM & MD cause a drop in whole site?

This thread has been viewed 12 times

  • 1.  Should tunnel going down between MM & MD cause a drop in whole site?

    Posted Sep 27, 2019 08:38 AM

    One of our sites with an MD on it has a site to site VPN connection that isn't the most stable i.e. drops once a week.

     

    The MD is local on site but when the site to site goes down and the MM sees the MD as "down" the actual site loses internet connectivity.  All I can see on the MD logs is:

     

    Sep 27 13:35:07  isakmpd[3445]: <103103> <3445> <WARN> |ike|   IKE SA Deletion: IKE2_delSa peer:xx.xx.xx.2:500 id:3652689924 errcode:ERR_IKESA_EXPIRED saflags:0x1 arflags:0x0
Sep 27 13:35:37  fpapps[3410]: <399838> <3417> <WARN> |fpapps|  Received TUN_DOWN from IKE for default-local-master-ipsecmap
Sep 27 13:35:37  isakmpd[3445]: <103103> <3445> <WARN> |ike|   DPD PEER DEAD: peer xx.xx.xx port:4500
Sep 27 13:35:37  isakmpd[3445]: <103103> <3445> <WARN> |ike|   IKE SA Deletion: IKE2_delSa peer:xx.xx.xx:4500 id:3652687756 errcode:ERR_IKESA_CLEARED saflags:0x1100002d arflags:0x200
Sep 27 13:35:37  isakmpd[3445]: <103103> <3445> <WARN> |ike|   IPSec SA Deletion: IPSEC_delSa SPI:72396100 OppSPI:7bd81800 Dst:xx.xx.xx Src:xx.xx.xx.3 flags:19 dstPort:0 srcPort:0
Sep 27 13:35:43  HCM[3944]: <359000> <3944> <ERRS> |HCM|  Unexpected HCM runtime error at hcm_keepalive_timer_handler 514 State changed to Down probeip xx.xx.xx src-intf 9 
Sep 27 13:35:43  fpapps[3410]: <399838> <4092> <WARN> |fpapps|  ipMapDeleteUplinkDefaultGateway: Deleting static vlan 9 gateway xx.xx.xx.1
Sep 27 13:35:43  fpapps[3410]: <399838> <4092> <WARN> |fpapps|  updateUplinkReachState: Wired uplink vlan 9 (bkp NO) reachability (to xx.xx.xx) changed from 1(UP) to 2(DOWN). mode 0. lb-state: ENABLED
Sep 27 13:35:45  snmp[3611]: <399803> <3611> <ERRS> |snmp|  An internal system error has occurred at file ../unix/aruba_main.c function snmpRequestProcessing line 704 error Cannot send snmp response.
Sep 27 13:35:46  cfgm[3380]: <399838> <3380> <WARN> |cfgm|  LmsHeartBeatResultAction: State(READY:UPDATE SUCCESSFUL:CFGID-494:PEND-488:INITCFGID:473) FD=33:Cannot heartbeat with the master.
Sep 27 13:35:50  snmp[3611]: <399803> <3611> <ERRS> |snmp|  An internal system error has occurred at file ../unix/aruba_main.c function snmpRequestProcessing line 704 error Cannot send snmp response.
Sep 27 13:35:53  ntpwrap: ntpdPollingTimer:512 Upstream servers not reachable via local interface.
Sep 27 13:35:54  ntpwrap: getNtpSrvRouteAddr:304:connect() failed
Sep 27 13:35:56  cfgm[3380]: <399838> <3380> <WARN> |cfgm|  LmsHeartBeatResultAction: State(READY:UPDATE SUCCESSFUL:CFGID-494:PEND-488:INITCFGID:473) FD=33:Cannot heartbeat with the master.
Sep 27 13:36:00  snmp[3611]: <399803> <3611> <ERRS> |snmp|  An internal system error has occurred at file ../unix/aruba_main.c function snmpRequestProcessing line 704 error Cannot send snmp response.
Sep 27 13:36:06  cfgm[3380]: <399838> <3380> <WARN> |cfgm|  LmsHeartBeatResultAction: State(READY:UPDATE SUCCESSFUL:CFGID-494:PEND-488:INITCFGID:473) FD=33:Cannot heartbeat with the master.
Sep 27 13:36:07  snmp[3618]: <301250> <3618> <ERRS> |snmp|  Sendto failed, unable to send trap to manager xx.xx.xx80:161.
Sep 27 13:36:07  snmp[3618]: <301250> <3618> <ERRS> |snmp|  Sendto failed, unable to send trap to manager xx.xx.xx80:162.
Sep 27 13:36:10  authmgr[3478]: <124006> <3478> <WARN> |authmgr|  {580} TCP srcip=172.21.201.101 srcport=54323 dstip=192.168.168.34 dstport=7680, action=deny, role=Kainos-GDN-kWireless, policy=kainos-gdn-kwireless
Sep 27 13:36:10  dhcpdwrap[3553]: <202078> <3553> <ERRS> |dhcpdwrap|  Relay ERROR: Network is unreachable DISCOVER server=172.21.0.29 giaddr=192.168.160.3 MAC=38:f9:d3:94:04:8e
Sep 27 13:36:10  dhcpdwrap[3553]: <202078> <3553> <ERRS> |dhcpdwrap|  Relay ERROR: Network is unreachable DISCOVER server=172.21.0.31 giaddr=192.168.160.3 MAC=38:f9:d3:94:04:8e
Sep 27 13:36:10  isakmpd[3445]: <103103> <3445> <WARN> |ike|   IKE SA Deletion: IKE2_delSa peer:xx.xx.xx.2:500 id:3652689925 errcode:ERR_IKESA_EXPIRED saflags:0x1 arflags:0x0
Sep 27 13:36:10  snmp[3611]: <399803> <3611> <ERRS> |snmp|  An internal system error has occurred at file ../unix/aruba_main.c function snmpRequestProcessing line 704 error Cannot send snmp response.
Sep 27 13:36:10  snmp[3618]: <301250> <3618> <ERRS> |snmp|  Sendto failed, unable to send trap to manager xx.xx.xx80:161.
Sep 27 13:36:10  snmp[3618]: <301250> <3618> <ERRS> |snmp|  Sendto failed, unable to send trap to manager xx.xx.xx80:162.
Sep 27 13:36:12  dhcpdwrap[3553]: <202078> <3553> <ERRS> |dhcpdwrap|  Relay ERROR: Network is unreachable DISCOVER server=172.21.0.29 giaddr=192.168.160.3 MAC=38:f9:d3:94:04:8e
Sep 27 13:36:12  dhcpdwrap[3553]: <202078> <3553> <ERRS> |dhcpdwrap|  Relay ERROR: Network is unreachable DISCOVER server=172.21.0.31 giaddr=192.168.160.3 MAC=38:f9:d3:94:04:8e
Sep 27 13:36:12  snmp[3611]: <399803> <3611> <ERRS> |snmp|  An internal system error has occurred at file ../unix/aruba_main.c function snmpRequestProcessing line 704 error Cannot send snmp response.
Sep 27 13:36:14  dhcpdwrap[3553]: <202078> <3553> <ERRS> |dhcpdwrap|  Relay ERROR: Network is unreachable DISCOVER server=172.21.0.29 giaddr=192.168.160.3 MAC=38:f9:d3:94:04:8e
Sep 27 13:36:14  dhcpdwrap[3553]: <202078> <3553> <ERRS> |dhcpdwrap|  Relay ERROR: Network is unreachable DISCOVER server=172.21.0.31 giaddr=192.168.160.3 MAC=38:f9:d3:94:04:8e
Sep 27 13:36:16  cfgm[3380]: <399838> <3380> <WARN> |cfgm|  LmsHeartBeatResultAction: State(READY:UPDATE SUCCESSFUL:CFGID-494:PEND-488:INITCFGID:473) FD=33:Cannot heartbeat with the master.
Sep 27 13:36:16  snmp[3611]: <399803> <3611> <ERRS> |snmp|  An internal system error has occurred at file ../unix/aruba_main.c function snmpRequestProcessing line 704 error Cannot send snmp response.
Sep 27 13:36:18  dot1x-proc:2[4061]: <121010> <4061> <ERRS> |dot1x-proc:2| |aaa| Error 128,Network is unreachable sending 215 bytes on radius socket 41
Sep 27 13:36:18  dot1x-proc:2[4061]: <138057> <4061> <ERRS> |dot1x-proc:2|  Failed to send the radius request for Station 38:f9:d3:94:04:8e 80:8d:b7:61:74:31
Sep 27 13:36:19  isakmpd[3445]: <103103> <3445> <WARN> |ike|   IKE SA Deletion: IKE2_delSa peer:xx.xx.xx:4500 id:3652687756 errcode:OK saflags:0x1110002d arflags:0x200
Sep 27 13:36:19  isakmpd[3445]: <103103> <3445> <WARN> |ike|   IKE SA Deletion: IKE2_delSa peer:xx.xx.xx:4500 id:3652689926 errcode:ERR_IKESA_EXPIRED saflags:0x41000005 arflags:0x20
Sep 27 13:36:23  dot1x-proc:2[4061]: <121010> <4061> <ERRS> |dot1x-proc:2| |aaa| Error 128,Network is unreachable sending 227 bytes on radius socket 41
Sep 27 13:36:23  dot1x-proc:2[4061]: <138057> <4061> <ERRS> |dot1x-proc:2|  Failed to send the radius request for Station 38:f9:d3:94:04:8e 80:8d:b7:61:74:31
Sep 27 13:36:25  authmgr[3478]: <124006> <3478> <WARN> |authmgr|  {581} TCP srcip=172.21.201.101 srcport=54323 dstip=192.168.168.34 dstport=7680, action=deny, role=Kainos-GDN-kWireless, policy=kainos-gdn-kwireless
Sep 27 13:36:26  cfgm[3380]: <399838> <3380> <WARN> |cfgm|  LmsHeartBeatResultAction: State(READY:UPDATE SUCCESSFUL:CFGID-494:PEND-488:INITCFGID:473) FD=33:Cannot heartbeat with the master.
Sep 27 13:36:27  dot1x-proc:1[4058]: <121010> <4058> <ERRS> |dot1x-proc:1| |aaa| Error 128,Network is unreachable sending 223 bytes on radius socket 42
Sep 27 13:36:27  dot1x-proc:1[4058]: <138057> <4058> <ERRS> |dot1x-proc:1|  Failed to send the radius request for Station 98:01:a7:9b:53:71 80:8d:b7:61:74:31
Sep 27 13:36:27  snmp[3611]: <399803> <3611> <ERRS> |snmp|  An internal system error has occurred at file ../unix/aruba_main.c function snmpRequestProcessing line 704 error Cannot send snmp response.
Sep 27 13:36:27  snmp[3618]: <301250> <3618> <ERRS> |snmp|  Sendto failed, unable to send trap to manager xx.xx.xx80:161.
Sep 27 13:36:27  snmp[3618]: <301250> <3618> <ERRS> |snmp|  Sendto failed, unable to send trap to manager xx.xx.xx80:162.
Sep 27 13:36:28  dot1x-proc:2[4061]: <121010> <4061> <ERRS> |dot1x-proc:2| |aaa| Error 128,Network is unreachable sending 227 bytes on radius socket 41
Sep 27 13:36:28  dot1x-proc:2[4061]: <138057> <4061> <ERRS> |dot1x-proc:2|  Failed to send the radius request for Station 38:f9:d3:94:04:8e 80:8d:b7:61:74:31
Sep 27 13:36:28  dot1x-proc:2[4061]: <138094> <4061> <WARN> |dot1x-proc:2|  MIC failed in WPA2 Key Message 2 from Station 64:a2:f9:a9:33:b6 80:8d:b7:61:13:a0 GDN-AP-08
Sep 27 13:36:29  dot1x-proc:2[4061]: <138094> <4061> <WARN> |dot1x-proc:2|  MIC failed in WPA2 Key Message 2 from Station 64:a2:f9:a9:33:b6 80:8d:b7:61:13:a0 GDN-AP-08
Sep 27 13:36:32  dot1x-proc:1[4058]: <121010> <4058> <ERRS> |dot1x-proc:1| |aaa| Error 128,Network is unreachable sending 223 bytes on radius socket 42
Sep 27 13:36:32  dot1x-proc:1[4058]: <138057> <4058> <ERRS> |dot1x-proc:1|  Failed to send the radius request for Station 98:01:a7:9b:53:71 80:8d:b7:61:74:31
Sep 27 13:36:33  dot1x-proc:2[4061]: <121010> <4061> <ERRS> |dot1x-proc:2| |aaa| Error 128,Network is unreachable sending 227 bytes on radius socket 41
Sep 27 13:36:33  dot1x-proc:2[4061]: <138057> <4061> <ERRS> |dot1x-proc:2|  Failed to send the radius request for Station 38:f9:d3:94:04:8e 80:8d:b7:61:74:31
Sep 27 13:36:34  dot1x-proc:1[4058]: <138094> <4058> <WARN> |dot1x-proc:1|  MIC failed in WPA2 Key Message 2 from Station 64:a2:f9:a9:33:b6 80:8d:b7:61:13:b0 GDN-AP-08
Sep 27 13:36:36  cfgm[3380]: <399838> <3380> <WARN> |cfgm|  LmsHeartBeatResultAction: State(READY:UPDATE SUCCESSFUL:CFGID-494:PEND-488:INITCFGID:473) FD=33:Cannot heartbeat with the master.
Sep 27 13:36:37  dot1x-proc:1[4058]: <121010> <4058> <ERRS> |dot1x-proc:1| |aaa| Error 128,Network is unreachable sending 235 bytes on radius socket 42
Sep 27 13:36:37  dot1x-proc:1[4058]: <138057> <4058> <ERRS> |dot1x-proc:1|  Failed to send the radius request for Station 98:01:a7:9b:53:71 80:8d:b7:61:74:31
Sep 27 13:36:38  dot1x-proc:2[4061]: <121010> <4061> <ERRS> |dot1x-proc:2| |aaa| Error 128,Network is unreachable sending 227 bytes on radius socket 41
Sep 27 13:36:38  dot1x-proc:2[4061]: <138057> <4061> <ERRS> |dot1x-proc:2|  Failed to send the radius request for Station 38:f9:d3:94:04:8e 80:8d:b7:61:74:31
Sep 27 13:36:38  snmp[3618]: <301250> <3618> <ERRS> |snmp|  Sendto failed, unable to send trap to manager xx.xx.xx80:161.
Sep 27 13:36:38  snmp[3618]: <301250> <3618> <ERRS> |snmp|  Sendto failed, unable to send trap to manager xx.xx.xx80:162.
Sep 27 13:36:39  ntpwrap: getNtpSrvRouteAddr:304:connect() failed
Sep 27 13:36:40  snmp[3611]: <399803> <3611> <ERRS> |snmp|  An internal system error has occurred at file ../unix/aruba_main.c function snmpRequestProcessing line 704 error Cannot send snmp response.
Sep 27 13:36:42  dot1x-proc:1[4058]: <121010> <4058> <ERRS> |dot1x-proc:1| |aaa| Error 128,Network is unreachable sending 235 bytes on radius socket 42
Sep 27 13:36:42  dot1x-proc:1[4058]: <138057> <4058> <ERRS> |dot1x-proc:1|  Failed to send the radius request for Station 98:01:a7:9b:53:71 80:8d:b7:61:74:31
Sep 27 13:36:43  dot1x-proc:2[4061]: <121010> <4061> <ERRS> |dot1x-proc:2| |aaa| Error 128,Network is unreachable sending 227 bytes on radius socket 41
Sep 27 13:36:43  dot1x-proc:2[4061]: <138057> <4061> <ERRS> |dot1x-proc:2|  Failed to send the radius request for Station 38:f9:d3:94:04:8e 80:8d:b7:61:74:31
Sep 27 13:36:46  cfgm[3380]: <399838> <3380> <WARN> |cfgm|  LmsHeartBeatResultAction: State(READY:UPDATE SUCCESSFUL:CFGID-494:PEND-488:INITCFGID:473) FD=33:Cannot heartbeat with the master.
Sep 27 13:36:46  snmp[3611]: <399803> <3611> <ERRS> |snmp|  An internal system error has occurred at file ../unix/aruba_main.c function snmpRequestProcessing line 704 error Cannot send snmp response.
Sep 27 13:36:47  dot1x-proc:1[4058]: <121010> <4058> <ERRS> |dot1x-proc:1| |aaa| Error 128,Network is unreachable sending 235 bytes on radius socket 42


  • 2.  RE: Should tunnel going down between MM & MD cause a drop in whole site?

    Posted Sep 27, 2019 09:54 AM

    Also seeing this on the logs when this tunnel drops:

     

    Sep 26 09:12:21  fpapps[3410]: <399838> <4092> <WARN> |fpapps|  updateUplinkReachState: Wired uplink vlan 9 (bkp NO) reachability (to xx.xx.xx0.203) changed from 2(DOWN) to 1(UP). mode 1. lb-state: ENABLED
Sep 27 13:35:43  fpapps[3410]: <399838> <4092> <WARN> |fpapps|  ipMapDeleteUplinkDefaultGateway: Deleting static vlan 9 gateway xx.xx.xx.1
Sep 27 13:35:43  fpapps[3410]: <399838> <4092> <WARN> |fpapps|  updateUplinkReachState: Wired uplink vlan 9 (bkp NO) reachability (to xx.xx.xx0.203) changed from 1(UP) to 2(DOWN). mode 0. lb-state: ENABLED
Sep 27 13:39:04  fpapps[3410]: <399838> <4092> <WARN> |fpapps|  ipMapAddUplinkDefaultGateway: Adding static vlan 9 gateway xx.xx.xx.1
Sep 27 13:39:04  fpapps[3410]: <399838> <4092> <WARN> |fpapps|  updateUplinkReachState: Wired uplink vlan 9 (bkp NO) reachability (to xx.xx.xx0.203) changed from 2(DOWN) to 1(UP). mode 1. lb-state: ENABLED

     This is showing the default gateway on the controller saying it has no reachability to the mobility master but why is it changing the wired uplink to down and up?



  • 3.  RE: Should tunnel going down between MM & MD cause a drop in whole site?

    EMPLOYEE
    Posted Sep 27, 2019 11:22 AM

    The MD does not rely on the MM for user connectivity.  Once the MD sends the user traffic to the local network, the local network is responsible for user traffic.