Network Management

The set up:

Unboxed the new 7210 and did basic configuration prior to making it a local controller - child of the 3600 it is intended to replace. It came up in airwave as expected.

Converted it to a master, rebooted, upgraded code from 6.3.1.10 to 6.4.2.4 and made some AP configuration and provisioning changes to prepare for the new AP205 and 275 mountain we're deploying.

Added the first 5 APs and all sems well on the controller and the 5 initial access points. Except airwave doesn't seem to have detected them.

The (probable) issue:

in the device monitoring page in airwave, I see the following under "recent AMP device events:

Time	User	Event
Mon Mar 16 12:37:17 2015 	matthew.sabin 	Polling now
Mon Mar 16 12:26:28 2015 	matthew.sabin 	Poll Now: no results after 10 minutes.

Sounds like an SNMP problem, so:

Quick comparison of SNMP settings on controllers

New:

snmp-server user "airwave" auth-prot sha 12345678 priv-prot des 12345678
mobility-manager 10.a.a.a user airwave 12345678 udp-port 162 rtls 8000

 Old:

snmp-server user "airwave" auth-prot sha 12345678 priv-prot des 12345678
mobility-manager 10.a.a.a user airwave 12345678 udp-port 162 rtls 8000

Quick comparison of Airwave's credential sets

New:

Credentials for 091aw009:
Community String:	
Telnet/SSH Username:	solarwinds
Telnet/SSH Password:	xxxxxxxxxx
"enable" Password:	xxxxxxxxxx
SNMPv3 Username:	airwave
Auth Password:	12345678
SNMPv3 Auth Protocol:	sha
Privacy Password:	12345678
SNMPv3 Privacy Protocol:	des
IP: 	10.n.n.n

 Old:

Credentials for 091boiid-wc1:
Community String:	
Telnet/SSH Username:	solarwinds
Telnet/SSH Password:	xxxxxxxxxx
"enable" Password:	xxxxxxxxxx
SNMPv3 Username:	airwave
Auth Password:	12345678
SNMPv3 Auth Protocol:	sha
Privacy Password:	12345678
SNMPv3 Privacy Protocol:	des
IP: 	10.o.o.o

 I'm not seeing any difference, yet old works fine, and new does not. Is there a checkbox in the controller config I missed which tells it to allow SNMP?

I did find a thread with advice to test SNMP from Airwave to the controller, however that was for v2c and my attempts to duplicate it for v3 have been unsuccessful.

Any ideas?

Did you check the Group -> select group -> Basic tab to make sure communication to the Aruba devices in that group switch from v2c to 3?

Yes, forgot to mention: both controllers live in the same group, which is set for v3.

It'd be worth it to try doing a snmpv3 walk from the AirWave CLI.  If the walk doesn't work from the AirWave then it's looking like it might be a controller config issue.  I typically do my controller SNMP mgmt from the UI, but it should be similar to what you've shown from the CLI.

# snmpwalk -v3 -u <v3 user> -A <auth pw> -a md5 -l AuthPriv -X <prviacy pw> <ip.address.of.controller> <OID>

example:

# snmpwalk -v3 -u snmpv3user -A myauthpw -a md5 -l AuthPriv -X myprivpw 10.10.10.110 sysName

On old I get:

[root@bgoair01 mercury]# snmpwalk -v3 -u airwave -A 12345678 -a md5 -l AuthPriv -X 12345678 10.30.0.9 sysName
Did not find 'zeroDotZero' in module SNMPv2-SMI (/usr/local/airwave/share/snmp/mibs/diffserv.my)
Did not find 'zeroDotZero' in module SNMPv2-SMI (/usr/local/airwave/share/snmp/mibs/CISCO-BULK-FILE-MIB.my)
Undefined OBJECT-GROUP (cLMobilityGroupRev01ConfigGroup): At line 1005 in /usr/local/airwave/share/snmp/mibs/CISCO-LWAPP-MOBILITY-MIB.my
Did not find 'zeroDotZero' in module SNMPv2-SMI (/usr/local/airwave/share/snmp/mibs/IP-MIB.my)
Group not found in module (ciscoLwappSysIgmpConfigGroup): At line 53 in /usr/local/airwave/share/snmp/mibs/CISCO-LWAPP-SYS-CAPABILITY.my
Bad timestamp format (11 or 13 characters) (2011122200000Z): At line 164 in /usr/local/airwave/share/snmp/mibs/CISCO-LWAPP-PMIP-MIB.my
Did not find 'zeroDotZero' in module SNMPv2-SMI (/usr/local/airwave/share/snmp/mibs/hh3c-epon-device.mib)
Did not find 'zeroDotZero' in module SNMPv2-SMI (/opt/airwave/share/snmp/mibs/DISMAN-SCHEDULE-MIB.txt)
Did not find 'zeroDotZero' in module SNMPv2-SMI (/opt/airwave/share/snmp/mibs/DISMAN-EVENT-MIB.txt)
snmpwalk: Authentication failure (incorrect password, community or key) (Sub-id not found: (top) -> sysName)

 On new I get:

[root@bgoair01 mercury]# snmpwalk -v3 -u airwave -A 12345678 -a md5 -l AuthPriv -X 12345678 10.30.0.8 sysName
Did not find 'zeroDotZero' in module SNMPv2-SMI (/usr/local/airwave/share/snmp/mibs/diffserv.my)
Did not find 'zeroDotZero' in module SNMPv2-SMI (/usr/local/airwave/share/snmp/mibs/CISCO-BULK-FILE-MIB.my)
Undefined OBJECT-GROUP (cLMobilityGroupRev01ConfigGroup): At line 1005 in /usr/local/airwave/share/snmp/mibs/CISCO-LWAPP-MOBILITY-MIB.my
Did not find 'zeroDotZero' in module SNMPv2-SMI (/usr/local/airwave/share/snmp/mibs/IP-MIB.my)
Group not found in module (ciscoLwappSysIgmpConfigGroup): At line 53 in /usr/local/airwave/share/snmp/mibs/CISCO-LWAPP-SYS-CAPABILITY.my
Bad timestamp format (11 or 13 characters) (2011122200000Z): At line 164 in /usr/local/airwave/share/snmp/mibs/CISCO-LWAPP-PMIP-MIB.my
Did not find 'zeroDotZero' in module SNMPv2-SMI (/usr/local/airwave/share/snmp/mibs/hh3c-epon-device.mib)
Did not find 'zeroDotZero' in module SNMPv2-SMI (/opt/airwave/share/snmp/mibs/DISMAN-SCHEDULE-MIB.txt)
Did not find 'zeroDotZero' in module SNMPv2-SMI (/opt/airwave/share/snmp/mibs/DISMAN-EVENT-MIB.txt)
snmpwalk: Authentication failure (incorrect password, community or key) (Sub-id not found: (top) -> sysName)

Which appear to be the same, (and not successful).

However Aiwave does appear to be able to SNMP-walk old, and not new - or I'm chansing a red herring.

Correction,  when I changed md5 to sha it works.

Both devices report their names correctly.

Sounds like that makes it the Airwave side of the conversation then?

Let me see about getting a QA resource to replicate this in the lab.  Stay tuned for an update after they reproduce.

QA tested this in the lab, here's what they had to say:

"Tried this and both sha and md5 worked without any problem. We ran into the problem Authentication failure (incorrect password, community or key)when the community or username/pwd isn’t configured correctly for either of the protocols, and when that is fixed, snmp worked just fine."

With that in mind, it might be worth it to open a TAC case since it should be working.

Heading that way now.

Thanks! for your quick dive in for me.

I'll report back here with what we find with TAC.

After much poking around and capturing internal processes etc. TAC determined "that something was wrong with the AP Watcher, or the SNMP Fetcher threads" and asked if it was OK to restart those processes.

I approved, and immediately the SNMP poll ran and my access-points got descovered.

Rebooting Airwave would have had the same effect, so if anyone found this thread to solve their own SNMP troubles, you might try the Airwave reboot to see if it helps.