Summary: This article tells us the basic steps to check if Airwave is processing the data that is received by AMON feed
Question: What are the basic steps to verify if Airwave is processing AMON feed or not?
Below are few troubleshooting steps to check if Airwave is receiving AMON feed from the controllers:STEP 1 :On Airwave, check if the AMONAggregator and PAPI daemonss are running. Type 'wd' and we will get the below output and check for the highlighted options to be running:
STEP 2:We can also monitor the AMON graphs under SYSTEM --> Performance page of the Airwave server.Example:
This gives more details on the AMON Packets and Messages processing rate.Also, if the AMON message loss graph is blank, it means there is no data loss and the requests are being processed properly on the Airwave server.STEP 3:Then try checking the logging:# qlog enable amon_dump_rawThe above command will create a file names 'amon_dump_raw' under /var/log/amp_diag directory.# tail -f /var/log/amp_diag/amon_dump_rawè Need to wait for the dump to populate.è This log captures data after it has come into AirWave, but before it hits the processing queues. From there, you can translate the data to be human readable using the below decode command:# qlog_decoder --decoder papi --nofollow /var/log/amp_diag/amon_dump_rawOnce the file is decoded, we can the file to see the data in human readable format which will give us more information on the data that is being sent from the controller to Airwave.
Thanks for the article, it gave me a place to start looking. Was running a wireshark on airwave via tcpdump, to find out some insight as to what was going on with AMON and PAPI traffic. There wasn't much use as the packets were being delievered although there was still an issue with it getting processed into airwave. It was suspected there was something with the payload that was not being processed.
Ended up having to use the following command "qlog enable amon" as the amon_dump_raw was not showing up in /var/log/amp_diag/ folder.
I was able to see some insight in regards to what was going on at this point. I was receiving the following error in the log file.
[root@jk-amp-01 mercury]# cat /var/log/amp_diag/amon1460724791.732955 24641 papi|dropping payload from 172.16.22.13 because id is not found1460724792.982381 24641 papi|dropping payload from 172.16.22.13 because id is not found1460724792.982667 24641 papi|dropping payload from 172.16.22.13 because id is not found
Since my contorller was configured to use the loopback interface as the controller-ip, this is what was causing me issues with getting the payload of papi traffic to be processed by airwave correctely. First I did try changing airwave to have the mgmt IP for the controller of 172.16.22.13, vs the loopback. This still did not help at all.
(jk-aruba7005) #show controller-ip
Switch IP Address: 172.16.20.100
Switch IP is configured to be loopback interface
After configuring my controller to use the L3 interface vlan199 (172.16.22.13), the payload was then interperitated correctly by airwave, and it was processing amon traffic correctly.
[root@jk-amp-01 mercury]# cat /var/log/amp_diag/amon
1460725292.565372 24641 papi|dropping payload from 172.16.22.13 because id is not found1460725299.505907 24641 papi|dropping payload from 172.16.22.13 because id is not found1460725322.634027 24641 papi|dropping payload from 172.16.22.13 because id is not found1460725467.677425 24523 publish_controller_vap_stats|published stats for 0 aps1460725725.080516 24641 papi|dropping payload from 172.16.22.13 because id is not found1460725731.480864 24647 slice_by_node_id|publish to AMP_inline_stats MQ for AMON_DHCP_STATION_INFO_MESSAGE took 0.000330 seconds1460725741.130303 24647 slice_by_node_id|publish to AMP_amon_1 MQ for AMON_USER_INFO_MESSAGE took 0.000201 seconds1460725746.151027 24647 slice_by_node_id|publish to AMP_amon_1 MQ for AMON_USER_INFO_MESSAGE took 0.000135 seconds1460725746.152604 24647 slice_by_node_id|publish to AMP_inline_stats MQ for AMON_DOT1X_MESSAGE took 0.000123 seconds1460725750.025544 24647 slice_by_node_id|publish to AMP_inline_stats MQ for AMON_PASSIVE_AP_STATION_STATS_MESSAGE took 0.000141 seconds1460725751.173871 24647 slice_by_node_id|publish to AMP_amon_1 MQ for AMON_USER_INFO_MESSAGE took 0.000133 seconds1460725755.170318 24647 slice_by_node_id|publish to AMP_amon_1 MQ for AMON_VAP_STATS_MESSAGE took 0.000136 seconds1460725755.171294 24647 slice_by_node_id|publish to AMP_amon_1 MQ for AMON_VAP_STATS_MESSAGE took 0.000131 seconds
Not sure exactly what the problem was wiht using the loopback as the controller-ip in this case. I have inspected the wiresharks from both before / after changing the controller-ip. There isn't really anything that sticks out and the field for the ID: 0x4972 is populated under the PAPI values for both before and after wiresharks.
I suspect that since the papi traffic was being sent to airwave with the L3 interface IP from vlan199 vs sourceing with the IP of the loopback interface, this was the main culprit and I was forced to change teh controller back to the L3 interface vs loopback. I also had tried multiple versions of code, without any success either (126.96.36.199 and 188.8.131.52)
At Aruba, we believe that the most dynamic customer experiences happen at the Edge. Our mission is to deliver innovative solutions that harness data at the Edge to drive powerful business outcomes.
© Copyright 2021 Hewlett Packard Enterprise Development LPAll Rights Reserved.