Network Management

last person joined: yesterday 

Keep an informative eye on your network with IMC and AirWave network management solutions.
Expand all | Collapse all

AMP Device Events retention in AirWave 8.2.7

Jump to Best Answer
This thread has been viewed 1 times
  • 1.  AMP Device Events retention in AirWave 8.2.7

    Posted Nov 02, 2018 05:32 AM

    With the upgrades to ArubaOS-Switch monitoring and management in AirWave 8.2.7, it seems like AMP Device Events are fetched directly from the main event log instead of being retained per device. When the main event log apparently has a hard limit of 1000 lines, which fills up pretty quickly with 240 switches, 780 access points and "Restarting service Client Monitor Worker" popping up every couple of minutes, I have no way of knowing what happened to a downed switch on friday when I get back to work on monday. If something serious happens, which is when I really need to examine the logs, we're talking hours. Access points, Cisco switches and Aruba MAS still retain the last 30 events, regardless of age.

     

    Is there any way to store events for HPE Aruba switches per device, or at least massively increase the main event log limit?



  • 2.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 05, 2018 10:58 AM

    That 1000 line limit is only applicable for UI display.  If you generate the diagnostics tarball from System -> Status page (link is near top of page), you can extract and view the full AMP events log (path var.log.system.amp_events).  The actual log observes a sizing rotation - some of my lab AMPs haven't hit that mark to rollover yet, with a first entry timestamp from 2 yrs ago, and about 10k lines of entries.



  • 3.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 06, 2018 04:42 AM

    Undoubtedly promising, if the diagnostics tarball hadn't failed to generate. System -> Download Log Files -> amp_events, on the other hand, worked just fine, and yielded a log of 3736 lines starting october 27th. That is an artificial limit, as the event log was somehow limited to 10 hours instead of the preset 10 days just after the update. I fixed that, so I'm hoping the actual limit will do the trick for me.

     

    That solves the problem with missing data, now I just need it to show up on each device. While the 1000 line UI limit is fine for the main event log, it's less than ideal to manually download and search through the full event log whenever I need to see what happened to a device a couple of days ago, and impossible for support staff with limited access.

     

    Is that something TAC might be able to fix, or will it need to be fixed in an upcoming update?



  • 4.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 06, 2018 11:29 AM

    I see what you're getting at, great idea.  It'd be a feature request to enable log visibility to specific roles.



  • 5.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 08, 2018 08:26 AM

    Are you thinking about downloading the complete event log for non-admins? That would undoubtedly be useful, but doesn't address the issue of events being hidden in device view.



  • 6.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 08, 2018 12:30 PM

    I think there's merit in that.  I haven't seen many feature requests about logging visibility.  But adding more logs to that download log section, and adding a role option for log visibility are feature requests that haven't been filed yet in the innovation portal.



  • 7.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 09, 2018 04:15 AM

    There's definitely merit to that. I'd love to be able to customize role access, like giving our support staff access to more logs and the troubleshooting tab in the new switch management pages, but I'm guessing that's a bigger, and relatively unrelated, task. What I need now is easy access to my missing device events.

     

    I'm also generally worried about only reading from the central event log. Yesterday one of our locations went down a few times due to fiber issues, and its 5 switches and 30 access points quickly generated 1500 log lines, of which 1300 were AP related. If that were to happen with our biggest location, with 46 switches and 179 access points, the 10000 line log would fill in an instant. Just running a firmware upgrade would get pretty close. If it's not an option to go back to storing events per device, something like a separate log for wired devices would go a long way to mitigate that.



  • 8.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 09, 2018 10:13 AM

    There should be parsing to display the events per device.  On the Device's monitoring page, there's the option to click to 'Alerts & Events' - that page has a section for Device Events, and if more details is needed, there's the Audit log link at the bottom.  The Audit log is a separate link since it displays more history than the Device Event log.



  • 9.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 12, 2018 05:10 AM

    That's not entirely accurate. While the aptly named audit log does go way further back than the event log (so it's apparently still possible), it merely tracks configuration changes, and has nothing to do with device events.

     

    Today I experienced how useless the AMP Device Event log has become. I went to investigate a major event that happened sometime tonight but lo and behold, there was next to no information available. The event had apparently caused the event log to max out, and instead of rolling over, it decided to start fresh. So now I'm left with a log of 287 useless lines.

     

    To add insult to injury, this occured at 04:02 AM, 13 minutes before the nightly backup, so whichever awkward workaround I try, I'll have no clue as to what happened tonight. That didn't go over well with my boss...

     

    Seriously, guys - what gives?



  • 10.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 12, 2018 03:24 PM

    Is there a support case opened?  Even if the log rotates, there's no ceiling cap on the log file since it doesn't blow away.  The previous log after rotate should be in a pigz tarball.

     

    # ls -sh /var/log/system/*event*
    1.5M amp_events
    108K amp_events.1.gz
    76K amp_events.2.gz
    120K amp_events.3.gz
    108K amp_events.4.gz

    If you're in the CLI:

    # unpigz amp_events.1.gz

    to expand the previous day's log

     

    There should be no circumstance where the log files would mysteriously disappear.



  • 11.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 13, 2018 02:40 AM

    Phew... Good to know that was a bug, not a feature :-) I'll see about contacting TAC to try to find out why and run those log commands for me.

     

    Back to the device event history - have you noted that as a feature request, or do you want me to formally request it somewhere?



  • 12.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 13, 2018 03:12 AM

    Update: I got diagnostics tarball to generate via cli, and it contains complete event, audit, config and command logs for every single device. Seeing as each device page already uses the per-device audit log, it seems logical that it should do the same for events. At least the information is readily available, unhindered by any limits on the central event log.



  • 13.  RE: AMP Device Events retention in AirWave 8.2.7
    Best Answer

    Posted Nov 13, 2018 01:42 PM

    it's best if you can submit the request.  innovate.arubanetworks.com is the portal address.  if you can't access, let me know and i can try - it just looks better when PLM sees that there's an actual customer attached.



  • 14.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 15, 2018 08:11 AM

    Thanks, I'll do that :-)



  • 15.  RE: AMP Device Events retention in AirWave 8.2.7

    Posted Nov 15, 2018 08:53 AM