Network Management

Reply
Contributor I

AMP Device Events retention in AirWave 8.2.7

With the upgrades to ArubaOS-Switch monitoring and management in AirWave 8.2.7, it seems like AMP Device Events are fetched directly from the main event log instead of being retained per device. When the main event log apparently has a hard limit of 1000 lines, which fills up pretty quickly with 240 switches, 780 access points and "Restarting service Client Monitor Worker" popping up every couple of minutes, I have no way of knowing what happened to a downed switch on friday when I get back to work on monday. If something serious happens, which is when I really need to examine the logs, we're talking hours. Access points, Cisco switches and Aruba MAS still retain the last 30 events, regardless of age.

 

Is there any way to store events for HPE Aruba switches per device, or at least massively increase the main event log limit?

Moderator

Re: AMP Device Events retention in AirWave 8.2.7

That 1000 line limit is only applicable for UI display.  If you generate the diagnostics tarball from System -> Status page (link is near top of page), you can extract and view the full AMP events log (path var.log.system.amp_events).  The actual log observes a sizing rotation - some of my lab AMPs haven't hit that mark to rollover yet, with a first entry timestamp from 2 yrs ago, and about 10k lines of entries.


Rob Gin
Senior QA Engineer - Network Services
Aruba Networks, a Hewlett Packard Enterprise Company
Contributor I

Re: AMP Device Events retention in AirWave 8.2.7

Undoubtedly promising, if the diagnostics tarball hadn't failed to generate. System -> Download Log Files -> amp_events, on the other hand, worked just fine, and yielded a log of 3736 lines starting october 27th. That is an artificial limit, as the event log was somehow limited to 10 hours instead of the preset 10 days just after the update. I fixed that, so I'm hoping the actual limit will do the trick for me.

 

That solves the problem with missing data, now I just need it to show up on each device. While the 1000 line UI limit is fine for the main event log, it's less than ideal to manually download and search through the full event log whenever I need to see what happened to a device a couple of days ago, and impossible for support staff with limited access.

 

Is that something TAC might be able to fix, or will it need to be fixed in an upcoming update?

Moderator

Re: AMP Device Events retention in AirWave 8.2.7

I see what you're getting at, great idea.  It'd be a feature request to enable log visibility to specific roles.


Rob Gin
Senior QA Engineer - Network Services
Aruba Networks, a Hewlett Packard Enterprise Company
Contributor I

Re: AMP Device Events retention in AirWave 8.2.7

Are you thinking about downloading the complete event log for non-admins? That would undoubtedly be useful, but doesn't address the issue of events being hidden in device view.

Moderator

Re: AMP Device Events retention in AirWave 8.2.7

I think there's merit in that.  I haven't seen many feature requests about logging visibility.  But adding more logs to that download log section, and adding a role option for log visibility are feature requests that haven't been filed yet in the innovation portal.


Rob Gin
Senior QA Engineer - Network Services
Aruba Networks, a Hewlett Packard Enterprise Company
Contributor I

Re: AMP Device Events retention in AirWave 8.2.7

There's definitely merit to that. I'd love to be able to customize role access, like giving our support staff access to more logs and the troubleshooting tab in the new switch management pages, but I'm guessing that's a bigger, and relatively unrelated, task. What I need now is easy access to my missing device events.

 

I'm also generally worried about only reading from the central event log. Yesterday one of our locations went down a few times due to fiber issues, and its 5 switches and 30 access points quickly generated 1500 log lines, of which 1300 were AP related. If that were to happen with our biggest location, with 46 switches and 179 access points, the 10000 line log would fill in an instant. Just running a firmware upgrade would get pretty close. If it's not an option to go back to storing events per device, something like a separate log for wired devices would go a long way to mitigate that.

Moderator

Re: AMP Device Events retention in AirWave 8.2.7

There should be parsing to display the events per device.  On the Device's monitoring page, there's the option to click to 'Alerts & Events' - that page has a section for Device Events, and if more details is needed, there's the Audit log link at the bottom.  The Audit log is a separate link since it displays more history than the Device Event log.


Rob Gin
Senior QA Engineer - Network Services
Aruba Networks, a Hewlett Packard Enterprise Company
Contributor I

Re: AMP Device Events retention in AirWave 8.2.7

That's not entirely accurate. While the aptly named audit log does go way further back than the event log (so it's apparently still possible), it merely tracks configuration changes, and has nothing to do with device events.

 

Today I experienced how useless the AMP Device Event log has become. I went to investigate a major event that happened sometime tonight but lo and behold, there was next to no information available. The event had apparently caused the event log to max out, and instead of rolling over, it decided to start fresh. So now I'm left with a log of 287 useless lines.

 

To add insult to injury, this occured at 04:02 AM, 13 minutes before the nightly backup, so whichever awkward workaround I try, I'll have no clue as to what happened tonight. That didn't go over well with my boss...

 

Seriously, guys - what gives?

Moderator

Re: AMP Device Events retention in AirWave 8.2.7

Is there a support case opened?  Even if the log rotates, there's no ceiling cap on the log file since it doesn't blow away.  The previous log after rotate should be in a pigz tarball.

 

# ls -sh /var/log/system/*event*
1.5M amp_events
108K amp_events.1.gz
76K amp_events.2.gz
120K amp_events.3.gz
108K amp_events.4.gz

If you're in the CLI:

# unpigz amp_events.1.gz

to expand the previous day's log

 

There should be no circumstance where the log files would mysteriously disappear.


Rob Gin
Senior QA Engineer - Network Services
Aruba Networks, a Hewlett Packard Enterprise Company
Search Airheads
cancel
Showing results for 
Search instead for 
Did you mean: