Wired Intelligent Edge (Campus Switching and Routing)

Reply
MVP Expert

Aruba 5400R zl2 (KB.16.05.0007) system uptime reset after 500 days of switch uptime

Hi all!

 

Few days ago I discovered that, after about 500 days of system uptime, two of our Aruba 5400R zl2 switches show a system uptime of about 2 days only...as they were just rebooted two days before (which - I'm pretty sure - didn't happen at all).

 

Is anybody able to help me in finding if this (probable) bug was recognized and which exact Software Branch/Versions eventually solved it?

 

Thanks in advance.

 

[*] both Switched were depolyed during 2018 with KB.16.05.0007 software version...yes...I know...they were running an old software version...not easy to find a suitable downtime for maintenance.

 

This post was initially posted on HPE Community (here) but, as of today, no one replied.

Highlighted
Aruba Employee

Re: Aruba 5400R zl2 (KB.16.05.0007) system uptime reset after 500 days of switch uptime

500 days is pretty impressive, even if they did miss some updates in that time!

 

Try a "show logging -a" and see if there is any data past the 2-3 day timeframe. It wasn't clear if the switches had actually restarted, or just the counter has restarted. Depending on the frequency of log entries, you may get a few hours or few months of history. [And that is why it is a good idea to minimise the spurious entries like PVID mismatches where you can.]

 

Ideally you would also have an external NMS (eg IMC, Airwave) that collects SNMP and Syslog info that you could review.



Richard Litchfield, HPE Aruba
Consulting System Engineer
Network Ambassador
MVP Expert

Re: Aruba 5400R zl2 (KB.16.05.0007) system uptime reset after 500 days of switch uptime

Hi Richard,

 

We have IMC and it didn't report anything strange 5 days and 19 hours ago.

 

uptime.png

 

No device unreachablility or relevant alarms...so far.

 

Edit: ...indeed IMC shows a good 502 days of Switch uptime, see below:

 

uptime_real.png

 

Our VSF - see below - has a total uptime just 2 hours shorter. 

 

From what I see/saw I think the uptime counter simply - at some point near the 500 days of switch uptime - just suddenly reset (not the Switch!).

 

Actual show uptime and show ntp outputs:

 

aruba# show uptime
0005:19:46:14.37

aruba# show ntp status NTP Status Information NTP Status : Enabled NTP Mode : Unicast Synchronization Status : Synchronized Peer Dispersion : 0.00000 sec Stratum Number : 3 Leap Direction : 0 Reference Assoc ID : 0 Clock Offset : 0.23849 sec Reference ID : 126.0.0.59 Root Delay : 0.03059 sec Precision : 2**-18 Root Dispersion : 0.89423 sec NTP Up Time : 480d 19h 23m Time Resolution : 419 nsec Drift : 0.00051 sec/sec System Time : Tue Oct 1 12:55:58 2019 Reference Time : Tue Oct 1 12:09:10 2019

Note that strange "NTP Up Time" of 480 days...I checked, just to be sure, and our two NTP Servers (configured on the Aruba) have both 288 days of uptime...so I really don't know what that line actually means.

 

The event - if any - should have happened about 5 days 19 hours and 46 minutes ago from present time)...so around, if my time calculations are correct, last Wednesday 25th late evening / night (CET).

 

On that timeframe the show logging -a command reports nothing interesting:

 

I 09/25/19 02:39:31 04908 ntp: AM1: The system clock time was changed by 0 sec
            208192588 nsec.	 The new time is Wed Sep 25 02:39:48 2019
.
I 09/25/19 02:39:31 04909 ntp: AM1: The NTP Stratum was changed from 3 to 4.
I 09/25/19 02:39:31 04910 ntp: AM1: All the NTP server associations reset.
I 09/25/19 03:41:59 00828 lldp: AM1: PVID mismatch on port E24(VID 6)with peer
            device port up1(VID 16)(694659)
I 09/25/19 03:51:09 00828 lldp: AM1: PVID mismatch on port E23(VID 6)with peer
            device port up1(VID 16)(694664)
I 09/25/19 08:03:53 04908 ntp: AM1: The system clock time was changed by 0 sec
            234348395 nsec.	 The new time is Wed Sep 25 08:04:10 2019
.
I 09/25/19 08:03:53 04909 ntp: AM1: The NTP Stratum was changed from 4 to 3.
I 09/25/19 08:03:53 04910 ntp: AM1: All the NTP server associations reset.
W 09/25/19 09:20:58 04281 dhcp-server: AM1: Unsolicited Echo Reply from
            10.140.100.125.
I 09/25/19 09:26:23 03362 auth: AM1: User 'manager' logged in from 126.1.4.15 to
            SSH  session
I 09/25/19 09:26:25 00179 mgr: AM1: SME SSH from 126.1.4.15 - MANAGER Mode
I 09/25/19 13:11:08 04908 ntp: AM1: The system clock time was changed by 0 sec
            245933331 nsec.	 The new time is Wed Sep 25 13:11:25 2019
.
I 09/25/19 13:11:08 04909 ntp: AM1: The NTP Stratum was changed from 3 to 4.
I 09/25/19 13:11:08 04910 ntp: AM1: All the NTP server associations reset.
I 09/25/19 15:49:59 03362 auth: AM1: User 'manager' logged in from 10.140.2.253
            to SSH  session
I 09/25/19 15:50:01 00179 mgr: AM1: SME SSH from 10.140.2.253 - MANAGER Mode
I 09/25/19 15:50:14 03363 auth: AM1: User 'manager' logged out of SSH  session
            from  10.140.2.253
I 09/25/19 15:54:00 00828 lldp: AM1: PVID mismatch on port E24(VID 6)with peer
            device port up1(VID 16)(695391)
I 09/25/19 16:03:12 00828 lldp: AM1: PVID mismatch on port E23(VID 6)with peer
            device port up1(VID 16)(695396)
I 09/25/19 17:18:33 03363 auth: AM1: User 'manager' logged out of SSH  session
            from  126.1.4.15
I 09/25/19 17:50:00 04908 ntp: AM1: The system clock time was changed by 0 sec
            229283176 nsec.	 The new time is Wed Sep 25 17:50:17 2019
.
I 09/25/19 17:50:00 04909 ntp: AM1: The NTP Stratum was changed from 4 to 3.
I 09/25/19 17:50:00 04910 ntp: AM1: All the NTP server associations reset.
I 09/25/19 19:59:46 03362 auth: AM1: User 'manager' logged in from 10.140.2.253
            to SSH  session
I 09/25/19 19:59:48 00179 mgr: AM1: SME SSH from 10.140.2.253 - MANAGER Mode
I 09/25/19 20:00:00 00131 tftp: AM1: Transfer completed
I 09/25/19 20:00:00 03363 auth: AM1: User 'manager' logged out of SSH  session
            from  10.140.2.253
I 09/25/19 20:00:01 03362 auth: AM1: User 'manager' logged in from 10.140.2.253
            to SSH  session
I 09/25/19 20:00:03 00179 mgr: AM1: SME SSH from 10.140.2.253 - MANAGER Mode
I 09/25/19 20:00:15 00131 tftp: AM1: Transfer completed
I 09/25/19 20:00:15 03363 auth: AM1: User 'manager' logged out of SSH  session
            from  10.140.2.253
I 09/25/19 22:06:06 04908 ntp: AM1: The system clock time was changed by 0 sec
            182248863 nsec.	 The new time is Wed Sep 25 22:06:24 2019
.
I 09/25/19 22:06:06 04910 ntp: AM1: All the NTP server associations reset.
I 09/26/19 02:39:24 04908 ntp: AM1: The system clock time was changed by 0 sec
            215638221 nsec.	 The new time is Thu Sep 26 02:39:42 2019
.
I 09/26/19 02:39:24 04910 ntp: AM1: All the NTP server associations reset.

Our HPE IMC performs backup at 20:00 daily.

 

The very same happened on our VSX which run the same KB.16.05.0007 software version: both nodes show an uptime of only 5 days an 17 hours, again:

 

vsf# show uptime
 VSF Virtual Chassis 
 5d 17h 16m  

 VSF Member 1
 5d 17h 16m  

 VSF Member 2
 5d 17h 23m  

vsf# show ntp status

 NTP Status Information

  NTP Status             : Enabled         NTP Mode        : Unicast         
  Synchronization Status : Synchronized    Peer Dispersion : 0.00000 sec     
  Stratum Number         : 3               Leap Direction  : 0               
  Reference Assoc ID     : 0               Clock Offset    : 0.21387 sec     
  Reference ID           : 126.0.0.59      Root Delay      : 0.03089 sec     
  Precision              : 2**-18          Root Dispersion : 0.92992 sec     
  NTP Up Time            : 493d 22h 22m    Time Resolution : 400 nsec        
  Drift                  : 0.00051 sec/sec 

  System Time            : Tue Oct  1 13:01:53 2019                
  Reference Time         : Tue Oct  1 11:23:59 2019

Note again that strange "NTP Up Time" of 493 days...NTP Servers are the same as above.

 

Probably the "event-threshold" wasn't exactly 500 days...but it was very near to that value (I recall one or two weeks ago to have seen the uptime and said it was very near 500 days...time for maintenance).

MVP Expert

Re: Aruba 5400R zl2 (KB.16.05.0007) system uptime reset after 500 days of switch uptime

Hi Richard, with the help of an HPE Community member I was able to understand what was going on...show uptime uses sysUpTime [**] to poll ticks every 1/100 seconds [*] ("...switch is obtaining the uptime from an SNMP OID (I think it is sysUpTime) which has a 32 bit counter...") so 2 exp 32 means 4294967296 (1/100 second) ticks = 42949672.96 seconds ticks = 497.1 days (1 day = 86400 seconds)...so goodbye good old days where one can see higher uptimes...

 

[*] IMHO 1/100 seconds...is really too much as time resolution for system uptime...probably a resolution of just 1/10 second will be good enough and will leave us with 4971 days as the maximum upper threshold before the counter resets.

 

[**] It should be OID 1.3.6.1.2.1.1.3 "The time (in hundredths of a second) since the network management portion of the system was last re-initialized."

MVP Expert

Re: Aruba 5400R zl2 (KB.16.05.0007) system uptime reset after 500 days of switch uptime

Nice found here: as I discovered that's probably valid for the entire ArubaOS-Switch based portfolio and not only for HP 2910...

Search Airheads
cancel
Showing results for 
Search instead for 
Did you mean: