Wired Intelligent Edge

I've got 6300 that I can't get to boot without errors. VSF process fails and need to press CTRL-C to get to login prompt. When logged in no commands work. Can't erase config or zeroise it. I've tried erasing config from the ServiceOS slice, but still no joy.

 

Any help appreciated. Thanks Jon.

 

Screen output below.

 

 

0. Service OS Console

1. Primary Software Image [FL.10.04.0003]

2. Secondary Software Image [FL.10.04.0003]

 

Select profile(primary): 

 

 

Booting primary software image...

Verifying Image...

Image Info:

 

        Name: ArubaOS-CX

     Version: FL.10.04.0003

    Build Id: ArubaOS-CX:FL.10.04.0003:ff84f1ebd5b2:201911151752

  Build Date: 2019-11-15 10:37:55 PST

 

Extracting Image...

Loading Image...

Done.

kexec_core: Starting new kernel

System is initializing

[FAILED] Failed to start VSF Daemon.

See 'systemctl status vsfd.service' for details.

         Starting PVNET namespace move script...

[  OK  ] Started PVNET namespace move script.

[  OK  ] Created slice system-inbound_nginx_vrf.slice.

[FAILED] Failed to start HPE Credential Manager.

See 'systemctl status hpe-credmgr.service' for details.

[  OK  ] Reached target VSF Discovery System.

[  OK  ] Started HA Type Check Service.

[  OK  ] Reached target Check HA Target to Boot.

[  OK  ] Stopped HPE Credential Manager.

         Starting HPE Credential Manager...

 

^CCreated symlink from /etc/systemd/system/bannerd.service to /dev/null.

Warning: bannerd.service changed on disk. Run 'systemctl daemon-reload' to reload units.

         Stopping bannerd.service...

[  OK  ] Stopped bannerd.service.

[  OK  ] Stopped Emergency Login.

[  OK  ] Created slice system-serial\x2dgetty.slice.

[  OK  ] Created slice system-getty.slice.

[  OK  ] Reached target Network.

         Starting The NGINX HTTP and reverse proxy server...

         Starting Hostname setup service...

[  OK  ] Started Getty on tty1.

         Starting VSF setup recovery node...

[  OK  ] Started VSF setup recovery node.

[  OK  ] Started HPE Credential Manager.

[  OK  ] Started Serial Getty on ttyS0.

[  OK  ] Reached target Login Prompts.

[  OK  ] Reached target Start VSF Recovery Shell.

[  OK  ] Started Hostname setup service.

[  OK  ] Started The NGINX HTTP and reverse proxy server.

 

 

 (C) Copyright 2017-2019 Hewlett Packard Enterprise Development LP

 

                      RESTRICTED RIGHTS LEGEND

 Confidential computer software. Valid license from Hewlett Packard Enterprise

 Development LP required for possession, use or copying. Consistent with FAR

 12.211 and 12.212, Commercial Computer Software, Computer Software

 Documentation, and Technical Data for Commercial Items are licensed to the

 U.S. Government under vendor's standard commercial license.

 

We'd like to keep you up to date about:

  * Software feature updates

  * New product announcements

  * Special events

Please register your products now at: https://asp.arubanetworks.com

 

 

 

**********************************************************************

WARNING! Entering emergency support login mode. This mode is for

support use only and the system will not be fully operational.

The system must be rebooted to restore full operation.

**********************************************************************

6300 login: admin

Password:

 

6300# systemctl daemon-reload

Invalid input: systemctl

6300# erase

  all             Erase all customer data and reset the switch to factory

                  defaults.

  checkpoint      Checkpoint information

  core-dump       Erase daemon or kernel coredump

  startup-config  Contents of startup configuration

6300# erase startup-config

Cannot execute command. Internal error.

6300# erase all

% Command incomplete.

6300# erase all

  zeroize  Perform secure erase when removing customer data.

6300# erase all zeroize

Cannot execute command. Internal error.

6300# 

Hi,

 

Open a case to the TAC...

I'm on a voyage into the unknown here - first ever deployment of Aruba kit and hit this hurdle early on. My provider doesn't appear to have switch on support yet, can I open TAC case directly ?

 

Regards Jon.

You can go on https://asp.arubanetworks.com on case section

Hi,

 

can you post the all the options you have after logging in?

 

Aarón

This is the output when I'm logged in, but none of the commands work,

 

 

**********************************************************************

WARNING! Entering emergency support login mode. This mode is for

support use only and the system will not be fully operational.

The system must be rebooted to restore full operation.

**********************************************************************

6300 login: admin

Password:

 

6300#

  auto-confirm  Disables user confirmation, and executes the operation without

                prompting

  boot          Reboot all or part of the system; configure default boot

                parameters

  checkpoint    Checkpoint information

  clear         Reset functions

  configure     Configuration from vty interface

  copy          Copy data or files to/from the switch

  debug         Configure debug logging

  diagnostics   Change diagnostic commands availability

  disable       Turn off privileged mode command

  end           End current mode and change to enable mode

  erase         Erase device information or files

  exit          Exit current mode and change to previous mode

  https-server  HTTPS Server management

  list          Print command list

  member        VSF member selection

  no            Negate a command or set its defaults

  page          Enable page break

  ping          Send IPv4 ping requests to a device on the network

  ping6         Send IPv6 ping requests to a device on the network

  port-access   Port based network access.

  repeat        Repeat a list of commands from history

  show          Show running system information

  ssh           Configure SSH.

  start-shell   Start Bash shell

  top           Top command

  traceroute    Trace the IPv4 route to a device on the network

  traceroute6   Trace the IPv6 route to a device on the network

  usb           Commands to control the USB Port

  vsf           Virtual Switching Framework (VSF) commands

  write         Write running configuration to memory, network, or terminal

6300#

6300#

6300#

6300#

6300# start-shell

Cannot execute command. Internal error.

6300# wr mem

Cannot execute command. Internal error.

6300#

Oh, I was going to suggest to try start-shell and then run the systemctl command.

 

Definitely open a case with support, that's not an expected behaviour at all.

I've now got a call raised, and will post the solution back here in case anyone runs into the same problem.

Hi! probably I arrive here too late...but have - net of details not provided (as example: are you experiencing this issue on a VSF Member or on a standalone switch?) - you tried to update the switch operating system (ArubaOS-CX) to latest ArubaOS-CX 10.04.0042? ...the running software looks a little bit old (10.04.0003).

 

I think you're doing some confusion between the ServiceOS and the ArubaOS-CX...better if you don't use the ServiceOS if you're still able to work at ArubaOS-CX level.

 

I suggest the step above because I saw you were able to reach the ArubaOS-CX login prompt (#) despite the errors you saw during the normal boot process (see through switch console).

Never too late for help. Thankyou.

 

For more information:-

 

Switch is standalone, with no VSF connections or config. New out of the box.

 

I've not tried updating the code yet, not had chance and unable to get a working system to do that. Will look at that next.

 

In the ServiceOS I am able to use commands. ArubaOS-CX will only boot up in emergency support login mode. Every command at that point generates an error apart from using "?" for help. No commands work at all, not ever reboot.

 

I am not able to get to ArubaOS-CX login prompt without it being in emergency support mode.

 

Have got a TAC case logged, so will hopefully have an answer soon, even if it is a replacement switch.

On this type of issues it's usually recommended to zeroize the unit so you can start with a fresh install but as you mentioned before, you can't even do that so I do believe a RMA will be the best approach.

The answer was indeed to zeroize it, but it needed to be done from within the ServiceOS.

 

Normal caveats apply, use at your own risk etc, I take no responsibility etc.

 

Command was "erase zeroize"  

 

Simple when you know how - support files will be uploaded to identify the root cause. Probably user error on my part somewhere.  

same thing here with 10.07.0004

[ OK ] Started Boot failure triage.
[FAILED] Failed to start HPE Credential Manager.
See 'systemctl status hpe-credmgr.service' for details.
Stopping AAA Utils Cfg Daemon...
[ OK ] Stopped HPE Credential Manager.
Starting HPE Credential Manager...

done this:

recovery-3# erase all zeroize
This will securely erase all customer data and reset the switch
to factory defaults. This will initiate a reboot and render the
switch unavailable until the zeroization is complete.
This should take several minutes to one hour to complete.
Continue (y/n)? y
The system is going down for zeroization.



and the switch comes back OK.

after configure stack links and vsf-renumber and wait for reboot to complete "Failed to start HPE Credential Manager."

For the sake of curiosity, why not trying to update to ArubaOS-CX 10.07.0020 as very first after the zeroization procedure and before any VSF related configuration?

------------------------------
Davide Poletto
------------------------------

Original Message:
Sent: Aug 13, 2021 03:10 PM
From: Unknown User
Subject: OS-CX - Cannot execute command. Internal error.

same thing here with 10.07.0004

[ OK ] Started Boot failure triage.
[FAILED] Failed to start HPE Credential Manager.
See 'systemctl status hpe-credmgr.service' for details.
Stopping AAA Utils Cfg Daemon...
[ OK ] Stopped HPE Credential Manager.
Starting HPE Credential Manager...

done this:

recovery-3# erase all zeroize
This will securely erase all customer data and reset the switch
to factory defaults. This will initiate a reboot and render the
switch unavailable until the zeroization is complete.
This should take several minutes to one hour to complete.
Continue (y/n)? y
The system is going down for zeroization.



and the switch comes back OK.

after configure stack links and vsf-renumber and wait for reboot to complete "Failed to start HPE Credential Manager."

That is a good suggestion but i have manage to solve it within the existing release.

What i found:

1) zeroize the config
2) configure stack links in ring
3) vsf renumber and reboot
4) after the reboot it return with recoveryos due to failed service (ignored)
5) power off the switch, connect stack cables and power-on
6) stack joined and all happy.


What me caught off guard is the failed service and recoveryos, funky stuff if you ask me.
Original Message:
Sent: Aug 13, 2021 06:27 PM
From: Davide Poletto
Subject: OS-CX - Cannot execute command. Internal error.

For the sake of curiosity, why not trying to update to ArubaOS-CX 10.07.0020 as very first after the zeroization procedure and before any VSF related configuration?

------------------------------
Davide Poletto

Original Message:
Sent: Aug 13, 2021 03:10 PM
From: Unknown User
Subject: OS-CX - Cannot execute command. Internal error.

same thing here with 10.07.0004

[ OK ] Started Boot failure triage.
[FAILED] Failed to start HPE Credential Manager.
See 'systemctl status hpe-credmgr.service' for details.
Stopping AAA Utils Cfg Daemon...
[ OK ] Stopped HPE Credential Manager.
Starting HPE Credential Manager...

done this:

recovery-3# erase all zeroize
This will securely erase all customer data and reset the switch
to factory defaults. This will initiate a reboot and render the
switch unavailable until the zeroization is complete.
This should take several minutes to one hour to complete.
Continue (y/n)? y
The system is going down for zeroization.



and the switch comes back OK.

after configure stack links and vsf-renumber and wait for reboot to complete "Failed to start HPE Credential Manager."