Network Management

 View Only
Expand all | Collapse all

6300 Firmware update issue

This thread has been viewed 74 times
  • 1.  6300 Firmware update issue

    Posted Sep 10, 2024 05:05 PM

    Hello,

    I am bringing up an issue that I have been having with my 6300s and updating their firmware. If other people have seen this issue let me know. This is probably an issue I need to put a ticket in with support on, but I wanted to get some feedback from here first.

    I am updating some 6300s before I put them into a production environment. The current version that they were on is FL.10.10.1070 and I need to get them to FL.10.13.1040. Per the documentation I need to run the command below. After figuring out the tftp issue in my environment I was able to run this command and not get any errors. 

    ArubaOS-CX 10.05 Fundamentals Guide 6300, 6400 Switch Series - copy {primary | secondary} <REMOTE-URL> (arubanetworks.com)

    I run this command, and the firmware does not update. It stays at version 10.10.1070. Per the release notes for the version, I should be able to go directly to 10.13.1040. Just to eliminate the possible need to step up the versions I download and run the command to bring the version up to 10.12.1050. Here is a screenshot of the result. This was successful

    My next step is to run the command again but upgrade the version to 10.13.1040. After running the command successfully, the version returns back to 10.10.1070 in this screenshot.

    This is the head scratcher. My next step is to log into the WebUI and update the firmware this way. I did that and the version updated without an issue. As you can see here from the WebUI.

    QUESTION:

    This leads me to think that the 10.13.1040 version is setup to only be applied through the web and not able to be applied through the CLI. Is this true? Has anybody else run across this scenario as well? 



  • 2.  RE: 6300 Firmware update issue

    Posted Sep 11, 2024 12:29 PM
    Hi, upgrades between MRs Major Releases (say from 10.10 to 10.13) - which are both LSRs -  should work (doing one single jump without restrictions), upgrades between Major Releases NOT being both two consecutive LSRs admit instead one jump every two MRs at maximum, say an upgrade path admits a jump from 10.11 (SSR) to 10.13 (LSR) - thus avoiding the 10.12 (SSR) - but it does not from 10.11 (SSR) to 10.14 (SSR) since that jump would involve three MRs.
    Never tried the GUI way and never had a single issue updating/upgrading via CLI (since 10.0!)





  • 3.  RE: 6300 Firmware update issue

    Posted Sep 11, 2024 12:43 PM

    I did not think about the LSR and SSR difference. From the release notes I saw that I should be able to go from 10.10.1071 to 10.13.1040 since its later then 10.10.0002. What is funky though is that when I go from 10.12.xxxxx to 10.13.xxxx it jumps back to 10.10.xxxx. I have never seen a system automatically do that unless there was an issue with the new version and I requested or approved going back.




  • 4.  RE: 6300 Firmware update issue

    Posted Sep 12, 2024 02:11 AM

    It looks like you are booting from secondary partition.

    Connect serial/usb console and select primary partition at the bootstrap time.

    Or you can use "boot setdefault primary" to set next boot partition.

    In any case serial console should give you a clue what it is going on. Maybe the primary partition is faulty and need to be reformated or flash need to be replaced.

    Best, Gorazd



    ------------------------------
    Gorazd Kikelj
    MVP Guru 2024
    ------------------------------



  • 5.  RE: 6300 Firmware update issue

    Posted Sep 12, 2024 03:52 AM

    Hi Russ,
    i believe Gorazd is on the right track here.

    You can check the LOG of the Switch with "show logg -a" to see all information since the last reboot. There could be a record of a broken image on one of the filesystem-partitions.

    Also 10.10.x and 10.13.x are both LSR-Releases which should give you a longer lifecycle of the installed software and also features like hot-patching and ISSU (botgh features only on 10.13).

    You can check the firmware-image-checksum also via CLI (Command: show images verify primary or show images verify secondary). Then crosscheck the hash to the published hash on HPE/ANW-Webpage.

    Also check the images after uploading with "show images" and then set default to the new (and correct) firmware (command by goradz).

    Happy troubleshooting

    Franz



    ------------------------------
    prefers to not experiment with live customer equipment...
    ------------------------------



  • 6.  RE: 6300 Firmware update issue

    Posted Sep 23, 2024 06:07 AM

    Unsure where that LSR-SSR story related to upgrades comes from, but as far as I know that applies to compatibility on AOS8 wireless with mobility conductors.

    For switching and Instant, if there are required intermediate steps, these are listed out in the release notes. It also mentions if there is a bootloader/ROM upgrade, which means that the upgrade will take longer (additional reboots).

    Still it's strange/unexpected that when you upgrade the firmware that it jumps back to the old firmware.

    Russ, were you able to upgrade the firmware already?



    ------------------------------
    Herman Robers
    ------------------------
    If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

    In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.
    ------------------------------



  • 7.  RE: 6300 Firmware update issue

    Posted Sep 23, 2024 08:44 AM

    Hi Herman, I don't understand what reply you were referring to when you wrote:

    "Unsure where that LSR-SSR story related to upgrades comes from, but as far as I know that applies to compatibility on AOS8 wireless with mobility conductors."

    but, if I'm not mistaken, this thread is focused on AOS-CX software (not particularly on AOS, at least as far as I understand the whole thread) so the references about AOS-CX SSR/LSR make sense (maybe those references don't provide the OP any solution but they add some information about how AOS-CX software jumps from non consecutive Major Software releases could/couldn't be done).

    If I remember correctly (that's historical...) the single jump from AOS-CX LSR 10.(x) to LSR 10.(x+3) - e.g. from AOS-CX 10.10 to AOS-CX 10.13 - is admitted by HPE Aruba (no matter that the newer LSR 10.(x+3) is three Major Software releases far from the older LSR 10.(x) one), any other scenario instead - IF this restriction is still true today (as I believe) - involving any SSR/LSR (it doesn't really matter) only a two Major Software releases jump is admitted, example we can go from AOS-CX 10.(x) to 10.(x+2) and so we can skip one intermediate Major Software release but we can't go from 10.(x) to 10.(x+3) if both x or x+3 aren't LSR (so, as example, from 10.11 to 10.13 is OK but from 10.11 to 10.14 is not OK).

    Hope the above is still true (AFAIK it was prior to AOS-CX 10.10 era).




  • 8.  RE: 6300 Firmware update issue

    Posted Sep 23, 2024 12:31 PM

    I'm not aware of such a rule about LSR/SSR upgrades AOS-CX (or ArubaOS-Switch). Unless the release notes mention otherwise, you can do direct upgrades to any version in my understanding.

    I've never done any intermediate upgrades on CX or AOS-Switch. So the LSR/SSR status is not relevant for upgrades; unless something is written in the release notes (which normally is not LSR/SSR related). Never heard this related to CX, but I may be wrong, if so, please provide the pointers (I searched but could not find). The +/- 3 versions is valid for ArubaOS 8.x controllers (and maybe AOS10 which can be the source of confusion), but I don't believe it applies anywhere on switching.

    As these forums are seen by many people as 'source of truth', it's good to make sure the information is as accurate as possible.



    ------------------------------
    Herman Robers
    ------------------------
    If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

    In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.
    ------------------------------



  • 9.  RE: 6300 Firmware update issue

    Posted Sep 24, 2024 07:13 AM
    Edited by parnassus Sep 24, 2024 07:36 AM

    Hi Herman! I always added "if I'm not mistaken" or "historically" or some other similar words in my reply implying that I was referring to some sort of learned practice coming from an ancient past.

    I recall AOS-CX Release Notes from the pre-AOS-CX 10.10 era (at a time when the LSR and SSR distinction was embrional to AOS-CX software releases and I simply upgraded sequentially from 10.00 to 10.01 to 10.02 and so on...) where there was a table (now definitly gone or changed?) like this one taken from AOS-CX 10.09.1040 Release Notes (July 2022):

    Since I am a Release Notes collector (I own them since AOS-CX 10.00 ;-) ) I noticed the above table recurring along the years and when I asked (don't remember when and if I asked here on AirHeads or, previously, on the now defunct HPE Networking Forum) and someone at some point confirmed the "rule" that the table implied (from LSR to LSR is OK), so example from 10.04 to 10.06 skipping 10.05 is OK, from 10.07 to 10.09 skipping 08 is also OK, and so on...but any other way (not involving LSRs) was not OK.

    Note that Aruba used the world "must", not "should".

    To be totally honest I used the above recommendations to upgrade our:

    8320 VSX Cluster from 10.05 (SSR) to 10.07 (SSR) skipping one, the 10.06 (LSR) to then 10.10 (LSR) skipping two SSRs, the 10.08 and 10.09

    8360 VSX Cluster from 10.08 (SSR) to 10.10 (LSR) skipping one, the 10.09 (LSR)

    entering for both of them into the "true" LSR era with the 10.10 (yeah, the 10.06 was an LSR but we never tried that Software level for various reasons).

    Ignorantely (like a goat, I would say) - I must admit - I'm keen to follow that "dumb rule" also today in preparing any AOS-CX upgrade procedure...but...I'm speaking about jumping between LSRs so I'm going to play an easy game here (going from 10.10 to 10.13, that's it).

    In any case I learned that current Release Notes (say those of 10.10, 10.13 and 10.14 at least - I didn't check 10.11 and 10.12) changed a lot about that table (on some, a similar table is totally gone!) so, maybe, Aruba silently "lifted the ban" and, as of now, one can do anything with AOS-CX (any to any in upgrading).

    Edit: totally possible that, at least after 2022, the "rule of 2" was eliminated as it was a non intentional restriction due to past AOS-CX releases...and now really, as current RN reports, one can do whatever type of jump with AOS-CX (as reference I rediscovered this, a little bit self-referential...Vincent Giles answered to Elias this "There is no rule of 2 as such (I mean not intentional, but a posterio due to technical constraints, 10.03 to 10.06 being a proof)." I have the mail...even if the thread doesn't have such reply).




  • 10.  RE: 6300 Firmware update issue

    Posted Sep 23, 2024 03:38 PM

    Would there be something in this that I would need to look for?




  • 11.  RE: 6300 Firmware update issue

    Posted Sep 23, 2024 09:14 AM

    Hi @Russ_Altorfer,

    I think that Herman (Robers) is onto something.

    I just wanted to Update an old 6300f from 10.08.x to 10.13.1050. I can do that, but only if i allow "unsafe updates". Maybe this is the reason, why your switch was reset to factory-default in the first place while upgrading. When i allowed those updates, it siad, it will rewrite 6 devices instead of 3 (which gives me a clue).

    Probably you didn't allow those "unsafe updates" (which basicly allows the switch to rewirte it's boot-block and the recovery-Partition!!!). Those updates are "unsafe", because if you mess up (loose power or something else), the device is bricked (can only be revived by HPE-tech or lots of Serial-Cable-Fun with xmodem...). 

    I 'd assume, that if the firmware is to new to be recognized by the boot-loader, than a default-config is beeing loaded to make sure, that the switch is still functional.

    But what i can see from your screenshots is, that you didn't reboot the switch while updating. beforehand you need to switch the default boot to the updated image-partition.

    If you need those commands, just write me an PN.

    Best of Luck to you.

    Franz



    ------------------------------
    prefers to not experiment with live customer equipment...
    ------------------------------



  • 12.  RE: 6300 Firmware update issue

    Posted Sep 23, 2024 10:12 AM

    Hi.

    Unsafe updates are necessary when underlying FW for various HW modules inside the switch need to be updated. Switch OS expect certain behavior from underlaying components to function properly. Hence those FWs are provided with it. When upgrading it is vise to check console output to see, if unsafe updates are needed. If you skip to many versions, (jump too far or did not enable unsafe updates in previous versions) then it is possible that latest version upgrade will not succeed until you allow unsafe updates.

    Best, Gorazd  



    ------------------------------
    Gorazd Kikelj
    MVP Guru 2024
    ------------------------------



  • 13.  RE: 6300 Firmware update issue

    Posted Sep 23, 2024 11:02 AM
    Hi! the allow unsafe-updates approach is, de-facto, a preliminary command to be executed before attempting any AOS-CX update/upgrade procedure (that's the main rule). It is heavily suggested - not really mandatorily required - by Aruba. That's because AOS-CX Release Notes don't declare explicitly if (unsafe) firmware updates to Switch modules are going to be carried out as part of the update/upgrade procedure or not (there is a way to see in AOS-CX if loaded software version will include it or not in order to understand if the allow unsafe-update should be granted or not).





  • 14.  RE: 6300 Firmware update issue

    Posted Sep 23, 2024 03:36 PM

    Thank you everyone for the information. I am able to follow a lot of what everyone is saying. 

    I apply the command 'allow-unsafe-updates 10' as per your recommendations for 10 minutes.

    I then apply the command, which does not show an error.

    After rebooting the switch, the primary partition gets reverted back to 10.10.1071.

    I was hoping that after applying the unsafe updates it would work but it did not. 

    I do have a ticket open with Aruba. I did think it might be with my TFTP server so I tried updating with a USB and it did work. Thinking back though to the TFTP server I was able to update the firmware to a previous version through TFTP with no issue.




  • 15.  RE: 6300 Firmware update issue

    Posted Sep 24, 2024 12:42 PM

    So, an overall update this issue.

    I had a support call today with Aruba, and I was able to reproduce the issue on two different switches. While on the call with them I was able to upgrade the partition to 10.13.1050 without any issue. Even though I was able to get to the version I wanted to via the WebUI and USB initially, this issue has not hindered me with putting these switches into production. I hope that I have provided enough evidence and information to Aruba so that they can troubleshoot the issue. 

    Thank you everyone for the responses and insight. This whole experience has been interesting, and I look at it as a learning experience.




  • 16.  RE: 6300 Firmware update issue

    Posted Sep 25, 2024 10:29 AM

    Hi! you wrote: "While on the call with them I was able to upgrade the partition to 10.13.1050 without any issue." and so, out of curiosity, what you did differently compared to your previous attempts?