Security

 View Only
last person joined: 22 hours ago 

Forum to discuss Enterprise security using HPE Aruba Networking NAC solutions (ClearPass), Introspect, VIA, 360 Security Exchange, Extensions, and Policy Enforcement Firewall (PEF).
Expand all | Collapse all

Mosyle Clearpass Extension Issue

This thread has been viewed 135 times
  • 1.  Mosyle Clearpass Extension Issue

    Posted Dec 08, 2023 08:26 AM

    We use the Mosyle Clearpass extension to sync all of our Apple devices to the Endpoint Database. It had been working until recently it seems something broke. I keep seeing these errors in the logs and the Endpoint Database does not get updated. 

    I haven't changed anything in the configuration of the extension but after poking around in Mosyle a bit it looks like they made some changes to their API in October. I'm wondering if that's what broke the extension. Has anyone else experienced this issue or have ideas how to fix?
    Here's my configuration for reference:
    Is the Mosyle extension maintained by Aruba or by Mosyle themselves? Not sure who I should open a ticket with if I can't find a solution


    ------------------------------
    Craig Russell
    ------------------------------


  • 2.  RE: Mosyle Clearpass Extension Issue

    Posted Dec 08, 2023 10:24 AM

    Consider Aruba TAC as your point of contact for ClearPass extensions.

    Please open a Support Case with Aruba TAC. 



    ------------------------------
    Herman Robers
    ------------------------
    If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

    In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.
    ------------------------------



  • 3.  RE: Mosyle Clearpass Extension Issue

    Posted Dec 21, 2023 03:52 PM

    Just an update in case anyone else has this issue and because I find the run-around I'm getting humorous...

    • I created a ticket with TAC - they say it's a bug in the extension and I need to contact Mosyle since it's their extension
    • I created a ticket with Mosyle - they made a change to their API that appears to have broken the extension (as was my suspicion). They say the extension is maintained by Aruba

    I just emailed TAC back, we'll see what happens next....



    ------------------------------
    Craig Russell
    ------------------------------



  • 4.  RE: Mosyle Clearpass Extension Issue

    Posted Dec 22, 2023 05:51 AM

    ClearPass extensions are published by Aruba and if there are issue with those, Aruba TAC is your point of contact. Please ask the TAC case to be escalated to engineering with the information that you got from Mosyle about the API changes that broke the extension. Aruba engineering should work with Mosyle if needed.

    Please send me a personal message with the TAC case number and I can see if I can get things moving in the right direction.



    ------------------------------
    Herman Robers
    ------------------------
    If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

    In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.
    ------------------------------



  • 5.  RE: Mosyle Clearpass Extension Issue

    Posted Dec 22, 2023 10:06 AM

    Thanks Herman, I figured Aruba maintained the extension which is why was surprised/annoyed that TAC had me reach out to Mosyle. I've asked Mosyle if there is any documentation they could send that I could forward on to the Aruba engineers. Currently the only info I can find is from inside the Mosyle management site so I can't really send a link to that. I'll send you the TAC ticket number shortly



    ------------------------------
    Craig Russell
    ------------------------------



  • 6.  RE: Mosyle Clearpass Extension Issue

    EMPLOYEE
    Posted Dec 22, 2023 03:30 PM

    Mosyle introduced OAuth authentication for any new integrations that are created after October 9th. Existing integration would continue to work using the token based authentication method which was being used earlier. The APIs for fetching endpoint information has not changed and still continues to work.

    If you were hitting issue with OAuth, the error message would have been something like the one below:

    [2023-12-15T12:13:04.429] [ERROR] Mosyle - Request failed with status code 401
    [2023-12-15T12:13:04.429] [ERROR] Mosyle - { error: 'Unauthorized' }

    The get addrinfo error you are seeing looks like something different. Herman has shared the TAC ticket number and we will advice TAC to take a deeper look.




  • 7.  RE: Mosyle Clearpass Extension Issue

    Posted Dec 22, 2023 03:38 PM

    I found that I had an "https" in the extension config that didn't need to be there, once I removed that I started getting these errors instead:

    [2023-12-19T08:56:20.981] [INFO] Mosyle - Processing ios devices...
    [2023-12-19T08:56:20.981] [INFO] Mosyle - Getting page 1 of devices...
    [2023-12-19T08:56:21.361] [DEBUG] Mosyle - 0232ca76-af35-4c3a-a033-201cd87074da Failed request "POST 'https://managerapi.mosyle.com/v2/listdevices'" took 380 ms.
    [2023-12-19T08:56:21.361] [ERROR] Mosyle - Request failed with status code 401
    [2023-12-19T08:56:21.361] [ERROR] Mosyle - { error: 'Unauthorized' }
    [2023-12-19T08:56:21.363] [ERROR] Mosyle - null
    [2023-12-19T08:56:21.363] [INFO] Mosyle - Processing mac devices...
    [2023-12-19T08:56:21.363] [INFO] Mosyle - Getting page 1 of devices...
    [2023-12-19T08:56:21.962] [DEBUG] Mosyle - bad7ad2d-9bb8-439f-a898-2972d01554cf Failed request "POST 'https://managerapi.mosyle.com/v2/listdevices'" took 598 ms.
    [2023-12-19T08:56:21.962] [ERROR] Mosyle - Request failed with status code 401
    [2023-12-19T08:56:21.962] [ERROR] Mosyle - { error: 'Unauthorized' }
    [2023-12-19T08:56:21.963] [ERROR] Mosyle - null
    [2023-12-19T08:56:21.963] [INFO] Mosyle - Processing tvos devices...
    [2023-12-19T08:56:21.963] [INFO] Mosyle - Getting page 1 of devices...
    [2023-12-19T08:56:22.110] [DEBUG] Mosyle - 4a5ed162-dd8c-40d2-8b71-8b5941aff3bc Failed request "POST 'https://managerapi.mosyle.com/v2/listdevices'" took 147 ms.
    [2023-12-19T08:56:22.110] [ERROR] Mosyle - Request failed with status code 401
    [2023-12-19T08:56:22.110] [ERROR] Mosyle - { error: 'Unauthorized' }
    [2023-12-19T08:56:22.111] [ERROR] Mosyle - null
    [2023-12-19T08:56:22.112] [INFO] Mosyle - Sync complete. Added: 0, Updated: 0, Skipped: 0, No MAC: 0, Errors: 0, Time Taken: 1.242 seconds
    [2023-12-19T08:56:22.117] [INFO] Mosyle - The sync on start full update has completed.
    [2023-12-19T08:56:22.415] [DEBUG] Mosyle - 44e4b209-f88d-42c3-8fdc-716b40e06306 Request "GET '/endpoint/mac-address/004172756261'" took 303 ms.
    [2023-12-19T08:56:22.521] [DEBUG] Mosyle - 8b9531ed-d833-4b84-9c7a-4387f8f28be6 Request "PATCH '/endpoint/mac-address/004172756261'" took 105 ms.
    [2023-12-20T03:00:00.084] [INFO] Mosyle - Stats database cleanup completed.
    [2023-12-21T03:00:00.035] [INFO] Mosyle - Stats database cleanup completed.
    [2023-12-22T03:00:00.033] [INFO] Mosyle - Stats database cleanup completed.


    ------------------------------
    Craig Russell
    ------------------------------



  • 8.  RE: Mosyle Clearpass Extension Issue

    EMPLOYEE
    Posted Dec 22, 2023 04:09 PM

    Okay this looks like the OAuth issue. You can confirm this by checking the authentication type under the token as shown below. Previously type would have been "Basic". JWT token support from extension side is in the works to be available before Feb 2024 when Mosyle will officially cut over to using only JWT tokens. Can you reach out to Mosyle to check if they can switch your account to use Basic authentication until we have an update for the extension?




  • 9.  RE: Mosyle Clearpass Extension Issue

    Posted Dec 22, 2023 04:17 PM

    Honestly if the extension is going to be updated by February that's great, I can wait. That's the answer I've been looking for

    Thanks!



    ------------------------------
    Craig Russell
    ------------------------------



  • 10.  RE: Mosyle Clearpass Extension Issue

    EMPLOYEE
    Posted Dec 22, 2023 04:51 PM

    Great, Will update this thread once the updated extension is available.




  • 11.  RE: Mosyle Clearpass Extension Issue
    Best Answer

    EMPLOYEE
    Posted Feb 05, 2024 04:18 PM

    Updated Mosyle Extension V4 with support for JWT tokens is now available in the store. Both Mosyle Manager and Mosyle Business are going to deprecate support for basic authentication by Feb 8th 2024.

    Tech note available here: https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=dp00003969en_us




  • 12.  RE: Mosyle Clearpass Extension Issue

    Posted Feb 05, 2024 04:27 PM

    Hey Matt, I just happened to check the extension store this morning and saw the update was available. I got it installed and was able to successfully sync with Mosyle. Thanks a ton!



    ------------------------------
    Craig Russell
    ------------------------------



  • 13.  RE: Mosyle Clearpass Extension Issue

    Posted Feb 28, 2024 05:12 AM

    Hi there, I recently updated our Mosyle extension to V4 too because of the API changes. Things were going great but now I'm seeing 429 errors and info complaining about duplicate macs as shown as in the example below. I'm going to make a TAC request but I was wondering if either craigland or others are seeing similar things in their logs since applying the extension update.

    ----

    [2024-02-27T18:18:00.605] [WARN] Mosyle - MAC c4********** belongs to at least 491 devices.
    [2024-02-27T18:18:00.605] [INFO] Mosyle - No updates to 5907 (c4**********) in ClearPass - skipping.
    [2024-02-27T18:18:00.605] [WARN] Mosyle - MAC c42ad0d91eed belongs to at least 420 devices.
    [2024-02-27T18:18:00.605] [INFO] Mosyle - Getting page 492 of devices...  Last page took 351.301461 ms
    [2024-02-27T18:18:00.683] [ERROR] Mosyle - Request failed with status code 429
    [2024-02-27T18:18:00.683] [ERROR] Mosyle - 
    [2024-02-27T18:18:00.683] [ERROR] Mosyle - null
    [2024-02-27T18:18:00.684] [INFO] Mosyle - Processing mac devices...
    [2024-02-27T18:18:00.684] [INFO] Mosyle - Getting page 1 of devices...
    [2024-02-27T18:18:00.765] [ERROR] Mosyle - Request failed with status code 429
    [2024-02-27T18:18:00.765] [ERROR] Mosyle - 
    [2024-02-27T18:18:00.765] [ERROR] Mosyle - null
    [2024-02-27T18:18:00.765] [INFO] Mosyle - Processing tvos devices...
    [2024-02-27T18:18:00.766] [INFO] Mosyle - Getting page 1 of devices...
    [2024-02-27T18:18:00.866] [ERROR] Mosyle - Request failed with status code 429
    [2024-02-27T18:18:00.866] [ERROR] Mosyle - 
    [2024-02-27T18:18:00.866] [ERROR] Mosyle - null




  • 14.  RE: Mosyle Clearpass Extension Issue

    Posted Feb 28, 2024 05:22 AM

    The 429 code means 'Too many requests', so Mosyle is seeing too many request from you (or your IP); or it may be just a busy moment, in which case the error should disappear automatically.

    Could it be that you have more integrations running? It may be best to check with Mosyle support to see what requests they are seeing, which may point you to a misconfigured computer or so. Or in case they reduced the number of allowed requests recently, they either may increase it for you or can point to to a way to stay within the amount of API queries.



    ------------------------------
    Herman Robers
    ------------------------
    If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

    In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.
    ------------------------------



  • 15.  RE: Mosyle Clearpass Extension Issue

    Posted Feb 28, 2024 09:00 AM

    I just started getting this 429 error this morning as well. I installed the latest plugin the day after it came out, and everything was running great until this morning. It kind of looks like when it syncs, it gets a bulk of the endpoints synced without any issue, then part way through it starts getting the 429 error and so it's missing a lot of devices on the latter side of the sync. 

    I actually ran into something similar with an older version of this plugin. I don't know if it was the same, but when I investigated it with Mosyle and Aruba, it turned out to be a coding error that was hitting the limitations of the API and it didn't get resolved until another update to the extension. 

    I might be speaking out of my lane on this next part, but this was my understanding of the issue...I think before it was coded to make several API calls when pulling information. Maybe even a new call per device or something which is inefficient. It needed to be rewritten to make fewer calls, maybe even 1 call, to pull all the information. Maybe something here helps track down the issue. 




  • 16.  RE: Mosyle Clearpass Extension Issue

    Posted Feb 28, 2024 09:10 AM

    This is essentially what I saw too. It was working for a few days after I got it installed recently and now im hitting the 429 api call limits. I've put in a tac request and I will get a support request into Mosyle too in case they recently changed something on their end. 




  • 17.  RE: Mosyle Clearpass Extension Issue

    Posted Feb 28, 2024 08:04 AM

    I am not seeing any errors with my Mosyle extension, so there must be something else going on with yours. Hopefully TAC can help you out



    ------------------------------
    Craig Russell
    ------------------------------



  • 18.  RE: Mosyle Clearpass Extension Issue

    Posted Feb 28, 2024 09:14 AM

    What I described would probably only present itself if it's a larger number of endpoints. How many are you syncing? we're syncing 5,886 devices




  • 19.  RE: Mosyle Clearpass Extension Issue

    Posted Feb 28, 2024 09:50 AM

    I guess I spoke to soon earlier...

    I restarted the extension to initiate a full sync and after a while I started seeing the same thing as you. We have just under 8000 devices in Mosyle



    ------------------------------
    Craig Russell
    ------------------------------



  • 20.  RE: Mosyle Clearpass Extension Issue

    Posted Feb 28, 2024 09:58 AM

    Yeah, that actually sounds like what I'm having too. It seems like it's working at first (because I think it is), until it's hitting some limit. 

    I've had the config syncing for every 5 minutes, but since having this error, I've backed that down to every 30 minutes, but still no change. I'm also opening a ticket with TAC and mosyle. 




  • 21.  RE: Mosyle Clearpass Extension Issue

    Posted Feb 28, 2024 10:32 AM

    Mine is set to only sync once a week (on Saturday). When I looked at my logs earlier it appeared that the last sync worked properly so whatever changed must have happened in the past few days



    ------------------------------
    Craig Russell
    ------------------------------



  • 22.  RE: Mosyle Clearpass Extension Issue

    EMPLOYEE
    Posted Feb 28, 2024 10:46 AM

    Latest version of Mosyle extension only added support for JWT tokens. The APIs themselves have remained unchanged. It seems there might be a API limit added. Please open a ticket with both Aruba and Mosyle TAC.

    Also this error is interesting. There is something causing extension to think MAC address is associated with multiple devices.

    [2024-02-27T18:18:00.605] [WARN] Mosyle - MAC c42ad0d91eed belongs to at least 420 devices.




  • 23.  RE: Mosyle Clearpass Extension Issue

    EMPLOYEE
    Posted Feb 28, 2024 01:36 PM

    @kircher-MASD, the 'belongs to at least' message is how it sounds.  For Mosyle, during a sync event we track every wifi_mac_address and ethernet_mac_address and record how many records from the listdevices API included it.  Most times we see this it is due to a shared hub or dongle used by a lot of people.  Don't recall Apple having dongles/hubs like that but it is in their range.  I would suggest looking at the logs and following up against the devices themselves.




  • 24.  RE: Mosyle Clearpass Extension Issue

    EMPLOYEE
    Posted Feb 28, 2024 01:50 PM

    As for the rate limit, this seems to be on Mosyle's end.  Unfortunately, the Extension at this time does not expose a config to increase the devices per page.  I believe they default it to 100.

    It is unclear if there will be a perceptible change, but to the config you can add:

      "asyncOperationLimit": 1,

    This controls how many Endpoints are processed at a time to CPPM.  Default, and recommended value is 3.  Setting to 1 may extend the time to get out of the rate limiter.  You also want to ensure you have:

      "syncUpdatedOnly": true,

    As this can be a means to have less devices returned in the first place.




  • 25.  RE: Mosyle Clearpass Extension Issue

    EMPLOYEE
    Posted Feb 28, 2024 09:06 PM

    To anyone experiencing the 429, can you look in the Extension logs for a line looking like 'Got X devices to process.'.  The X should be 100, but for at least one of you is 4.  That makes for a 25x spike in API requests to them.  Feel free to let me know.  A private reply is fine.




  • 26.  RE: Mosyle Clearpass Extension Issue

    Posted Feb 29, 2024 09:39 AM

    So for me I see the following in my logs (a snip from the tail end of the logs):

    [2024-02-29T08:09:21.446] [INFO] Mosyle - Getting page 501 of devices...  Last page took 824.341237 ms
    [2024-02-29T08:09:21.648] [INFO] Mosyle - Got 20 devices to process.

    As for the duplicate mac warnings I see it may be because when I enroll my iPads in bulk I do so using a MacOS laptop and then share the wired internet connection to the iPads during the enrollment procedure. This ensures that they connect to Mosyle and that the info reaches Clearpass about an hour later. I then disconnect them from the Macbook and they connect to wifi because Clearpass has gotten its info from the extension. I looked at Mosyle and checked some of the mac address complaining to be duplicates but the devices in question only show their wifi mac address and nothing more. Maybe this is data tied to the iPads that can't be seen from the Mosyle front end GUI?




  • 27.  RE: Mosyle Clearpass Extension Issue

    Posted Mar 01, 2024 03:31 PM

    Anyone else notice that the API calls to Mosyle just started mysteriously working today? First noticed it this morning, and so far it's been working correctly again all through the day. 




  • 28.  RE: Mosyle Clearpass Extension Issue

    Posted Mar 01, 2024 03:41 PM

    Well that's interesting, maybe Mosyle fixed something on their end? My next sync is schedule for tomorrow morning I believe so I won't know if mine's working until then



    ------------------------------
    Craig Russell
    ------------------------------



  • 29.  RE: Mosyle Clearpass Extension Issue

    Posted Mar 01, 2024 03:59 PM

    That's what I'm wondering as well. I think the "official" word from Mosyle is that they didn't change anything but I can't see how that's accurate. I'll be interesting to see what everyone else' sync is doing right now.




  • 30.  RE: Mosyle Clearpass Extension Issue

    Posted Mar 07, 2024 11:14 AM

    I'm stilling seeing 429 errors and Mosyle still has my ticket in monitoring status. TAC seems to think the issue is fixed so I am checking to see if others are still seeing problems. 




  • 31.  RE: Mosyle Clearpass Extension Issue

    Posted Mar 07, 2024 11:39 AM

    I am also still seeing errors on my most recent sync



    ------------------------------
    Craig Russell
    ------------------------------



  • 32.  RE: Mosyle Clearpass Extension Issue

    EMPLOYEE
    Posted Mar 07, 2024 11:56 AM
    Craig, can you do a Collect Logs? Do you have an existing support ticket open? Otherwise we can find a way to get the team the logs.




  • 33.  RE: Mosyle Clearpass Extension Issue

    Posted Mar 07, 2024 11:56 AM

    It is still broken for us.




  • 34.  RE: Mosyle Clearpass Extension Issue

    Posted Mar 07, 2024 12:01 PM

    For whatever reason, ours has been syncing fine. I believe it was one of the developers that hopped on our instance to collect logs, which ended up being pointless because it started mysteriously syncing again at that point. We were hoping to get some failures, but chalked it up as something Mosyle must have quietly tweaked. 

    I checked it again this morning and our sync is completing without errors. I don't believe we've changed anything with the config or the extension. 




  • 35.  RE: Mosyle Clearpass Extension Issue

    EMPLOYEE
    Posted Mar 08, 2024 12:05 PM

    Greetings all.  We just bumped a release 4.1.3.  Anyone on any 4.x release this is seamless.  It should account for these scenarios Mosyle has a mismatch at the end of the API.  Let us know if you have any problems.




  • 36.  RE: Mosyle Clearpass Extension Issue

    Posted Mar 08, 2024 04:09 PM

    I applied the update and it looks like sync is working properly for me now

    Thanks!!



    ------------------------------
    Craig Russell
    ------------------------------