Developer

 View Only
last person joined: 8 days ago 

Expand all | Collapse all

IoT BLE advertisement reception

This thread has been viewed 85 times
  • 1.  IoT BLE advertisement reception

    Posted Mar 29, 2021 10:59 AM
    I am thinking about a tool tracking application built on Aruba devices.  We have thousands of Aruba access points around campus.  For now, they are beaconing so that low-power devices can figure out where they are, but I don't yet know how to receive advertisement beacons.  I started by reading This Document which was a great starting point, but I would like to have a more interactive way to learn.  I tried the HP DEV slack (which has an #aruba room) but it's not very active.  So, I'm hoping I can start the conversation here instead.

    We do not have nor do we plan to use Meridian.  The idea is to develop our own thing specifically geared toward our application.

    Advertisement and scan response frames for unknown devices are stored in raw format, that is, the payload for each frame type is stored as a hexadecimal string, along with the RSSI value. BD maintains a database of IoT devices with attributes populated for known or classified devices and complete advertisement or scan frames for unclassified devices.

    If I understand this correctly, then this means that on each access point runs a background daemon that collects advertisements (beacons) that it hears and stores them in a table, along with their RSSI.  In my case the beacons are going to be in Apple iBeacon format, including a type, UUID, major/minur user-defined IDs, beacon TX power, and RSSI.  The document I quoted above says "Other iBeacons" which I think means it understands them i.e. an iBeacon is a "known device."

    Transport Interval. The transport interval determines how often IoT or BLE data is sent from BD to BR. ArubaOS supports the following transport intervals:Asset tag RSSI data is sent every 4 seconds from each AP to MeridianAruba beacon management data is sent every 30 minutes

    If I understand this, the BR is the BLE Relay, another program that runs on each individual AP.  The BR collects the beacons it hears from the BD process, and, if configured correctly, it can push them somewhere of your choosing: either an HTTPS POST to somewhere, or over a websocket.  This has to be done on each individual access point, in my case that's thousands of them.  They then all individually will HTTP POST their 'heard beacons' tables to an endpoint of my choosing.

    This leads to some questions, based on my limited understanding of how this works.

    First, is the format of these messages defined somewhere I can see (a url etc.)?  I gather that if it is HTTP POST that it's JSON (what's the format?) and if it's via websocket it's protobuf (where can i find the .proto definitions?)

    Second, if I have this right, then I'll need to stand up a server that can handle a LOT of concurrent connections (7,500 access points).  Do I have that right?  Can these APs be configured to only send changes?

    Third, there is some notion of filtering.  That might be useful for me but I think to do that I would have to maintain a list of UUIDs and push them to every single access point.  Do I have that right?  I don't think I can use this but I still would like to know how it works.  It sounds like a useful way to limit traffic, but I might want to limit what users see what (i.e. have multiple whitelists).  Based on that, I think I need to have my own central server receive everything, then filter based on the individual user or client.

    Finally ... if I bought just one aruba access point for home (like an AP22) could I experiment with all of this without being tied to the full campus environment, Aruba Central, etc.?

    --Chris







    ------------------------------
    Christopher Piggott
    ------------------------------


  • 2.  RE: IoT BLE advertisement reception

    EMPLOYEE
    Posted Mar 29, 2021 01:02 PM
    Hi Chris,
    Thanks for posting this in our Dev community!  I've notified one of our IoT/BLE experts to take a look at some of the beacon specific sections. 
    As for your last question, you can set an AP to act as an Instant Access Point (IAP), which wouldn't necessarily need a gateway to function.  IAPs do have different APIs available versus the Central managed APs (these utilize the Central API).

    For some information for the IAP APIs, you can check out the ArubaOS 8.7 REST guide here:
    https://support.hpe.com/hpesc/public/docDisplay?docId=a00101273en_us

    ------------------------------
    Alvin Castro
    ------------------------------



  • 3.  RE: IoT BLE advertisement reception

    Posted Mar 29, 2021 01:08 PM
    > you can set an AP to act as an Instant Access Point (IAP), which wouldn't necessarily need a gateway to function.  

    Does this mean I could ask my IT department to borrow an access point and they could just set it into IAP mode so I could use it standalone?  Does it require a different firmware load, or is it just a configuration change?

    ------------------------------
    Christopher Piggott
    ------------------------------



  • 4.  RE: IoT BLE advertisement reception

    EMPLOYEE
    Posted Mar 29, 2021 04:05 PM
    Hi Chris,

    To begin with, each Aruba AP(all 5xx & 3xx series APs and 2xx series APs by attaching external USB dongles) listen to BLE devices in the environment once you configure and enable the radio profile on the controllers. You can then configure 'Transport Streams' on the controllers to transport the IoT/BLE telemetry data to a websocket server of your choice. 
    Currently we do support HTTP/HTTPS servers as well to send the telemetry data to, but they are soon to be deprecated. So we recommend using websocket only.

    It is good that in your case, the BLE endpoints are going to be iBeacons since we support these and all the attributes like UUID, major/minor, RSSI, battery level etc can be understood and forwarded fully by Aruba controllers.

    All the northbound communication towards the websocket happens through the controllers and not through individual APs. 

    We have dedicated guide for JSON and Protobuf specifications:
    https://asp.arubanetworks.com/downloads;search=iot

    For filtering, you can set filters in the Transport Streams to just listen to iBeacons and ignore rest of the BLE devices. There are a few other filters based on RSSI values, distance and movement as well. You can also set UUID and major/minor values.

    AP22 is Aruba Instant On AP which doesn't support IoT features. If you can grab one of the 5xx series APs or 3xx series IAP, then you could configure them as standalone AP and test the IoT features independently without them being tied to Campus or Central.

    Refer to the 'IoT Basic Setup Guide' to configure the radio and transport profiles on your Aruba device:
    https://asp.arubanetworks.com/downloads;search=iot

    ------------------------------
    Nikunj Naresh Shah
    ------------------------------



  • 5.  RE: IoT BLE advertisement reception

    Posted Mar 29, 2021 06:01 PM
    Thanks, that is all extremely helpful!!!  One follow up question to begin with:

    Currently we do support HTTP/HTTPS servers as well to send the telemetry data to, but they are soon to be deprecated. So we recommend using websocket only.

    OK that's fair.  I can't get to some of those documents because I don't have a support contract (others at my university do, though, so I'll have to ask them).  However, I gather that the websocket interface uses protobuf format exclusively.  That isn't a problem for me.  What's more of a problem is how long-lived these websocket connections live.  Do the devices stay connected indefinitely?  If that's the case, then from a scaling perspective I'll have to have a websocket able to handle 'n' concurrent connections - either 7500 (if it's per AP) or num_controllers (which I don't know what that number is).  Initially it seemed to me that having a large number of controllers sending me updates every 300 or 600 seconds is only a few connections per second (average) which is a pretty low overhead server.  I'm wondering what the server load implications (especially number of concurrent connections) are using websockets.





    ------------------------------
    Christopher Piggott
    ------------------------------



  • 6.  RE: IoT BLE advertisement reception

    EMPLOYEE
    Posted Mar 31, 2021 04:16 AM
    Hi Chris,

    I have created some documentation for a small IoT-Utilities Android App you could use to learn and get used to the Aruba IoT integrations capabilities.
    This also includes a chapter about Aruba IoT concepts that might help you.

    Starting with AOS/Instant 8.8 we add some apHealthUpdate messages though the web socket that keep the connection up. They are sent every 120 s if I am not wrong. As previously stated every Aruba controller configured with an iot transport profile will establish a web socket tunnel. In an Aruba Instant setup every AP will establish a web socket tunnel. There is also an option to use an HTTP web proxy server for the server connectivity.

    If you use BLE telemetry the traffic should be moderate, depending on the reporting interval you set. BLE telemetry reports are a summary of what every APs see. This can be even further filtered using device class filter or even per UUID and so on when using iBeacon.
    Using BLE data forwarding is another story, here all BLE advertisements and scan responses are forwarded. This creates much more traffic depending on your environment. But this can also be filtered by device class.

    I think you don't need a service contract to get to the Aruba IoT server API documents and the protobuf schema. You just need to register an HPE Passport account to sign in to the Aruba Support Portal. Let us know if you can't access the protobuf schema with an HPE passport account.

    I hope that helps.

    Regards,

    Jens

    ------------------------------
    Jens Fluegel
    ------------------------------



  • 7.  RE: IoT BLE advertisement reception

    Posted Mar 31, 2021 08:36 AM
    Thanks, super helpful info.

    Can the android app emulate an AP/IAP to send protobuf test messages to my listening websocket server?


    ------------------------------
    Christopher Piggott
    ------------------------------



  • 8.  RE: IoT BLE advertisement reception

    EMPLOYEE
    Posted Apr 01, 2021 03:43 AM
    No, the App only provides a websocket server.

    ------------------------------
    Jens Fluegel
    ------------------------------



  • 9.  RE: IoT BLE advertisement reception

    Posted Apr 02, 2021 05:34 PM
    Jens,

    ble-opmode scanning beaconing

    Does 'scanning' here mean the API issues active discovery (scan) requests?  In other words, will I be able to receive iBeacons even if my ble-opmode is just beaconing?


    ------------------------------
    Christopher Piggott
    ------------------------------



  • 10.  RE: IoT BLE advertisement reception

    EMPLOYEE
    Posted Apr 02, 2021 06:07 PM
    Hi Chris,

    ble-opmode"scanning" means the AP is listening as for BLE advertisements and scan responses. This way the AP receive e.g. iBeacon around it, adds them to its BLE table and forwards the information in BLE telemetry reports or as raw BLE data to the remote server.
    The ble-opmode "beaconing" the AP's BLE radio is sending iBeacon advertisements itself. 

    Regards,

    Jens


    ------------------------------
    Jens Fluegel
    ------------------------------



  • 11.  RE: IoT BLE advertisement reception

    Posted Apr 02, 2021 03:59 PM

    I have an HPE account but it's not tied to the university account so I can't download those documents.  They may approve (we'll see - I don't work for the IT department).  However, I did find what I think is an older version of the .proto file.  It doesn't have a version number in it, but the document name is HPE_a00058376en_us_Aruba Telemetry API Proto File.proto and everything in it seems to jibe with the sample file, which is great.

    It does not have apHealthUpdate in it anywhere, though, which is what makes me think it's a little out of date.

    It was basically trivial to protoc this file into something that works from go, and from that, make a websocket server that receives telemetry messages and parses the protobuf.  If I were to take out all the comments, then you can make a server that spits out the beacons in 10 or 12 lines of go.  I definitely have some questions about some of the usage, though.  Most of them are pretty simple, and if I had access to hardware I would just try it.  My contact at our ITS dept may have an AP-303h so if he's feeling generous with his time I'm hoping he can use a campus controller to convert it to an IAP and let me borrow it.

    So a basic question about the format: A report has a Reported[] array.  Each item in that array has a mac address (of the found item), then it has a beacons[] array.  In the JSON example, the beacons[] array has a ibeacon object and an eddystone object.  Is it really likely that a single beacon would have both (dual format) ?  Or would it typically be either one or the other?







    ------------------------------
    Christopher Piggott
    ------------------------------



  • 12.  RE: IoT BLE advertisement reception

    EMPLOYEE
    Posted Apr 02, 2021 06:13 PM
    Hi Chris, 

    There are several multi-protocol beacon vendors out there and it is not unusual that beacons can sent multiple protocols/advertisement formats e.g. iBeacon and Eddystone or vendor specific once. Even our Aruba APs can send iBeacon in parallel to AP BLE console advertisements. 
    Multi-protocol beacons are useful in cases where e.g. iBeacon does not provide the required fields in the iBeacon header to encored required data. 

    Regards,

    Jens


    ------------------------------
    Jens Fluegel
    ------------------------------



  • 13.  RE: IoT BLE advertisement reception

    Posted Apr 09, 2021 09:50 AM
    I have come to understand that the AP is actually reporting "advertisements" not "beacons", is that correct?  What if it receives one that it doesn't recognize as iBeacon, Eddystone, EnOcean, or any of the other supported beacons?  Will I still populate that info in beacons: [ ... ] but as hex?  I'm designing my own beacon and I'm looking for a way to send periodic battery level indications.

    [edit]: I can see from the the .proto that's not the case:
    message Beacons {
    optional Ibeacon ibeacon = 1;
    optional Eddystone eddystone = 2;
    // reserved 3;
    }
    I guess the advertisement data could be in vendorData but then there could only be one of those per reported which might not be correct (for example, a single beacon emitting an iBeacon, an eddystone, and a proprietary message).  It looks ilke vendorData is only supported for Enocean and Aruba beacons.  Maybe what I'm looking for isn't possible.


    Along those lines, reported has a field publicKey which makes me think there's some kind of secure advertisement format.  I googled for it but I can't find such a thing.  I'm wondering if that field only shows up if it's one of the proprietary beacon formats.  Can you shed any light on this?

    ------------------------------
    Christopher Piggott
    ------------------------------



  • 14.  RE: IoT BLE advertisement reception

    EMPLOYEE
    Posted Apr 10, 2021 03:32 AM
    If you are developing your own beacon with custom advertisements the BLE telemetry reporting is not the right way using the Aruba IoT integration. Only data of known/supported beacons is reported. For 'unclassified' beacons you only get the BLE MAC address and RSSI information. 

    The right way for yo would be to use the BLE data forwarding service. Here BLE advertisements and scan responses are directly forwarded as raw BLE data packet to the backend system. The APs don't process the BLE packets and the processing of the BLE data would be completely with the backend system. Unfortunately, BLE data forwarding is only supported for known BLE vendors (requiring Aruba SW version 8.8 or higher), which would not be the case if you use a custom BLE vendors id/name, if at all. A workaround here would be that you would send multiple beacon formats from you beacon, e.g. one iBeacon + you custom beacon format. In that case the AP would classify you beacon as iBeacon and could forward every BLE advertisement, regardless of the format. 

    But for you to get the required latest protobuf specification I would suggest you join the Aruba technology partner program here:
    https://www.arubanetworks.com/partners/programs/contact-form/

    After joining the program which includes a partner contract and an NDA we can share all necessary information with you. The program comes with no cost for you and you could get all you questions answered.

    ------------------------------
    Jens Fluegel
    ------------------------------



  • 15.  RE: IoT BLE advertisement reception

    Posted Apr 11, 2021 10:24 AM
    > I would suggest you join the Aruba technology partner program 

    I will try to do that.

    Does Aruba Eddystone support the TLM format?

    /* TLM advertising data: */
    uint8_t advdata_tlm[] =
    {
      0x03,  // Length
      0x03,  // Param: Service List
      0xAA, 0xFE,  // Eddystone ID
      0x11,  // Length
      0x16,  // Service Data
      0xAA, 0xFE, // Eddystone ID
      0x20,  // TLM flag
      0x00, // TLM version
      /* [10] */ 0x00, 0x00,  // Battery voltage
      /* [12] */ 0x80, 0x00,  // Beacon temperature
      /* [14] */ 0x00, 0x00, 0x00, 0x00, // Advertising PDU count
      /* [18] */ 0x00, 0x00, 0x00, 0x00 // Time since reboot
    };


    ------------------------------
    Christopher Piggott
    ------------------------------



  • 16.  RE: IoT BLE advertisement reception

    EMPLOYEE
    Posted Apr 12, 2021 02:56 AM
    Yes, we support Eddystone with telemetry reporting,

    We support
    - Eddytone TLM temperature and voltage
    - EddySTone URL
    - EddyStone UID

    Here are example screenshots from the app.




    ------------------------------
    Jens Fluegel
    ------------------------------



  • 17.  RE: IoT BLE advertisement reception

    Posted Nov 21, 2022 04:36 PM

    Hi Jens,

    Firstly, thanks for your detailed answers in this forum.

    I saw your reply about the BLE data forwarding service and in particular the sentence here:

    "Unfortunately, BLE data forwarding is only supported for known BLE vendors (requiring Aruba SW version 8.8 or higher)"

    I'm trying to get a large Aruba network to forward all BLE data packets for all types of devices,  so essential to get as much data from nearby devices as possible, the scope is e.g. to measure wait times. ( I know about the telemetry interface, but if we can make the BLE data interface work this would have better data for our case) 

    I read the quoted reply as I would have to ask the system to get BLEData for known BLE vendors ... is that via the Vendor Filter ? .. can this filter contain multiple at the same time ? Is there another way to get a system to report BLEData for virtually all devices in the range ?  Can you register your own device class filters, .. and e.g. register a very general one ? 

    Thanks for your support

    Best regards 

    Lars Tørholm




  • 18.  RE: IoT BLE advertisement reception

    EMPLOYEE
    Posted Dec 05, 2022 04:29 AM
    Hi Lars,

    BLE data forwarding is the recommended way to go forward. With ArubaOS/InstantOS 8.9 we have added what is called "generic filters" which allow you to define you own device filters in addition to the existing device class filters. 


    You can define multiple filters, even in combination with existing device class filters in an IoT transport profile. BLE data packets of all matching devices are forwarded to you backend. It is an OR relationship between the filters. With that you should be able to match a lot BLE device types. 
    In order to protect the APs resources there no "forward all" option. Usually customers are only interested in specific BLE data packets for their use cases. 

    You fill find details how the filters work in the latest version of the Aruba IoT Websocket Interface Guide

    I hope that answers your question.

    Regards,

    Jens