Wireless Access

I'm just curious how common the hardware MM is vs. the VA.  I'm considering upgrading my 6.5.x deployment to 8.x this summer, but am leaning strongly towards a hardware MM setup due to the amount of unpleasant hackery that has to be done to vSphere to make the VA happy.

 

Anyone else elect not to use the VA for this reason and go hardware?  If the hardware isn't common I'd likely elect to stand by and wait for the 8.x codebase to continue to revise and hope that maybe a later release will require less ripping and tearing on vSphere's network configuration.  I don't want to get stuck with something that only a small minority are running that doesn't get as much in the way of attention.

I don't see an need to waste the hardware on a MM.  It's only really managing the controllers as far as configs, etc..  It's barely consuming any resources.  I'd hate to waste hardware on something that takes up so littel resources especially if you're going to do a redundancy with dual MM's. The AP's still terminate on the controllers.  Unless you have the resources and rack space to stand up a dedicated hardware MM I'd stick with Virtual.

It's not so much the resources on vSphere that concern me, it's all the nasty stuff you have to do to the network stack to make the VM work, like enabling promiscuous mode and doing LACP through multiple vSwitches and sending all the VLANs up to it that I don't like. 

 

If it were "upload OVA, assign NIC to VLAN, power up" I'd be fine with it.  It's the "enable forged transmits, enable promiscuous mode, trunk it all VLANs" and all that which I'm not okay with.  I have to put a lot of trust in that software not to behave badly.

Just keep in mind that the purpose of MM is more then just pushing configs. MM starts to remove controlplane services from the controllers that terminate AP's like client-match, UCC, ARM (Airmatch), and Airgroup. These services are now centralized by MM vs running independently. 

 

Since these services are centralized, design issues like salt-and-pepper, are no longer a major design issue. Client-match is a chatty process and it performs updates every 3 seconds. This means that the AP's are constantly sending data back to MM so it can be processed. Airgroup is also a chatty protocol and it can be run both centralized or distributed. One of the percs about centralized is that you no longer have to deal with different airgroup domains and you have a single place you need to go to look and review. 

 

Another perc about MM is that there is a loadable service module which allows centralized services to be upgraded if needed. This way your not foced to perform a maint window to apply a patch to fix an issue with client-match. 

 

With all of the controlplane services now starting to get migrated. It really means that MM is much more imporant piece of the enviornment when comparing it to a master thats sole responsiblity was configs, licensing, CPSEC / RAP whitelist, local-userdb. 

 

I'm not hurting for bandwidth, CPU, or any of that in vSphere. My sole issue is the networking requirements that the appliance forces you to set up in your environment. If I had the hardware unit it'd be plug in and power up like my existing controllers. I've read every piece of documentation I can get my hands on and I'm not even sure why the appliance requires half the configuration changes it does.


#AirheadsMobile

Please eloborate on the networking requirements your concerned about. MM doesn't force you to setup a 802.3ad or LACP. There isn't anything stoping you from using a single interface setup as an access port. 

 

Each customer design is different, and one customer may consider an interface failover somewhat "ok" as there is a backup MM server and it will just take over. Some other customers may not want an upstream interface failure to create a failover and thats were port redudnancy may become an important aspect. 

I'm just going by the setup requirements in the documentation. One issue is disabling all the vSwitch security. IMO that's a bit risky, alongside trunking all the VLANs to the appliance, which I'm not fond of either, also for security reasons. Then it says I have to set up two links on two discrete vSwitches, which you say isn't necessary, I have 10 gig vSwitch link redundancy on the host so I should be good there. I won't be using VRRP, my routers are a bit intolerant of it in a "hello broadcast storm, goodbye datacenter" sort of way. I'll probably layer 3 cluster to another VM in my hot site if I can get the first one to work without wrecking the place. There are so many gotchas and caveats in the documentation "just a heads up, do this wrong and we'll take down your host or broadcast storm your network to death" but at the same time being ambiguous and contradictory as to what actually needs to be done. Ive seen a webinar where one data interface is used, but the manual says you need two or it won't work. Maybe I've read too much documentation. It wants three network interfaces, I'm assuming first is management, which is the same as the OOB management on the hardware controllers. If I can disable the third, the second becomes the primary link that the rest of the environment talks to and where the web configuration portal resides? Can I just give it one VLAN and not wory about port group 4095 and tagging traffic? If that's the case, this is really nearly identical to my 7220s, just more or less mapped to a VM?


#AirheadsMobile

 

I cant explain in detail on some of the items you had listed, although if you had concerns with disabling the vSwich security I would recommened opening a TAC case or reaching out to your local SE and asking to validate why. Just because the detail explaination may not be listed in the user-guides it doesn't mean you cant find out why. 

 

If the reasons as to why the features need to be disabled do not fit with your security needs; then you can always fallback to the hardware deployment model. 

 

L2 vs L3 redundancy with MM has its pro's and con's. In this case VRRP is used for only L2. I dont see how you would have an issue with your routers as your not trying to route the traffic. VRRP provides a faster failover as it uses the same concepts as master/redundancy. L3 failover takes around 15-20 minutes to failover, and only sync's its databases every 2 hours (lowest value supported). This includes configs, whitelist, etc. 

 

When you refer to multiple interfaces, its similar to any other virtual networking appliance. Just because the interfaces are required to perform an install, it doesn't mean your forced to use all of them. If you dont use an OOB interface, then just disable it. 

 

When you refer to making the interface as a trunk and tagging all the vlans. Im not sure as to your reasons behind this. You have to remember that the Mobility Master is there for centralization of mgmt-plane and control-plane functionality. The data-plane functionality still exists at the MD's itself. 

 

Services like Wired Airgroup devices would still work the same way by extending the broadcast domain to the MD, or using multicast-aggregation from the AP (L3 vlan that doesn't reside locally). Openflow is used to extend the control-palne functionality on the MM. There isn't a need to extend this vlan to the MM in order to utilize Airgroup. 

 

My recomendation would be to reach out to your local SE and obtain some evals so you can get the gear setup in the lab. VMC's can be used to terminate AP's and you can go through the security features you have concerns about in detail. 

 

The problem is more or less I'm on my own. You are the only person I've gotten any actual information from. Everything else has been manuals, TIDs, and webinar recordings. I stopped when it seemed like things stopped making sense and the setup requirements were going off the rails. I reached out to my account team and got back in essence "I don't know what you're complaining about." I reached out to an HPE partner for help and was told "oh man, you're going for 8? Good luck, we are still doing 6.5 deployments!" So, I'm trying to wing it off the documentation on my own because I haven't been able to get clarification on anything. This was my last attempt to get answers before I gave up and just stayed with 6.5. My boss wasn't keen on the price premium for the hardware appliance unless it was absolutely necessary. The VA installation manuals say trunk the VLANs, so did one of the webinars, so that's what I assumed I had to do. When I get back to the office I can reference the exact document and page. My routers run VRRP for their redundancy and appear to be a bit territorial about it, I had to take it off my controllers because it killed my entire datacenter, so no VRRP. I can't risk that again. My SE said I should be fine without it. I'm a small organization in a very cramped building, my "lab" is a few VLANS carved off my datacenter in the basement, if this is done it's more or less done semi-hot, so I have to be very careful.


#AirheadsMobile

You should only require a single ip address for an MM (not a trunk).  There is an option to port channel more ports for additional bandwidth, but that is not necessary.  You can deploy a VRRP on and MM and give it a password on each side so that it does not conflict with HSRP or any other broadcast-based redundancy.  If you only deploy a single MM, you will not have a way to configure your controllers if it fails.  That is why you deploy a redundant MM.

 

Some people want to deploy an physical appliance because that is how they deploy physical servers.  If you wanted to deploy it on VMWARE, that is fine, as well.  You would only need to allow forged transmits etc. so that the server can see  and send VRRP advertisements, etc.

 

Please let us know if you have any more questions so that we can guide you.

There's a lot of bad information here so let's cover the basics.

 

1. You do NOT need to enable promiscuous mode and forged transmits for ALL vSwitches on the ESX hypervisor to allow the MM to work. If promiscuous mode and forged transmits bother you, you just create a unique port group with promiscuous mode and forged transmits enabled, while the vSwitch that contains that port group has those settings disabled, and you put the VMM into that port group, and things work fine. This is a common requirement for *any* virtualized switch or router and is not unique or nasty or dangerous, and doesn't carry any more risk than anything else deployed when done properly.

 

Note as well, these requirements (promiscuous mode and forged transmits) will likely *never* change and will always be required whenever you virtualize a switching product like our controllers.  

 

2. You do not need to trunk up all VLANs to the MM, the MM only need to be routable from and to the controllers the MM is managing. So the MM in effect only needs one VLAN/network minimum. Now, you CAN trunk up multiple VLANs to the MM depending on your use case, etc. But otherwise, it only need one IP and one VLAN (or two IPs if you are doing VRRP).

 

3. While HMM and VMM offer equivalent capabilities, the VMM has a significant cost savings associated with it. To support 1000 APs with redundant MMs, you only need to purchase one MM-VA-1K license and you can deploy up to four VMMs (2xL2 with L3 between DCs). If you want redundancy with Hardware MM, you have to purchase two MM-HW-1K appliances to support 1,000 APs (or if you want to have four HMMs for 2xL2 with L3 between DCs), then you are paying 4x as much. So VMM *is* the better way to go.

 

4. LACP is never required for any Aruba product, but some people like to. For VMM, LACP is involved when your ESX hypervisor's vSwitch is connected to multiple NICs. VMWare tries to make it easy with NIC Teaming, however, virtual switches with firewalls won't just accept frames from one or the other, so you just have to specify that that vSwitch's multiple NICs be configured between the hypervisor and uplink switch with LACP (instead of NIC teaming with no config on the upstream switch). This is also common when carrying virtualized switching products across a vSwitch with multiple physical uplinks from the hypervisor, and is more efficient.

 

5. In no way should the use of L2 VRRP create any kind of 'storm' on your network. We have VRRP L2 redundancy deployed on networks with 10k APs + carrying over 100k concurrent client connections without issue. L2 VRRP is the primary mechanism for controller clustering as well. This is not a risk unless it's done incorrectly (which then could cause issues sure, if you use the same VIP ID and passphrase for multiple devices instead of a unique VRRP ID and passphrase for each paired devices). 

 

6. The VMM deployes with three virtual interfaces/network adapters. 'Network Adapter 1' maps to the OOB management interface. 'Network Adapter 2' maps to gig0/0/0, and 'Network Adapter 3' maps to gig0/0/1. You can certainly deploy on a single interface (I have multiple MMs on my lab on the same VMware hypervisor on the same vSwitches and all of them are all only using 'Network Adapter 2 mapped to gig0/0/0'. You just disable the network adapters you don't want to use, and don't assign them to any vSwitch (or put them in a null vswitch if it makes you feel better) and just leave one of the gig0/0/0-1 interfaces in whatever vSwitch/port group you want.

 

 

7. Most entities that use or decide on the Hardware MM is because a) their security policy requires physical appliance (think government, military, etc), or b) the network team doesn't work well/trust the virtual infrastructure team, or c) they just prefer hardware appliances regardless of cost.

 

 

If you are a partner, you should have access to whatever training docs you need. As a customer, I agree we could do better with some kind of 'intro/hand-holding how-to-deploy' cookbooks and it's being worked on. But this is a good place to start and there's plenty of people here that can help. However, I will add that the vast majority of your issues and concerns really should not be an issue. promiscuous mode/forged transmits are a requirement. If that is a deal breaker and you just annot do it, even when enabled only in the port group, then hardware is your ONLY option. If you cannot figure out how to run VRRP, that will be an issue regardless of hardware or virtual since we use L2 VRRP for not only MM redundancy, but also cluster redundancy among controllers. So long as your VRRP IDs and passphrases are unique, there should never be any risk of a 'storm' or interruption of service (unless you have some oddity on your network where something is intercepting or bloccking VRRP broadcasts). 

Also, don't forget to provide links or attach the documents you are using to formulate your descision/process. Thanks!

Last one, you can watch multiple videos here 

 

https://www.youtube.com/channel/UCFJCnuXFGfEbwEzfcgU_ERQ/search?query=AOS+8

 

Covering AOS 8 setup, config, operation, etc. This is a site informally run by Aruba SEs and partners (because sadly getting training docs on the official Aruba channel is a nightmare, as I've tried). So the guys that contribute to this channel provide excellent information on multiple topics. The link above is sorted on the 'AOS 8' search.

Just to build a bit on what jhoward2 said...

 

We ended up going with a pair of hardware MM controllers because of option B.  More specifically, our virtualization environment at the time just wasn't up to snuff to handling the controllers.  They've been behaving just fine.

 

As for the  VMware networking stack issues, as he said they are pretty much guaranteed to occur with virtualizing any kind of advanced network appliance.  The vswitch is designed around the assumption that it has complete and correct knowledge of what MAC addresses are present on what virtual ports, which is true for the case of all virtual hardware MAC addresses, as those are assigned by the hypervisor.  Network appliances typically break this is two ways.

 

The first is forwarding traffic.  This obviously entails a ton of (what appear to the vswitch to be) made up addresses coming out of the guest, and what's more, the appliance is expecting the vswitch to forward all of those addresses back to it.  This also applys for devices that have been quiet, meaning that the appliance needs all traffic flooded to it.

 

This obviously isn't the case for the MM, though.  Here, it's more needed simply for the floating VRRP VIP.  Search through the interwebs for VMware and VRRP or CARP (which runs very similarly) and you'll find that it's a very, very long running issue, with no better solution that creating a separate port group with tweaked settings.  I've been running a pair of virtual MM in my lab environment, and once I gave them an appropriate port group, they've behaved themselves since.

I do have to say, 8.x has been a really tough bear to deal with. Lack of documentation for fundamental things. Every single version of the software has had major problems. I upgrade to every single release to fix current problems, only to find it breaks new major things.

 

To touch on the OPs topic, I had the same issue when installing the VM MM. Took me weeks to get real answers on how to deploy it properly.

milleriann,

 

Please be specific, so that others can learn.

Here is the exact order I ended up performing to install my VM MM. This part has been working fine. I only have one MM.

 

1) Single VLAN access port to MM. This VLAN also has to go L2 to MDs.

2) Promiscuous Mode set to "accept" and Forged Transmits set to "accept" on Port Group. You can use a different Port Group for other VMs using the same VLAN so you don't affect other VMs with these settings.

3) NIC 1 and 3 disconnected in VMware. NIC 2 set for Data and Management G-0/0/0.

4) Pre-Allocate Memory - "Reserve all Guest Memory".

5) 8 vCPUs, but 8 CORES on 1 SOCKET (for my sizing). This is a manual change.

6) Boot MM with default RAM and HDD first.

7) Do the initial config of IP address etc.

8) Turn off MM.

9) Upgrade RAM/HDD as needed for your sizing. I used 32 GB RAM and 32 GB HDD on Disk 2. You have to add a NEW VMware HDD that is larger than the OLD HDD disk 2. Boot MM back up and it will auto migrate the data on the drive 2 to drive 3. Once fully booted, remove old drive 2, which you can do with the MM on because it doesn't mount. (keep disk 1 untouched).

 

If there is a better way, someone please share their info.

 

Also, each version of AOS 8.x had major thinks break like APs disconnecting, NTP breaking, TACACS breaking, Clients disconnecting. SAPD issues, VLAN leakage, memory spikes, the cluster breaking, CPSEC errors, client slowness, CoA breaking, etc.

 

The current 8.2.1.0, that is the newest as of today, seems solid on all of the above issues, but AirGroup is completely broken and my users are dead in the water with that piece, right this second.

That's more or less it.

 

If there's issues with AirGroup, please open a TAC case. I can't speak to the other issues without more specifics.