Network Management

I would like to deploy Airwave in a virtualised environment.

I am interested to hear from anyone who operates Airwave on a VM. Do you have any performance issues? Are there any limitations or issues? What resources do you allocate to the VM?

For the deployment I would like to know:

1. Things I should definitely do.

2. Things I should definitely not do.

3. Tips for optimising Airwave in a virtualised environment.

Thanks for your help.

The usage of VM depends on several factors.  What type of deployment do you have? (business, school, etc).  How many devices do you have?  Will this number grow?

Here's a few knowledge base articles that could help:

http://kb.airwave.com/?sid=50140000000MetG - some guidelines when considering virtualizing AMP

http://kb.airwave.com/?sid=50140000000Mf99 - steps for installing on VMware ESXi

There's active support for VMWare ESX and ESXi installations, Hyper-V is not currently supported, but other customers have gotten it to work (one trick I found is that you have to install 3rd party legacy network drivers).

In the lab, I'm running several instances on a AMP-Pro box, but I never reach higher than 200 devices per instance in most of the test runs I perform.  If you're only running a single AMP instance, then it's suggested that you make the system a dedicated server to give AMP as much resources as possible.

Performance-wise, you'll see that the disk i/o usage is very high, that's due to the intensive amounts of read/writes to the database.  The server sizing guide on support.arubanetworks.com suggests you scale an additional 20% resources for a server.  The extra 20% accounts for some VMHost overhead and shared resource distribution.  We've seen a lot of success when the data is off-server on a SAN, but many use the minimum of 15k rpm drives.  The largest deployment I've worked with involved several instances of AMP-Pro (1000 devices per VMinstance), and couldn't grow any larger due to disk utilization.  I've yet to see any setup on VM that exceeded a 1000 device count.

1.  Things to do:

- Definitely make use of the snapshot feature in VMWare

- Make sure you size the VM properly

- Everyonce in a while check for jitter of time/date.  VM-tools is supposed to mitigate this, but doesn't always do so successfully.

- Take backups off of the instance to either the VMHost disk space or to another server / repository.

2.  Things to not do:

- If on a shared host, do not give AMP a low priority (data will reflect the low priority with gaps in graphs, and you'll noticeably see the slow down)

- Do not try to manage the OS, treat AMP as an appliance.

- After initial setup, do not rely heavily on VMware vSphere client as the interface to connect to the CLI.  I personally find the client slower than directly SSHing into the VMinstance.

3.  Optimization:

- You may have to trade off the granularity of data by reducing poll cycles if the disk i/o's climb too high.  This depends mostly on the type of deployment being monitored and number of devices in the deployment.

- You may also have to reduce the number of thread allocated to the system if your trying to run multiple VMs on the same host, and not enough CPUs to maintain all instance.

Support can give you more feedback as to other customers who've run the same.  What we do know is that if you have a high load of single instance users (like an airport or coffee shop hot spot), you may find yourself having a hard time unless you shorten your historical retention periods.  Also, if you have a high number of VisualRF floor plans, you'll see a performance hit there as well.  But these last two items are true across all AMP deployments in general.

My biggest concern was the Airwave sizing guide does not accurately reflect the amount of client data, etc..

If you are going to be monitoring a lot of devices and a lot of client data, make sure you provision a huge HD.

We have run out of diskspace on multiple occasions after following the sizing guide + 20%.

e.g. Monitoring 160-170 Devices with over 1500 clients at the same time.

Diskspace grew to over 400 gb

That's very true.  The guide was to assist self-starters with a good estimation for server size.  Due to the many variations of customer deployments, we had to take the middle ground when sizing for VMs.  My typical recommendation prior to the addition of the 20% clause was to plan a server size higher for VM, so a VM for AMP-Pro (1000) would be sized for AMP-Ent (2500).  That rule didn't always work either though.

Now I have an AMP server monitoring ~2100 devices with an avg of 8-11k clients on the system with a 1.5TB Hard drive, diskspace grew to 1.2TB, had to tune historical data collection but I need to get it lower.

Trying to figure out how, might need to speak to TAC but they always suggest reducing historical data collection values

@pmonardo:

It really depends on your network and clients.  If your network is a school or office, then shortening historical data retention is the way to go.  The minimum would be to age out Inactive Clients more aggressively.  So if a users typically take no longer than a week of vacation, 7-14 days age out would be a good choice.

If your network is like an airport or hotspot where clients are one time only, you may be able to get away with just shortening the 'Client Data Retention Interval'.  This setting determines the size of the files generated for each client.  By default it's set to 425 days, meaning for every client, there's a flat file for 425 days that exists for both bandwidth in and out.  It's recommended for locations that have a high number of one-time clients to keep this file to a smaller size (between 1-14 days), this will redunce the hard disk usage significantly.

Support can provide more specific recommendations based on your network and clients.

---

@rgin wrote:

@pmonardo:

 

It really depends on your network and clients.  If your network is a school or office, then shortening historical data retention is the way to go.  The minimum would be to age out Inactive Clients more aggressively.  So if a users typically take no longer than a week of vacation, 7-14 days age out would be a good choice.

 

If your network is like an airport or hotspot where clients are one time only, you may be able to get away with just shortening the 'Client Data Retention Interval'.  This setting determines the size of the files generated for each client.  By default it's set to 425 days, meaning for every client, there's a flat file for 425 days that exists for both bandwidth in and out.  It's recommended for locations that have a high number of one-time clients to keep this file to a smaller size (between 1-14 days), this will redunce the hard disk usage significantly.

 

Support can provide more specific recommendations based on your network and clients.

---

Thanks Rob.

It is a hotspot actually. What I am worried about is data retention as we need to keep data for 3 years, now we can do so by keeping regular backups of Airwave and offload it but if I need to visit a clients data 30 days ago, I will not be able to I am assuming.
Is this correct?

@pmonardo:

That's correct.  A practice that could help would be to run a report on all clients prior to aging them out, and use the 7.6 feature to export those reports to PDF files regularly (including details for all clients).  In the interim, go through the steps you normally do for tracing a client, trace the data you need and then go through the reports to make sure you're getting exactly the data you need.  The one thing you're not going to get from a PDF report is the ability to sort the tables, so it's a good idea to figure out the easiest way to trace things back when you start aging out data.

The other option that a few customers have done is to add a SAN to offload the data from the AirWave server.  SANs can hold more storage space and sustain the correct IOPs for AirWave, and might be the right solution for you to look into.

perfect thank you for the detailed response

Don't suppose you could go into any detail on those third party legacy drivers you speak of for Hyper-V?

I can now thanks to Aaron from the Network Services QA team, here are the steps...

~~~~~

Get Hyper-V to connect to network

In Hyper-V, when you first go through the steps to create a VM and try to boot off the ISO, you may get an error saying eth0 not found.  If this is the case, then shut down / turn off the instance.

1.  Go into the instance's settings

2.  Under Hardware -> select Add Hardware

3.  Choose 'Legacy Network Adapter'

4.  You should see a new network adapter populate, select the new adapter

5.  Make sure adapter points to the Hyper-V Virtual Network (may have another name depending on how the host was configured)

6.  Apply and start up the instance, the CentOS portion of the install should work

Excellent Rob, thank you.  Going to do a remote session with my customer and give it a shot!  Will let you know the results