Wireless Access

I have recently installed 2 x 3400 controllers and programmed master redundancy, this seems to work well.

Both controllers can see each other, hence they become MASTER and BACKUP.

This is all well, but I'm unable to get the database synchronization going, each time I enter the peer-ip (loopback, as the documentation states) of the other controller, the loopback addresses is not available any longer.

Logged into master or standby, I do not get any answer on ping from the loopback of the "other" controller. If I remove VRRP, the loopback is available by ping.

I have read the several treads about using VLAN 1 for VRRP or not, but can't say this should pose a problem like this.

Both controllers running 6.1.3.4

Roar Fossen

You should be able to run the "database synchronize" command from the commandline on the current master.

You can then type "show database synchronize" to see what is happening at the time.

Hi

Well, the sync fails, obviously, with the message that the standby does not ack the start of sync.

Ran the command as Colin suggested and the output:

(Riis_3400_Master) #database synchronize
(Riis_3400_Master) #show database synchronize

Synchronizing database now...  Please wait.
Current state is: "WAITING FOR ACK FROM STANDBY TO START".

(Riis_3400_Master) #show database synchronize

Last manual synchronization time: Wed Sep 12 20:31:15 2012
To Master Switch at 192.168.205.251:  *** FAILED ***
WMS Database backup file size: 0 bytes
Local User Database backup file size: 0 bytes
CPSec Database backup file size: 0 bytes
Total size of RF plan data: 0 bytes, 0 files
Synchronization took 30 second
Last failure cause: Standby switch did not respond to the start request or is not ready

660 synchronization attempted
660 synchronization have failed

Periodic synchronization is enabled and runs every 30 minutes
Synchronization includes RF plan data

I can also post the VRRP setup info:

(Riis_3400_Master) #show vrrp


Virtual Router 1:
    Description Preferred Master
    Admin State UP, VR State MASTER
    IP Address 192.168.205.209, MAC Address 00:00:5e:00:01:01, vlan 1
    Priority 110, Advertisement 1 sec, Preemption Enable Delay 0
    Auth type PASSWORD, Auth data: ********
    tracking type is master-up-time, duration 30 minutes, value 20
    tracked priority 130

and the master-redundancy info

(Riis_3400_Master) #show master-redundancy


Master redundancy configuration:
    VRRP Id 1 current state is MASTER
    Peer's IP Address is 192.168.205.251
    Peer's IPSEC Key is ********

All the interfaces both physical and loopback are in the same subnet, so this should be a fairly easy L2 setup.

The peer IP seen in the master redundancy config (192.168.205.251) is the loopback of the standby.

I'm pretty sure i have done this type of config before without problems

I'm also fairly sure the setup between these to controllers was working earlier, but then i did NOT use the .251 (loopback) but only the vlan 1 interface IP (.208).

Roar

Hi

I have now done several tests. First I erased all master redundancy setup on both controllers, and all VRRP setup as well.

Rebooted both controllers, when they came up again, I could ping across both controllers, both the vlan interface IP and the loopback.

I then programmed VRRP on both controllers and enabled the setup. I could now ping the VRRP IP as well from both controllers. Controllers reported correct status as well, one MASTER and one BACKUP.

As soon as I enter the master redundancy setup (master-vrrp and peer-ip-address) on both controllers, the addresses I have used for the peer are now unavailable.

On the BACKUP controller I get this message in the errorlog:

Sep 12 22:42:27 <stm 304001>  <ERRS> |stm|  Unexpected stm (Station management) runtime error at data_path_handler, 642, data_path_handler: recv - Network is down

I get the same error on the MASTER controller a few seconds later.

Sep 12 22:43:28 <stm 304001>  <ERRS> |stm|  Unexpected stm (Station management) runtime error at data_path_handler, 642, data_path_handler: recv - Network is down

Looks like a bug to me, especially when i'm pretty sure i had it up-and-running on code version 6.1.3.3.

Can not go back to 6.1.3.3 as this version is missing the USB modeswitch parameter under provisioning profile, as we push a 3G profile to several AP groups. On 6.1.3.3 this option is not available, opposed to 6.1.3.4.

Roar Fossen

Are you using the loopback for the peer ip address?  Try to VLAN ip address for the peer and double check your key.

Hi

This is one of the things i tried yesterday. As i erased all the master redundancy and VRRP setup, i used only the interface IP's when doing the master redundancy setup, same problem.

As i enter the peer-ip-address command, i'm suddenly unable to ping interface IP, loopback or even the VRRP IP. Tried rebooting after the change, same problem.

The key has been entered numerous times, to be certain its correct.

The errormessage i included in the last post does not suggest a problem with the key.

I also tried to upload the 6.1.3.4 software on the standby again, but that didn't change anything.

There is also a few other messages in the log of the standby. I ended my session last night around 2300, after that there are a few stranges mesages:

Sep 12 23:08:00  fpapps[1654]: <313328> <WARN> |fpapps|  vrrp: vrid "1" - VRRP state transitioned from INIT to BACKUP
Sep 12 23:08:00  snmp[1594]: PAPI_Send: To: 7f000001:8212 Type:0x4 Timed out.
Sep 12 23:08:00  snmp[1594]: PAPI_Send: To: 7f000001:8212 Type:0x4 Timed out.
Sep 12 23:08:02  certmgr[1434]: <118004> <ERRS> |certmgr|  Received unknown message
Sep 12 23:08:02  certmgr[1434]: <118004> <ERRS> |certmgr|  Received unknown message
Sep 12 23:08:02  cli[1431]: PAPI_Send: To: 7f000001:8372 Type:0x4 Timed out.
Sep 12 23:08:02  publisher[1534]: <306510> <WARN> |publisher|  Dropping message from 8214 for service '51 (service not found)'
Sep 12 23:08:03  KERNEL: 2:<4>process `snmpd' is using obsolete setsockopt SO_BSDCOMPAT
Sep 12 23:08:06  KERNEL: 1:<4>process `trapd' is using obsolete setsockopt SO_BSDCOMPAT
Sep 12 23:08:06  snmp[1594]: IP addr 0xc0a8cdfb port 0
Sep 12 23:08:06  snmp[1594]: trapSrcIp 0x0
Sep 12 23:08:07  authmgr[1566]: PAPI_AddSibyteOpcode: OVERWRITE: Registering NEW call back function for opcode 0x0040 sock = 23
Sep 12 23:08:07  authmgr[1566]: PAPI_AddSibyteOpcode: OVERWRITE: Registering NEW call back function for opcode 0x0040 sock = 23
Sep 12 23:08:07  authmgr[1566]: PAPI_AddSibyteOpcode: OVERWRITE: Registering NEW call back function for opcode 0x004b sock = 63
Sep 12 23:08:07  authmgr[1566]: PAPI_AddSibyteOpcode: OVERWRITE: Registering NEW call back function for opcode 0x004b sock = 63
Sep 12 23:08:07  httpd[1615]: PAPI_Send: To: 7f000001:8214 Type:0x4 Timed out.
Sep 12 23:08:07  publisher[1534]: <306510> <WARN> |publisher|  Dropping message from 8214 for service '51 (service not found)'
Sep 12 23:08:08  l2tp[1617]: PAPI_Send: To: 7f000001:8214 Type:0x4 Timed out.
Sep 12 23:08:08  l2tp[1617]: PAPI_Send: To: 7f000001:8214 Type:0x4 Timed out.
Sep 12 23:08:08  pptpd[1618]: PAPI_Send: To: 7f000001:8214 Type:0x4 Timed out.
Sep 12 23:08:11  cli[1431]: PAPI_Send: To: 7f000001:8372 Type:0x4 Timed out.
Sep 12 23:08:15  cli[1431]: SYSTEM: timezone clock changed from Wed Sep 12 23:08:15 UTC 2012  to Wed Sep 12 23:08:15 UTC 2012
Sep 12 23:08:18  stm[1567]: <304001> <ERRS> |stm|  Unexpected stm (Station management) runtime error at data_path_handler, 642, data_path_handler: recv - Network is down
Sep 12 23:08:18  stm[1567]: <304001> <ERRS> |stm|  Unexpected stm (Station management) runtime error at data_path_handler, 642, data_path_handler: recv - Network is down
Sep 12 23:08:18  stm[1796]: PAPI_Init: timeout of 0 specified set to default 100 millisec.
Sep 12 23:08:18  stm[1796]: PAPI_Init: timeout of 0 specified set to default 100 millisec.

Can anyone make out something from these messages?

Next step will be to open a TAC case if i don't get any good hints :D

Roar Fossen

The way you configured it, it is supposed to work.  Sorry that you are having so many problems.

If after you configure it, you type on the master controller "show switches" and it shows the master and the backup master, it should have enough connectivity to do a database synchronize.  As with master/local relationships that involve ipsec, you cannot always ping between devices, even though connectivity is there.

The error messages themselves are not conclusive, even though they seem that way.

Please open a case and have them take a look, because it should work the way you have it configured.