Battery sync unstable after upgrading to 6.6.x

MVP
MVP
Problem:

Battery sync unstable after upgrading to 6.6.x



Diagnostics:

Collect the server logs by navigating to Administration->Server Configuration->Collect Logs. Extract the logs and from PolicyManagerLogs folder, please click on 

 

 

 

 

From battery.log we can see below errors, that sync to node:172.17.x.x has failed :

 

2017-08-01 12:08:15,641 ERROR  Sync from node:172.17.x.x failed(http)
Traceback (most recent call last):
  File "/usr/local/avenda/platform/battery/lib/battery.py", line 259, in http_sync
    verify=False, timeout=SYNC_TIMEOUT)
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/sessions.py", line 448, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/sessions.py", line 554, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/adapters.py", line 359, in send
    timeout=timeout
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    _pool=self, _stacktrace=stacktrace)
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/packages/urllib3/util/retry.py", line 245, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/packages/urllib3/connectionpool.py", line 304, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/packages/urllib3/connectionpool.py", line 722, in _validate_conn
    conn.connect()
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/packages/urllib3/connection.py", line 195, in connect
    conn = self._new_conn()
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/packages/urllib3/connection.py", line 125, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/usr/lib/python2.6/site-packages/requests-2.4.0-py2.6.egg/requests/packages/urllib3/util/connection.py", line 87, in create_connection
    raise err
ProtocolError: ('Connection aborted.', error(113, 'EHOSTUNREACH'))

 

While doing a traceroute to 172.17.x.x, we see the below:

 

As we see above, the default route seems to happen via 172.17.0.1 which is not the default gateway configured for mgmt or data port in Clearpass

From 6.6.0, there was a new feature introduced for installing extensions and these make use of docker interface. The default IP address assigned for docker interface 172.17.0.1/16.

 

From the above, we see that since the Clearpass server IP is in the range of 172.17.x.x subnet, the packets to this server is being routed via the docker interface and fails to reach Clearpass node:172.17.x.x and hence we see battery unstable error.

 



Solution

Please get in touch with TAC team in order to change the docker IP if the Clearpass IP is in subnet range 172.17.x.x. A valid, non-routable IPv4 address must be supplied (matching 10.0.0.0/8, 192.168.0.0/16, or 172.16.0.0/12), and a valid prefix length must be supplied (>= 16 and <= 29). After the docker IP is changed, the server needs a reboot for effects to take place.

We noticed that post changing the docker IP in the node that shows battery sync unstable to 172.17.x.x IP of Clearpass, the battery sync was established.

 

From battery.log file:

2017-08-09 15:21:08,814 INFO   Sync from node[172.17.x.x] starting

2017-08-09 15:21:09,448 INFO   Connected to node:172.17.x.x for receiving change stream

2017-08-09 15:21:11,208 WARNING pid:38030 put [policyresult] took 0.129 sec

2017-08-09 15:21:11,342 WARNING pid:38031 put [policyresult] took 0.104 sec

2017-08-09 15:22:07,759 INFO   Checkpoint completed in 0.001 sec

 

 

Version history
Revision #:
2 of 2
Last update:
‎01-03-2018 02:43 AM
Updated by:
 
Labels (1)
Contributors
Search Airheads
cancel
Showing results for 
Search instead for 
Did you mean: