@cappali: both controllers are behind NAT, and it seems it can't get past IKE phase 1 when trying to utilize multi-master clustering feature.
@wajih.anees: I was talking about building another IPsec tunnel between the controllers in the previous post. I got the GRE tunnel up, but I didn't notice any improvement in performance.
@cjoseph: Topology looks correct.
CIR Controller A: 75/75 Mbit
CIR Controller B: 100/10 Mbit
The majority of traffic will go from Controller A to Controller B. As Controller B only has an uplink capacity of 10 Mbit, the tunnel manages to fully utilize that bandwidth, so it would be hard to test if the actual limitation is bidirectional.
Just did a new test now with "perf-test server start controller tcp window 256k" (using default values gave similar results) and it produced the following results:
Controller A to Controller B
------------------------------------------------------------
Client connecting to 192.168.0.254, TCP port 5001
TCP window size: 512 KByte (WARNING: requested 256 KByte)
------------------------------------------------------------
[ 40] local 192.168.5.9 port 41453 connected with 192.168.0.254 port 5001
[ ID] Interval Transfer Bandwidth
[ 40] 0.0- 1.0 sec 1.50 MBytes 12.6 Mbits/sec
[ 40] 1.0- 2.0 sec 1.00 MBytes 8.39 Mbits/sec
[ 40] 2.0- 3.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 40] 3.0- 4.0 sec 1.38 MBytes 11.5 Mbits/sec
[ 40] 4.0- 5.0 sec 1.25 MBytes 10.5 Mbits/sec
[ 40] 5.0- 6.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 40] 6.0- 7.0 sec 1.00 MBytes 8.39 Mbits/sec
[ 40] 7.0- 8.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 40] 8.0- 9.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 40] 9.0-10.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 40] 0.0-10.0 sec 11.8 MBytes 9.90 Mbits/sec
Controller B to Controller A
------------------------------------------------------------
Client connecting to 192.168.5.9, TCP port 5001
TCP window size: 512 KByte (WARNING: requested 256 KByte)
------------------------------------------------------------
[ 68] local 192.168.0.254 port 42481 connected with 192.168.5.9 port 5001
[ ID] Interval Transfer Bandwidth
[ 68] 0.0- 1.0 sec 1.38 MBytes 11.5 Mbits/sec
[ 68] 1.0- 2.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 68] 2.0- 3.0 sec 1.25 MBytes 10.5 Mbits/sec
[ 68] 3.0- 4.0 sec 1.25 MBytes 10.5 Mbits/sec
[ 68] 4.0- 5.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 68] 5.0- 6.0 sec 1.25 MBytes 10.5 Mbits/sec
[ 68] 6.0- 7.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 68] 7.0- 8.0 sec 1.25 MBytes 10.5 Mbits/sec
[ 68] 8.0- 9.0 sec 1.25 MBytes 10.5 Mbits/sec
[ 68] 9.0-10.0 sec 1.12 MBytes 9.44 Mbits/sec
[ 68] 0.0-10.0 sec 12.2 MBytes 10.2 Mbits/sec
As you can see they look quite similar, but I've earlier gotten results around ~14mbit which is consistent with the throughput I'm experiencing.
I'm using OpenVPN between the same locations which fully utilizes the CIR both ways. Same using other protocols such as http. Testing each connection with speedtest.net also gives expected results. The ISP's in question are Vikenfiber and Get (resides in Norway).
Best regards,
Daniel