SD-WAN

I have HP3800 switches configured as OpenLFow 1.3 SDN switches (firmware version KA.15.15.0006).

In OpenFlow SDN mode, If the switches reveive mutlicast packets, they broadcast the packets as per a traditional switch, without the SDN controller adding any rules in the flow tables.

If I direct the mutlciast stream to mulitple ports on the same switch (using flow rules) then remove a port, packets are broadcast on other ports for approx 0.1ms, even when there is a lower priority drop packet flow rule.

In SDN OpenFlow mode, this shoudl not happen.

Has this been seen before? Has it been fixed in newer firmware versions?

First off, that switch is running a very old firmware version (~ 2 years).  Please try updating the 3800 to a newer firmware.  KA.16.01.0007 would be a good target, download link below:

https://h10145.www1.hpe.com/downloads/SoftwareReleases.aspx?ProductNumber=J9573A

What SDN Controller are you using?  Is it running in Hybrid Mode or Full Openflow Mode?  Is the Switch to Controller communication using OpenFlow version 1.0 or 1.3?

Hybrid - Packets that do not match OpenFlow pipeline are forwarded normally through the swtich. This is using the OpenFlow forward normal rule.

Full OpenFlow - Packets that do not match and OpenFlow flow in the OpenFlow pipeline are forward to controller for a decision if the flow is allowed / disallowed. 

I'm using HP VAN 2.0 in full OpenFlow mode. The switches are setup for OF 1.3, but the not using any of the OF 1.3 functionality. To implement multciast I am just adding or removing portd from the actio output port list.

I'm not using hybrid mode / any forward normal commands.

Hi Dave,

In this situation, it would help if you were able to post the following :

1. Before you removed the port (screenshot from VAN GUI, or "show openflow instance X flows" on 3800)
2. Example JSON of the flow you're pushing to remove the port (or the java flowmod captured in OpenFlow Trace)
3. After you removed the port (screenshot from VAN GUI, or "show openflow instance X flows" on 3800)

With this additional data, I think we'll be able to narrow in on what's going wrong. From your description above I'm confused as to whether you're saying data goes out the original port that was removed, or whether it goes out ports which were never in the multicast group to begin with.

I can send the following as .pdf if needed>

UDP Broadcast Using HP3800 Switch in OpenFlow Mode
D Butler
19 July 2016

1 Introduction
This work is part of investigation into IP Production on a SDN using Source Specific Multicast for live TV production.

2 Test Setup
The HP3800 switches with version KA.15.15.0006 firmware, configured in OF 1.3 mode, are connected in 4 switch mesh, as shown in Figure 2.1. A second, traditional, network using a separate IP range is employed to connect the SDN controller and SDN switches. The traditional network is also employed for SSH access to the source/ destination servers.

Figure 2.1: Test Network Setup

A source on one server streams UDP test packets or SSM video, using IPERF or VLC, which are received by one or more destination servers.
e.g. Using IPERF on the source server 10.0.0.100, to generate a 800Mb/s UDP stream:
$ iperf –u –p 5004 –c 232.13.1 –b 800M –t 86400
------------------------------------------------------------------
Client connecting to 232.1.3.1, UDP port 5004
Sending 1470 byte datagrams
Sending multicast TTL to 1
UDP buffer size: 208 KByte (default)
------------------------------------------------------------------
[ 3] local 10.0.0.100 port 3745 connected with 232.1.3.1 port 5004
[ID] Interval Transfer Bandwidth
[ 3] 0.0–24.8 sec 2.32 GBytes 803 Mbits/sec
[ 3] Sent 1692709 datagrams
$
e.g. Using VLC to stream a video file:
$ cvlc -vvv big.mov --sout "#rtp{proto=udp,mux=ts,name=big,sdp=sap,dst=232.1.6.1,port=5004}" --sout-keep --ttl 5 --loop

Destinations join and leave SSM streams using IGMPv3 messages which are processed by the 10.0.0.110 server, which in programmes flows using the REST interface on the HP VAN 2.0 SDN controller.

At the destinations, a simple multicast client counts dropped / repeated / out-of-order packets using the sequence number in the IPERF UDP packet payload:
$ ./ssmmeter 10.0.0140 10.0.0.100 232.1.3.1

Video streams are received and played using VLC:
$ vlc rtp://10.0.0.100@232.1.3.1:5004

A network tap and Wireshark are employed for investigating anomalous behaviour.

3 Anomalous Switch Behaviour
The HP3800 switches with version KA.15.15.0006 firmware have shown anomalous behaviour when modifying flows. The specific test setup is as show in Figure 3.1.

Figure 3.1: Test Setup Showing Anomalous Network Behaviour

If servers Dest1 (on SW2) and Dest2 (on SW4) have joined a multicast stream from Source1 (on SW1), when Dest2 leaves the stream, there are no repeat packets received by Dest1.
However, if servers Dest1, Dest2 (on SW4) and Dest3 (SW4) have all joined a multicast stream from Source1 (on SW1), when only Dest2 leaves the stream, repeat packets are received by Dest1.

The equivalent curl commands for the switches are to receive the stream at Dest1, 2 and 3 are:
SW1> $ curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.200:8443/sdn/v2.0/of/datapaths/00:02:c8:cb:b8:3e:fe:40/flows" -d "{\"flow\": {\"priority\": 20000,\"table_id\":100,\"idle_timeout\": 60000,\"match\": [{\"ipv4_src\":\"10.0.0.100\"},{\"ipv4_dst\":\"224.1.3.1\"},{\"eth_type\": \"ipv4\"}],\"instructions\": [{\"apply_actions\": [{\"output\": 25}, {\"output\": 26}]}]}}" --request POST
SW2> $ curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.200:8443/sdn/v2.0/of/datapaths/00:02:6c:3b:e5:62:b2:80/flows" -d "{\"flow\": {\"priority\": 20000,\"table_id\":100,\"idle_timeout\": 60000,\"match\": [{\"ipv4_src\":\"10.0.0.100\"},{\"ipv4_dst\":\"224.1.3.1\"},{\"eth_type\": \"ipv4\"}],\"instructions\": [{\"apply_actions\": [{\"output\": 17}]}]}}" --request POST
SW3> $ curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.200:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/flows" -d "{\"flow\": {\"priority\": 20000,\"table_id\":100,\"idle_timeout\": 60000,\"match\": [{\"ipv4_src\":\"10.0.0.100\"},{\"ipv4_dst\":\"224.1.3.1\"},{\"eth_type\": \"ipv4\"}],\"instructions\": [{\"apply_actions\": [{\"output\": 11}, {\"output\": 13}]}]}}" --request POST

Screen shots of the OpenFlow Monitor in the SDN Controller are shown in Figures 3.2, 3.3 and 3.4. The flows for 224.0.0.1 and 242.0.0.2 are for IGMP messages. The lower priority flow for 232.0.0.0/24 is to drop unjointed streams.

Figure 3.2: OF Monitor View for SW1

Figure 3.3: OF Monitor View for SW2

Figure 3.4: OF Monitor View for SW4

When Dest2 leaves the stream, port 13 is removed from the flow in SW4. All other flows are unchanged. The equivalent curl commands for the switches are to receive the stream at Dest1 and 3 are:
SW1> $ curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.200:8443/sdn/v2.0/of/datapaths/00:02:c8:cb:b8:3e:fe:40/flows" -d "{\"flow\": {\"priority\": 20000,\"table_id\":100,\"idle_timeout\": 60000,\"match\": [{\"ipv4_src\":\"10.0.0.100\"},{\"ipv4_dst\":\"224.1.3.1\"},{\"eth_type\": \"ipv4\"}],\"instructions\": [{\"apply_actions\": [{\"output\": 25}, {\"output\": 26}]}]}}" --request POST
SW2> $ curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.200:8443/sdn/v2.0/of/datapaths/00:02:6c:3b:e5:62:b2:80/flows" -d "{\"flow\": {\"priority\": 20000,\"table_id\":100,\"idle_timeout\": 60000,\"match\": [{\"ipv4_src\":\"10.0.0.100\"},{\"ipv4_dst\":\"224.1.3.1\"},{\"eth_type\": \"ipv4\"}],\"instructions\": [{\"apply_actions\": [{\"output\": 17}]}]}}" --request POST
SW3> $ curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.200:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/flows" -d "{\"flow\": {\"priority\": 20000,\"table_id\":100,\"idle_timeout\": 60000,\"match\": [{\"ipv4_src\":\"10.0.0.100\"},{\"ipv4_dst\":\"224.1.3.1\"},{\"eth_type\": \"ipv4\"}],\"instructions\": [{\"apply_actions\": [{\"output\": 11}]}]}}" --request POST

The Screen shot of the OpenFlow Monitor for SW4 is shown in Figures 3.5. Others are unchanged.

Figure 3.5: OF Monitor View for SW4 (after port removal).

Using a network tap and Wireshark, as shown in Figure 3.1, repeat packets are detected on the link SW4 to SW2. The Wireshark capture results are shown in Figure 3.6.

Figure 3.6: Wireshark Capture of Repeat Packets

As shown in Figure 3.7, the number of repeated packets increases with stream bit rate (from 10Mb/s to 900Mb/s). The duration of repeated packets is around 0.1ms for all stream rates.

Figure 3.7: Repeated Packets for Live Flow Movement using Individual Priorities

In a traditional Layer 2 network switch, multicast packets are broadcasts on all ports. In a SDN switch, packets should not be broadcast without a flow table rule to implement this. Packets should be forwarded to the SDN controller, which should then programme flows.
For a period of 0.1ms, it appears that the SDN switch is directing packets as a traditional switch.
When 1 of the 2 destinations on SW4 (Dest2 or Dest3) leaves the multicast stream, a burst of packets are broadcast on the SW4 links to SW2 and SW3. These are detected using a network tap and Wireshark.
The flow table rule for the multicast stream in SW2 direct these packets to Dest1, so the destination receives a burst of repeated packets.
Since the flow table rule for the multicast stream does not specify an incoming switch source port, incoming packets from port 7 and port 5 are delivered to Dest1 via port 17.
The rule used to drop packets from un-joined multicast streams is not applied, as it is at a lower priority.

Note:

The anomalous broadcast behaviour in the switch occurs when a flow rule is modified to remove a port from one of multiple output ports for an active stream.
If no flow rules are programmed in any switch, UDP packets are broadcast.
If the IGMP Proxy Application in server 10.0.0.110 is not running and there are no flows programmed or visible in the SDN VAN controller OF Monitor, then UDP packets are broadcast.
When packets are stream using IPERF from server 10.0.0.100:
1. Broadcast packets are detected using Wireshark and the network tap.
2. No flow rules are visible in the SDN VAN controller OF Monitor (the controller does not appear to add any rules of its own).
3. If ‘drop’ packets rule is programmed in SW4, using a curl command, the broadcasts detected by Wireshark stop.

4 SW4 Configuration and Flows

HP-3800-48G-4SFPP-4# show config
Startup configuration: 7

; J9576A Configuration Editor; Created on release #KA.15.15.0006
; Ver #05:19.ff.ff.3f.ef:cc

hostname "HP-3800-48G-4SFPP-4"
module 1 type j9576y
module 2 type j9576x
interface 49
flow-control
exit
snmp-server community "public" unrestricted
openflow
controller-id 1 ip 192.168.40.200 controller-interface oobm
instance "sdnm"
member vlan 2
controller-id 1
version 1.3 only
mode passive
enable
exit
enable
exit
oobm
ip address 192.168.40.74 255.255.255.0
exit
vlan 1
name "DEFAULT_VLAN"
no untagged 3-52
untagged 1-2
ip address 192.168.40.34 255.255.255.0
exit
vlan 2
name "VLAN2"
untagged 3-52
no ip address
exit
no autorun
no dhcp config-file-update
no dhcp image-file-update

HP-3800-48G-4SFPP-4# show openflow instance sdnm flows
OpenFlow Flow Table

Flow 1
Match
Incoming Port : Any Ethernet Type : Any
Source MAC : Any Destination MAC : Any
VLAN ID : Any VLAN priority : Any
Source Protocol Address : Any
Target Protocol Address : Any
IP Protocol : Any
IP ECN : Any IP DSCP : Any
Source Port : Any Destination Port : Any
Attributes
Priority : 0 Duration : 5270018 seconds
Hard Timeout : 0 seconds Idle Timeout : 0 seconds
Byte Count : 0 Packet Count : NA
Flow Table ID : 0 Controller ID : NA
Activity Count: NA Cookie : 0x0
Hardware Index : NA
Instructions
Goto Table ID : 100

Flow 2
Match
Incoming Port : Any Ethernet Type : IP
Source MAC : Any Destination MAC : Any
VLAN ID : Any VLAN priority : Any
Source Protocol Address : Any
Target Protocol Address : 224.0.0.1/32
IP Protocol : Any
IP ECN : Any IP DSCP : Any
Source Port : Any Destination Port : Any
Attributes
Priority : 20000 Duration : 6442 seconds
Hard Timeout : 0 seconds Idle Timeout : 0 seconds
Byte Count : NA Packet Count : 21
Flow Table ID : 100 Controller ID : 1
Activity Count: NA Cookie : 0x0
Hardware Index : 17
Instructions
Apply Actions
Output : 11
Output : 13

Flow 3
Match
Incoming Port : Any Ethernet Type : IP
Source MAC : Any Destination MAC : Any
VLAN ID : Any VLAN priority : Any
Source Protocol Address : Any
Target Protocol Address : 224.0.0.22/32
IP Protocol : Any
IP ECN : Any IP DSCP : Any
Source Port : Any Destination Port : Any
Attributes
Priority : 20000 Duration : 6440 seconds
Hard Timeout : 0 seconds Idle Timeout : 0 seconds
Byte Count : NA Packet Count : 10
Flow Table ID : 100 Controller ID : 1
Activity Count: NA Cookie : 0x0
Hardware Index : 18
Instructions
Apply Actions
Output : 5

Flow 4
Match
Incoming Port : Any Ethernet Type : IP
Source MAC : Any Destination MAC : Any
VLAN ID : Any VLAN priority : Any
Source Protocol Address : 10.0.0.100/32
Target Protocol Address : 232.1.3.1/32
IP Protocol : Any
IP ECN : Any IP DSCP : Any
Source Port : Any Destination Port : Any
Attributes
Priority : 20000 Duration : 3234 seconds
Hard Timeout : 0 seconds Idle Timeout : 0 seconds
Byte Count : NA Packet Count : 26891052
Flow Table ID : 100 Controller ID : 1
Activity Count: NA Cookie : 0x0
Hardware Index : 19
Instructions
Apply Actions
Output : 11

Flow 5
Match
Incoming Port : Any Ethernet Type : IP
Source MAC : Any Destination MAC : Any
VLAN ID : Any VLAN priority : Any
Source Protocol Address : Any
Target Protocol Address : 232.0.0.0/8
IP Protocol : Any
IP ECN : Any IP DSCP : Any
Source Port : Any Destination Port : Any
Attributes
Priority : 19999 Duration : 6444 seconds
Hard Timeout : 0 seconds Idle Timeout : 0 seconds
Byte Count : NA Packet Count : 8
Flow Table ID : 100 Controller ID : 1
Activity Count: NA Cookie : 0x0
Hardware Index : 0
Instructions
Drop

Flow 6
Match
Incoming Port : Any Ethernet Type : Any
Source MAC : Any Destination MAC : Any
VLAN ID : Any VLAN priority : Any
Source Protocol Address : Any
Target Protocol Address : Any
IP Protocol : Any
IP ECN : Any IP DSCP : Any
Source Port : Any Destination Port : Any
Attributes
Priority : 0 Duration : 4915674 seconds
Hard Timeout : 0 seconds Idle Timeout : 0 seconds
Byte Count : NA Packet Count : 23613373
Flow Table ID : 100 Controller ID : 1
Activity Count: NA Cookie : 0x0
Hardware Index : NA
Instructions
Goto Table ID : 200

Flow 7
Match
Incoming Port : Any Ethernet Type : Any
Source MAC : Any Destination MAC : Any
VLAN ID : Any VLAN priority : Any
Source Protocol Address : Any
Target Protocol Address : Any
IP Protocol : Any
IP ECN : Any IP DSCP : Any
Source Port : Any Destination Port : Any
Attributes
Priority : 0 Duration : 4915673 seconds
Hard Timeout : 0 seconds Idle Timeout : 0 seconds
Byte Count : 31997721733 Packet Count : 21873745
Flow Table ID : 200 Controller ID : 1
Activity Count: NA Cookie : 0x0
Hardware Index : NA
Instructions
Apply Actions
Controller Port

HP-3800-48G-4SFPP-4#

I updated the switch firmwae to KA.16.01.007 and updated the controller to SDN VAN 2.7.16 and it still does the same:

1)    UDB Broadcasts when no flow table rules.

2)    0.1ms of UDP Broadcasts when removing port from flow rule.

BTW: The license transfer website at https://h10145.www1.hpe.com/license/TransferSearch.aspx does not work!

Dave.

Hi Dave,

Thank you for the additional information and further detail. I will check with the team on both of these items.

Cheers,

-Scott

Hi Dave,

There are two options for your scenario to work without the packet leaking you witnessed. First option is to use the recently released switches based on our 3rd generation SDN capable ASIC; 5400R with v3 modules, 3810M, 2930F as there is an architecture change around this specific scenario that changes the behavior. Second option is to use OpenFlow groups instead of multiple output ports in your rules, the output port changes will be made by increasing, decreasing and changing the group buckets. This will work on the current 3800 and the newer switches mentioned above.

More details on our Group table implementation is available in the OpenFlow Admin Guide (16.02 manual - http://h20566.www2.hpe.com/hpsc/doc/public/display?docLocale=en_US&docId=emr_na-c05161702 – Section 4)

Below are a few examples on VAN to create/modify Groups.

Best Regards,
-Scott

DPID in the examples below is: 00:14:88:51:fb:35:1a:40, Controller IP in the examples is 10.15.155.61

Create Group with two buckets, each with one port:

POST: https://10.15.155.61:8443/sdn/v2.0/of/datapaths/00:14:88:51:fb:35:1a:40/groups

{
"version": "1.3.0",
"group": {
"id": 1,
"type": "all",
"command": "add",
"buckets": [
{
"weight": 0,
"watch_group": 4294967295,
"watch_port": "ANY",
"actions": [
{"output": 14}
]
},
{
"weight": 0,
"watch_group": 4294967295,
"watch_port": "ANY",
"actions": [
{"output": 15}
]
}
]
}
}

Group Mod, Change buckets and ports assigned to buckets.

PUT: https://10.15.155.61:8443/sdn/v2.0/of/datapaths/00:14:88:51:fb:35:1a:40/groups/1

{
"version": "1.3.0",
"group": {
"id": 1,
"type": "all",
"buckets": [
{
"weight": 0,
"watch_group": 4294967295,
"watch_port": "ANY",
"actions": [
{
"output": 16
}
]
},
{
"weight": 0,
"watch_group": 4294967295,
"watch_port": "ANY",
"actions": [
{
"output": 17
}
]
},
{
"weight": 0,
"watch_group": 4294967295,
"watch_port": "ANY",
"actions": [
{
"output": 14
}
]
}
]
}
}

Flow Mod:

Creat a flow matching a destination IP of 231.0.4.5 and output to group 1 defined above.

POST: https://10.15.155.61:8443/sdn/v2.0/of/datapaths/00:14:88:51:fb:35:1a:40/flows

{
"flow": {
"priority": 36000,
"idle_timeout": 0,
"hard_timeout": 0,
"match": [
{
"eth_type": "ipv4"
},
{
"ipv4_dst": "231.0.4.5",
"mask": "255.255.255.255"
}
],
"instructions": [
{
"apply_actions": [
{"group": "1"}
]
}
]
}
}

Scott,

Thansk for the reply.  

I've had a go at implementing the group appraoch in my code.

Is there a limit to the number of groups and is there anything special about the watch_group value?  e.g.  "watch_group": 4294967295?

I have mutliple groups and group flows in each switch. When I try to modify the ports in group 3, I get an error, e.g.

curl -H -ksS X-Auth-Token:$AUTH_TOKEN https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/groups/3 -d {"version": "1.3.0", "group": {"id": 3,"type": "all","command": "modify","buckets": [{"weight": 0,"watch_group": 4294967295,"watch_port": "ANY","actions": [{"output": 11}]},{"weight": 0,"watch_group": 4294967295,"watch_port": "ANY","actions": [{"output": 13}]}]}} --request PUT
{"error":"java.lang.IllegalArgumentException","message":"{ofm:[V_1_3,ERROR,76,156230],GROUP_MOD_FAILED/EPERM,#dataBytes=64,OFM-cause:[V_1_3,GROUP_MOD,80,156230]}"}

I'm using the same code to modify groups 1 and 2, without problem.

I'm currently on switch S/W version KA.16.01.0007. Is this an issue?

Dave.

Hi Dave,

I tried the example you posted, and got the same error under certain conditions. I don't think that the issue is caused by the group ID (3) or the watch_group value (4294967295).

After creating a group using a POST request, I was able to successfully modify the group using a PUT request as you have below. However, I also noticed that if I re-issued the same PUT request then I'd get the error you're getting below. If I changed all ports included in the group, then the PUT request was accepted.

I will need to confirm with our firmware team, but I believe the reason you're getting this error is that either port 11 or port 13 are already included in a group. You'd get this error even if 11 or 13 were already included in group 3 (the one you're trying to modify). I realize that if this is the case, it's inconvenient.

Could you please run the following command before issuing the PUT request. This will show the current state of configured groups, and which ports have already been assigned to a group:

curl -H -ksS X-Auth-Token:$AUTH_TOKEN https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/groups --request GET

If this is the behavior you're seeing, you could work around it by issuing one group-mod which puts an unused port into the group, then a second request which adds all desired ports to the group. This workaround would temporarily cause drops, but may prevent you from getting this error. If I receive any more information from our firmware team, I'll post it here.

Shaun

Hi Dave,

One more follow-up: I think the condition you're hitting is that you're issuing a group update (PUT) with the same port values as already exist on the switch. If any of the port values are different, the group update should be accepted. Could you please confirm that this is the only case where you're seeing this behavior? We'll hopefully get a duplicate group update accepted in a future revision of firmware. For now, if the group update (PUT) is rejected, please verify that the group is already set as expected (using GET) and that will confirm that you've hit this situation. If you hit this situation, you'd only need to ignore the rejection.

Shaun

Shaun,

Thanks for the reply.  I'm only getting the error when either port 11 or port 13 is already in group 3. In my S/W, groups 1 aqnd 2 are used for IGMP messages. Ports 11 and 13 are also in group 1. If I modify group3 for completely different ports (49 and 50), there is no error.

curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/groups
{"version":"1.3.0","groups":[{"id":1,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":11}]},{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":13}]}]},{"id":2,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":5}]}]},{"id":3,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":11}]}]}]}

curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/groups/3" -d "{\"version\": \"1.3.0\",\"group\": {\"id\": 3,\"type\": \"all\",\"command\": \"modify\",\"buckets\": [{\"weight\": 0,\"watch_group\": 4294967295,\"watch_port\": \"ANY\",\"actions\": [{\"output\": 11}]},{\"weight\": 0,\"watch_group\": 4294967295,\"watch_port\": \"ANY\",\"actions\": [{\"output\": 13}]}]}}" --request PUT
{"error":"java.lang.IllegalArgumentException","message":"{ofm:[V_1_3,ERROR,76,1052612],GROUP_MOD_FAILED/EPERM,#dataBytes=64,OFM-cause:[V_1_3,GROUP_MOD,80,1052612]}"}

curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/groups"
{"version":"1.3.0","groups":[{"id":1,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":11}]},{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":13}]}]},{"id":2,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":5}]}]},{"id":3,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":11}]}]}]}

curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/groups/3" -d "{\"version\": \"1.3.0\",\"group\": {\"id\": 3,\"type\": \"all\",\"command\": \"modify\",\"buckets\": [{\"weight\": 0,\"watch_group\": 4294967295,\"watch_port\": \"ANY\",\"actions\": [{\"output\": 49}]},{\"weight\": 0,\"watch_group\": 4294967295,\"watch_port\": \"ANY\",\"actions\": [{\"output\": 50}]}]}}" --request PUT
{"version":"1.3.0","group":{"id":3,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":49}]},{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":50}]}]}}

curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/groups"
{"version":"1.3.0","groups":[{"id":1,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":11}]},{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":13}]}]},{"id":2,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":5}]}]},{"id":3,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":49}]},{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":50}]}]}]}

The problem could be due to pre-exisitng groups or groups which are processing data. Groups 1 and 2 are for IGMP and have almost no bit rate. Group 3 is processing packets at approx 50Mb/s. If I create the group3 and its flows befoe groups 1 and 2 and there is no stream of data, then there is no error. e.g.

curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/groups" -d "{\"version\": \"1.3.0\",\"group\": {\"id\": 3,\"type\": \"all\",\"command\": \"add\",\"buckets\": [{\"weight\": 0,\"watch_group\": 4294967295,\"watch_port\": \"ANY\",\"actions\": [{\"output\": 11}]}]}}" --request POST

curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/flows" -d "{\"flow\": {\"priority\": 20001,\"table_id\":100,\"idle_timeout\": 60000,\"match\": [{\"ipv4_src\":\"10.0.0.100\"},{\"ipv4_dst\":\"232.1.3.1\"},{\"eth_type\": \"ipv4\"}],\"instructions\": [{\"apply_actions\": [{\"group\": 3}]}]}}" --request POST

curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/groups"
{"version":"1.3.0","groups":[{"id":3,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":11}]}]}]}

curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/groups/3" -d "{\"version\": \"1.3.0\",\"group\": {\"id\": 3,\"type\": \"all\",\"command\": \"modify\",\"buckets\": [{\"weight\": 0,\"watch_group\": 4294967295,\"watch_port\": \"ANY\",\"actions\": [{\"output\": 11}]},{\"weight\": 0,\"watch_group\": 4294967295,\"watch_port\": \"ANY\",\"actions\": [{\"output\": 13}]}]}}" --request PUT
{"version":"1.3.0","group":{"id":3,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":11}]},{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":13}]}]}}

curl --header "X-Auth-Token:$AUTH_TOKEN" -H "Content-Type:application/json" -ksS --url "https://192.168.40.180:8443/sdn/v2.0/of/datapaths/00:02:34:64:a9:59:8a:00/groups"
{"version":"1.3.0","groups":[{"id":3,"type":"all","buckets":[{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":11}]},{"weight":0,"watch_group":4294967295,"watch_port":"ANY","actions":[{"output":13}]}]}]}

Dave.

Hi Dave,

If I could summarize the actions in your last response, it would be this:

GET: Group1 {11,13} Group2 {5}, Group3 {11}

PUT: Group3 {11,13} -- Fails because this exactly matches the ports in Group1

GET: Group1 {11,13} Group2 {5}, Group3 {11}

PUT: Group3 {49,50} -- Succeeds because no other group is exactly {49,50}

GET: Group1 {11,13} Group2 {5}, Group3 {49,50}

The workaround you'd need to use would be one of the following:

1. Keep track of which groups contain which ports, to know when updating group3 will make it identical to group1. If they'd be identical, then update flows to refer to group1 rather than group3. If you have a small number of flows, this may be feasible, however updating a flow may potentially drop packets while the entry is updated in the switch.
2. Add an unused port to group3, to ensure that it is never identical to any other group. Since your groups have type 'all', this should not have any other affects.

Would either of these be acceptable workarounds?

Shaun,

Thanks for the reply. Groups 1 and 2 have different flow rules assocaited with them. The flow rules are for the IGMP query and response multicast addresses, so the match fields are different. e.g.

Group1 matches ipv4_dst: 224.0.0.1 for IGMP

Group 2 matches 224.0.0.22 for IGMP

Group 3 matches ipv4_src: 232.1.3.1 and ipv4-dst: 10.0.0.100 for a SSM video.

For my destinations to receive video or another SSM stream, ideally I need groups that can have the same ports. I can use flow rules without groups for my IGMP flows, but each destantion should be able to receive mulitple SSM streams.

Dave.

It is also not possible to modify a group where the ports are already used in a normal match flow, e.g.

add match 224.0.0.11, action: outpt port 11, output port 13 - works.

add group 1: output port 11 - works.

add match 232.1.31., action group 1 - works.

modify group 1: output port 11, output port 13 - fails.

However if ports 11 and 13 are not used in any other groups or any other flows, then the group can be modified to add a port to the list of ports.

Hi Dave,

Just to be clear: the behavior you should see is that a group add/modify will fail if that EXACT set of ports matches another group. In your case above, group3 failed because the ports in it (11,13) exactly matched the ports in group1 (11,13). If you add another port to either group1 or group3, both will be accepted.

For example, the following sequence should work and give the desired behavior if port 20 is disconnected:

• add group 1: output port 11, 13
• add match 224.0.0.11, action: group 1
• add group 3: output port 11, 13, 20
• add match 232.1.3.1, action group 3

After the above sequence, both 224.0.0.11 and 232.1.3.1 will forward to ports 11 and 13, but will be using different groups.

Please let us know if this does not work for you, or if you're seeing other behavior. Thanks!

Shaun

Shaun,

It also fails if the ports in the group match ports used in a non-group match action (i.e. actions: output 11, output 13, then modifiy group 1 : 11, 13).

I will try adding a disconected port to my port group.

Ideally I need multiple groups that can contain both the same and different ports. The ports in each group will change as destiantions join and leave different multicast streams.

Thanks,

Dave.

Shaun,

Adding an unused port to the group, so there are no identical groups, works.

Thanks,

Dave.

Hi Dave,

Will this workaround be sufficient for you to use long-term in your application? If so, please mark the thread as having an accepted solution. If not, could you give more details as to why this workaround (including an unused port) would not be acceptable and which timeframe it would be acceptable for?

Shaun

Shaun,

This solution (dummy NC ports) and the other solution (only unqiue groups per switch) are OK for the proof of concept I'm doing. The group action works much better than adding multiple ouptu ports in the flow action.

I can see why only allowing unique groups is a good idea, it reduces the number of group ids needed in each switch. However, it does mean that an application has to keep track of more flow data and potentially increases the number of REST get/put/post for each event.

Is only allowing unique groups somehting required by the OF1.3 spec, or a design decsison by the HP enginners? Are there anny issues with number of groups / group id?

This is an R&D project, I don't have a time scale for going from PoC to real service.

It would be nice if there was the flexibility of non-unique groups. It does make it easier to see what the app is doing by looking at the Controller OF monitor and it makes the app simpler to implement. Either way it is possible to work agroud the issue.

Thansk for the help, Dave.

Hi Dave,

Thanks for the confirmation, we realize this isn't ideal. From my reading of the OF1.3 spec, there is no requirement that the group action bucket contents must be unique. The only reference to uniqueness for an OF group is the group ID which is defined this way (section 5.6):

group identifier: a 32 bit unsigned integer uniquely identifying the group on the OpenFlow
switch.

The requirement that groups have a unique set of ports is likely an internal HPE design decision to ensure that OpenFlow can coordinate properly with other switch features which require port-based filtering (multicast, vlans, trunks, etc). If this becomes an inhibitor for further progress, please let us know. Thanks!

Shaun

Shaun,

Thanks for the reply. Futher to the issue of unique groups:

Changing a group number in a match action flow can cause dropped packets, which is a problem for media streams. The video can be disrupted at any pre-existing destiantions.

When I modify the ports in an exisitng group, I do not see any dropped, out-of-order or repeated packets. On my test network, adding and removing mutlicast branches using groups does not disrupt the video stream to any pre-exsiting destiantions.

Support for multicast is very important for the broadcast and media industry, for both production and delivery to viewers. Joing and leaving mutlicast streams must not cause disruption to the stream. As destiantions join/leave stream, the port groups will change. If groups are unique per switch, the match action flow would have to be changed when there is an exsiting group with the same ports.

For OF SDN to be adopted, we need multicast support. The sooner non-unique groups are supported, the better.

Dave.