Comware

Hi,

I have two HP 5820 (JC102A) stacked via IRF.

Topology Info
-------------------------------------------------------------------------
IRF-Port1                            IRF-Port2
MemberID Link neighbor Link neighbor Belong To
1                  UP    2              UP      2          0024-730c-1416
2                  UP  1               UP      1           0024-730c-1416

Both switches runs latest firmware (Release 1810P03).  I have two VLAN interfaces with IPs (VLAN 2 and VLAN 3). If I connect devices to each VLAN on the same switch, traffic is routed between these devices. But if I move one device to second switch at IRF stack, there is no routing at all.  Ports and its VLANs are correctly configured. I can ping IRF stack VLAN interface IPs from each device. If I connect devices to the same VLAN on different switches it is working without issues.

I'm newbie to Comware, so I probably missed some configuration. Can you please kindly help me with finding the issue?

Thanks a lot!

 

#5820
#routing
#irf

That's interesting.

So no routing between VLAN 2 and VLAN 3 if you test traffic flow between a device connected to, let's say, a VLAN 2 tagged Port on IRF Member 1 and a device connected to, let's say, a VLAN 3 tagged port on IRF Member 2.

Instead if you repeat the same test between two devices both connected to VLAN 2 tagged (different) ports of different IRF Members...the traffic flows.

Normally routing among VLANs (with IP assigned) should work out-of-the-box either if this routing happens within the same IRF Member (among differents VLANs on the same IRF Member) or among IRF Members (among different VLANs on different IRF Members) provided that something (1) doesn't block those VLANs from crossing/traversing the IRF stack (which, I think, could be quite difficult - if not impossible - to achieve), (2) those VLANs (as you wrote are configured) are assigned to ports that belong to all involved IRF Members (as example, VLAN 2 and 3 configured on some ports of IRF Member 1 and 2) and (3) devices you're using to test routing between VLANs are properly configured [*] (pay attention to DHCP and routes). So it should work...

 

[*] devices that belong respectively to VLAN 2 or VLAN 3 must use respective VLAN 2's (or 3) IP Address as their Next-Hop/Default Gateway IP Address.

Hi,

Sounds really wierd.

Do you have more than one physical link in your IRF ports? If so, there might be one of these links that have many errors, and maybe the link load sharing mode is so, that the IP you get on vlan 3 falls into that faulty link. And therefore it looks like there is no routing, but in fact its just a bad link. 

There are many ifs and I don't really find it plausible myself, but on the other hand I can't see anything else that looks faulty.
Have you tried with just one IRF port, and just one physical link in that port? (just to rule out the above scenario)

Regards

Interesting...but then a question: if each involved logical IRF port is binded to multiple physical interfaces (as example: two 10G interfaces per each logical IRF port on each IRF Member of the IRF Stack) then...shouldn't that logical IRF port use the gained physical redundancy and load balancing features to let the IRF Stack to continue to work flawlessly by adapting dynamically the traffic between involved IRF Members even if a physical interface shows an issue (e.g. link broken/interface broken and so on)?

Hi,

thanks for answers. I was using officially not supported J9152A optical modules (switch logs warning, but module works). Change them to supported X240 DAC cables, but it doesn't help. My colleague found that there is strange overbalance in packets dropped by FFP on IRF conections. To be honest I'm not sure what does it mean.

display packet-drop interface

IRF 1

Ten-GigabitEthernet1/0/23:
  Packets dropped by GBP full or insufficient bandwidth: 0
  Packets dropped by FFP: 1352
  Packets dropped by STP non-forwarding state: 0

Ten-GigabitEthernet2/0/24:
  Packets dropped by GBP full or insufficient bandwidth: 0
  Packets dropped by FFP: 486
  Packets dropped by STP non-forwarding state: 0

IRF 2

Ten-GigabitEthernet1/0/24:
  Packets dropped by GBP full or insufficient bandwidth: 0
  Packets dropped by FFP: 154266
  Packets dropped by STP non-forwarding state: 0

Ten-GigabitEthernet2/0/23:
  Packets dropped by GBP full or insufficient bandwidth: 0
  Packets dropped by FFP: 51161
  Packets dropped by STP non-forwarding state: 0

I tried to disconnect IRF 2 DAC cable, but it doesn't help. Does it makes sense to try IRF on different SFP+ ports?

My next step is reset to default config. Setup IRF stack again and set only two VLANs with IPs, default gateway and do the tests again. If it fails, I will paste my full config here.

It is nice to know that Comware skilled people can help me here. Thanks a lot again!

 

Wait...How your IRF Stack (Daisy Chain IRF Topology I presume) was exactly configured?

If a Daisy-Chain (AKA Bus) IRF Topology was used:

• Which physical port(s) of IRF Member 1 was binded to the logical IRF port 1 on IRF Member 1?
• Which physical port(s) of IRF Member 2 was binded to the logical IRF port 2 on IRF Member 2?

If a (sort of) Ring IRF Topology was used (even if a Ring IRF Topology made with only two IRF Members is really a sort of "double crossed" Daisy Chain IRF Topology):

• Which physical port(s) of IRF Member 1 was binded to logical IRF port 1 on IRF Member 1?
• Which physical port(s) of IRF Member 2 was binded to logical IRF port 2 on IRF Member 2?
• Which physical port(s) of IRF Member 2 was binded to logical IRF port 1 on IRF Member 2?
• Which physical port(s) of IRF Member 1 was binded to logical IRF port 2 on IRF Member 1?

Since you played with a total of four Ten-Gigabit Ethernet ports (two per IRF Member): port 1/0/23 and port 1/0/24 on IRF Member 1, port 2/0/23 and 2/0/24 on IRF Member 2...I presume you chose the Daisy Chain IRF Topology deployed with both ports 1/0/23 and 1/0/24 binded to the same Logical IRF Port 1 on IRF Member 1 - those ports are then physically connected by means of two optical cables via 10G SFP+ Transceivers or by means of two DAC cables - respectively to port 2/0/23 and 2/0/24 both binded to the equivalent Logical IRF Port 2 on IRF Member 2 (so forget about the "double crossed" Ring IRF Topology when you have just two IRF Members only, as I asked above).

Did you deployed IRF Stack that way or what?

Hi Parnassus,
thanks for your reply. Switches are connected via DAC cables from 1/0/23 to 2/0/24 and from 1/0/24 to 2/0/23. IRF stack is done as described here: https://www.vcloudnine.de/creating-an-hp-irf-stack-with-hp-5820-24xg-sfp-switches/

If I were you I would go straight with the HP 5820X & 5800 Switch Series IRF Configuration Guide (Configuring IRF on page 11): I would use the IRF Link Redundancy (since you own two good DAC Cables) to deploy a Daisy Chain IRF Topology between your two Switches (this means: on the first Switch do bind two physical 10G ports [so 1/0/23 and 1/0/24 together] to the Logical IRF port 1 on IRF Member 1 and, on the second Switch, do bind two physical 10G ports [so 2/0/23 and 2/0/24 together] to the Logical IRF port 2 on IRF Member 2; no other Logical IRF ports are then required for such Daisy Chain IRF Topology deployment...this is somewhat different with regard to the exampe you followed...in which four Logical IRF ports were created - two per IRF Member - instead of only two really necessary with the used IRF Link Redundancy option).

It's pretty straightforward.

Hi,

Would be good to see the IRF configuration. But from what you post, it looks like you've made a 2 member ring topology.

You could consider using both of the physical links in one IRF logical link. (daisy). Not that it should make a difference.

About the FPP drops. That does seem a bit suspecious. Are those drop-counts growing faster when you're clients are on different vlans? (compared to when they are on the same vlan, where you have normal functionality) 

Regards

---

sdide wrote: You could consider using both of the physical links in one IRF logical link. (daisy). Not that it should make a difference.

Hi Søren! Nice drawing!

For the benefit of the OP...well, a relevant difference...is that the canonical way of configuring a IRF Stack in Daisy Chain when only two members are available (with IRF Link Redundancy at Logical IRF port level, e.g. with two 10G physical ports binded together to one logical IRF port) with respect to the same IRF Stack configured with a "Loop" (without that IRF Link Redundancy at logical IRF port level) where two Logical IRF ports per switch are singularly and respectively binded to a single 10G physical port...that difference grants IRF Links redundancy and IRF Link load balancing (so higher bandwidth: 20Gbps Full Duplex per that single Logical IRF port), the second way (the "Loop" way) provides only IRF Links redundancy and no IRF Link higher bandwidth between the two IRF Members of the stack (10Gbps Full Duplex per single Logical IRF port).

At least this is what I've seen pointed out, more or less recently, here and here.

Personally I don't know how much difference the gained aggregated bandwidth of that single Logical IRF port implies - when only two Switches are involved on a IRF Stack - but *for sure* I prefer to kill two birds with one stone...if that is easily achievable (the OP already used two SFP+ Transceivers per member).

Update: worth mentioning also, if not already done before, this HP Support Communication: Customer Advisory n°c04706976 "Intelligent Resilient Framework (IRF) Ring Topology for Two Devices in IRF Fabric".

How incredibly idiotic that high end units like 7900 do NOT support 2 member IFR fabric. What is the usser supposed to do with 2 units then?

Seb

Hi Seb, 

The 7900 does support a 2 member fabric! (All IRF-enabled switches do)  We're just discussing which way to make it is best to make it - while troubleshooting a routing issue.

Regards

---

spgsitsupport wrote: How incredibly idiotic that high end units like 7900 do NOT support 2 member IFR fabric. What is the usser supposed to do with 2 units then?

Seb

---

First: Sorry but - IMHO - that sentence is totally Off Topic here...and then, second: where did you read that HPE high end Comware based Switch units can't form an IRF Stack simply with just two members? an IRF Stack can be formed with 2 members only...so that sentence, other than OT, is also totally false.

In YOUR OWN linked Customer Advisory n°c04706976

http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=emr_na-c04706976&sp4ts.oid=5400932

 

7900 Is IRF Ring Topology Supported With Only Two Units? = NO

 

 

Hi, 

Keyword here would be "ring". 

You cannot according to the linked advisory, form a "2 unit RING IRF topology". 

You can however form a 2 unit daisy chain IRF topology. 

Since 1 IRF link can contain several physical links, daisychain is  all good and redundant.  

Regards. 

Words matter.

They do indeed, pity word "daisy" does not appear on that page even once...

Probably because the IRF topological method - normally expected - to be used when only two IRF members are available is (or should be) de facto only the Daisy Chain one...so again, when an IRF Stack is formed using only two members, the exception (indeed - IMHO - the one discussed on that Advisory) is exactly and only one the IRF "Ring" topology...that Advisory shed some light about the fact that (a sort of) IRF "Ring" topology (I use the " marks because we discussed above the differences between that "Ring" with 2 members and a real Ring with 3 members) is also supported (on some Switch series models only) with only two IRF members...and that Advisory did so without recalling what is the advised/normal way to properly create a IRF Stack with two IRF members.

So being new to Comware, am I understanding this correctly that if a two switch IRF is to be configured, the recommend method is the Daisy Chain but for three or more switches, the Ring is the recommendation?

If that is true, what happens if you set up a two switch daisy chain then  want to add another switch to the IRF?

Hi,

Then you add the third switch
and then if you want to
1) continue a daisy (not recommended ) you connect it to either switch1 or switch2.
or
2) make a ring you add switch3 to switch1 AND switch2.

Nothing special happens - it works :)

A logical IRF port can consists of several physical ports, thats why in a two switch scenario a 2-daisy chain is at least as good as a 2-ring. As long as the 2daisy-IRF logical port consist of the same physical ports as a 2ring-IRF logical ports.
In a 3 switch scenario, this changes. Because if you daisy-chain 3 switches, and then center switch fails the two remaining switches loose contact, this is not the case with a ring.
Going from a 2-daisy-IRF to a 3-ring-IRF, is actually easier than going from a 2-ring-IRF to a 3-ring-IRF, if you look at the configuration.
Considering the above and also that not all Comware versions support 2-ring-IRFs - I'd go with a 2-daisy-IRF.
Regards.

That's (also) related to this other @GlenRB post.

Hi Søren and Parnassus,

thanks so much for your replies and explanations. I have to postpone new switches deployment because of other high priority tasks. I reset both switches and setup daisy chain topology today. As you kindly explained, ring topology do not have any benefit for this setup. So my IRF port configuration is:

irf-port 1/1
port group interface Ten-GigabitEthernet1/0/23 mode enhanced
port group interface Ten-GigabitEthernet1/0/24 mode enhanced

irf-port 2/2
port group interface Ten-GigabitEthernet2/0/23 mode enhanced
port group interface Ten-GigabitEthernet2/0/24 mode enhanced

I did some performance testing to check that up to 20Gbits can flow on IRF. I also check routing between VLANs on IRF stack. Everything works nicely.

Kind regards

Standa