Wired Intelligent Edge

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI? 

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Hello Stephan,

You may check Aruba 64xx series switches:

R0X38A-R0X43A and R0X38B-R0X40B Line Cards: 8MB packet buffer memory per line card
R0X44A-R0X45A Line Cards: 32MB packet buffer memory per line card

https://www.arubanetworks.com/assets/ds/DS_6400Series.pdf

Hope this helps!

------------------------------
-Alex-
------------------------------

Original Message:
Sent: Jun 13, 2022 06:03 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Y, saw that, but is in different price range too (as only line cards R0X44A and R0X45A have 32 MB, others have only 8 MB like 6300.) Both variants with 32 MB packet buffer (6400 with appropriate line cards or 8325) seem to be somewhere between 60K and 80K € per switch. Dell S5212F-ON (12 x 25 GbE, 3 x 100GbE, 32 MB packet buffer) is between 10K and 20K.
Original Message:
Sent: Jun 13, 2022 06:46 AM
From: Alexander Maroukian
Subject: iSCSI and packet buffer

Hello Stephan,

You may check 64xx series switches:

R0X38A-R0X43A and R0X38B-R0X40B Line Cards: 8MB packet buffer memory per line card
R0X44A-R0X45A Line Cards: 32MB packet buffer memory per line card

https://www.arubanetworks.com/assets/ds/DS_6400Series.pdf

Hope this helps!

------------------------------
-Alex-

Original Message:
Sent: Jun 13, 2022 06:03 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Well, we have Dell storage and Dell servers, and Dell's switch requirements clearly say that switches must have 32 MB packet buffer. Any Aruba switch for less than 80€K does not seem to meet that requirement, thus would leave our environment 'unsupported' from Dell side (even if it might work). Whereas with the Dell switch it will be supported, and Dell will have to fix it if it doesn't work.

Easy decision then.
Original Message:
Sent: Jun 13, 2022 06:03 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Well, to be honest - if we look at the number of offered/required ports - the Aruba 8360 16Y2C model (bundled with: Power Supplies and Fans Kit) seems to fit perfectly: it offers 16 ports 1/10/25 GbE (SFP/SFP+/SFP28) + 2 ports 40/100 GbE (QSFP+/QSFP28) and I doubt it will cost you more than 10/12 kEuro (VAT excluded) so quite far from the 80 kEuro you wrote about the Aruba 8325, at least I've seen that price range few times now (about 10kEuro VAT excluded) and the examined offers included also two SFP28 Transceivers, one Universal Rack Mounting kit, the Air Duct kit, a SFP28 DAC cable and also the 3Y Foundation Care support. Just to say. I'm referring specifically to the Aruba 8360 16Y2C v1 bundle (SKUs: JL702A/JL703A) but, I believe, the Aruba 8360 16Y2C v2 bundle (SKUs: JL702C/JL703C) would be near that price zone.
Original Message:
Sent: Jun 13, 2022 07:21 AM
From: Unknown User
Subject: iSCSI and packet buffer

Well, we have Dell storage and Dell servers, and Dell's switch requirements clearly say that switches must have 32 MB packet buffer. Any Aruba switch for less than 80€K does not seem to meet that requirement, thus would leave our environment 'unsupported' from Dell side (even if it might work). Whereas with the Dell switch it will be supported, and Dell will have to fix it if it doesn't work.

Easy decision then.
Original Message:
Sent: Jun 13, 2022 06:03 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Hey, indeed that looks the cheapest Aruba solution. 8325 price was so off (for our purposes) that I didn't even check 8360. Still, I couldn't find JL703A cheaper than 20K on quick googling, but will ask quote from our favorite vendor.

Dell switch is 7K though. 
Original Message:
Sent: Jun 13, 2022 10:25 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Well, to be honest - if we look at the number of offered/required ports - the Aruba 8360 16Y2C model (bundled with: Power Supplies and Fans Kit) seems to fit perfectly: it offers 16 ports 1/10/25 GbE (SFP/SFP+/SFP28) + 2 ports 40/100 GbE (QSFP+/QSFP28) and I doubt it will cost you more than 10/12 kEuro (VAT excluded) so quite far from the 80 kEuro you wrote about the Aruba 8325, at least I've seen that price range few times now (about 10kEuro VAT excluded) and the examined offers included also two SFP28 Transceivers, one Universal Rack Mounting kit, the Air Duct kit, a SFP28 DAC cable and also the 3Y Foundation Care support. Just to say. I'm referring specifically to the Aruba 8360 16Y2C v1 bundle (SKUs: JL702A/JL703A) but, I believe, the Aruba 8360 16Y2C v2 bundle (SKUs: JL702C/JL703C) would be near that price zone.
Original Message:
Sent: Jun 13, 2022 07:21 AM
From: Unknown User
Subject: iSCSI and packet buffer

Well, we have Dell storage and Dell servers, and Dell's switch requirements clearly say that switches must have 32 MB packet buffer. Any Aruba switch for less than 80€K does not seem to meet that requirement, thus would leave our environment 'unsupported' from Dell side (even if it might work). Whereas with the Dell switch it will be supported, and Dell will have to fix it if it doesn't work.

Easy decision then.
Original Message:
Sent: Jun 13, 2022 06:03 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

The DELL one looks - pardon - "half of a switch" (at least speaking in terms of number of front ports) more or less like the HPE M-series SN2100M or the HPE M-series SN2010M switch series (If I recall they have 16MB Packet Buffer but very low latency)...oh...not an easy decision IMHO.

QuickSpecs here and here.
Original Message:
Sent: Jun 13, 2022 10:41 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hey, indeed that looks the cheapest Aruba solution. 8325 price was so off (for our purposes) that I didn't even check 8360. Still, I couldn't find JL703A cheaper than 20K on quick googling, but will ask quote from our favorite vendor.

Dell switch is 7K though. 
Original Message:
Sent: Jun 13, 2022 10:25 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Well, to be honest - if we look at the number of offered/required ports - the Aruba 8360 16Y2C model (bundled with: Power Supplies and Fans Kit) seems to fit perfectly: it offers 16 ports 1/10/25 GbE (SFP/SFP+/SFP28) + 2 ports 40/100 GbE (QSFP+/QSFP28) and I doubt it will cost you more than 10/12 kEuro (VAT excluded) so quite far from the 80 kEuro you wrote about the Aruba 8325, at least I've seen that price range few times now (about 10kEuro VAT excluded) and the examined offers included also two SFP28 Transceivers, one Universal Rack Mounting kit, the Air Duct kit, a SFP28 DAC cable and also the 3Y Foundation Care support. Just to say. I'm referring specifically to the Aruba 8360 16Y2C v1 bundle (SKUs: JL702A/JL703A) but, I believe, the Aruba 8360 16Y2C v2 bundle (SKUs: JL702C/JL703C) would be near that price zone.
Original Message:
Sent: Jun 13, 2022 07:21 AM
From: Unknown User
Subject: iSCSI and packet buffer

Well, we have Dell storage and Dell servers, and Dell's switch requirements clearly say that switches must have 32 MB packet buffer. Any Aruba switch for less than 80€K does not seem to meet that requirement, thus would leave our environment 'unsupported' from Dell side (even if it might work). Whereas with the Dell switch it will be supported, and Dell will have to fix it if it doesn't work.

Easy decision then.
Original Message:
Sent: Jun 13, 2022 06:03 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Ah, I knew that I missed something. They have "storage switches" in addition to the now Aruba-branded "network switches." Still, all these models have only 16 MB packet buffer - and Dell's statement is quite clear: Less than 32 MB packet buffer -> unsupported.
Original Message:
Sent: Jun 13, 2022 10:58 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

The DELL one looks - pardon - "half of a switch" (at least speaking in terms of number of front ports) more or less like the HPE M-series SN2100M or the HPE M-series SN2010M switch series (If I recall they have 16MB Packet Buffer but very low latency)...oh...not an easy decision IMHO.

QuickSpecs here and here.
Original Message:
Sent: Jun 13, 2022 10:41 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hey, indeed that looks the cheapest Aruba solution. 8325 price was so off (for our purposes) that I didn't even check 8360. Still, I couldn't find JL703A cheaper than 20K on quick googling, but will ask quote from our favorite vendor.

Dell switch is 7K though. 
Original Message:
Sent: Jun 13, 2022 10:25 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Well, to be honest - if we look at the number of offered/required ports - the Aruba 8360 16Y2C model (bundled with: Power Supplies and Fans Kit) seems to fit perfectly: it offers 16 ports 1/10/25 GbE (SFP/SFP+/SFP28) + 2 ports 40/100 GbE (QSFP+/QSFP28) and I doubt it will cost you more than 10/12 kEuro (VAT excluded) so quite far from the 80 kEuro you wrote about the Aruba 8325, at least I've seen that price range few times now (about 10kEuro VAT excluded) and the examined offers included also two SFP28 Transceivers, one Universal Rack Mounting kit, the Air Duct kit, a SFP28 DAC cable and also the 3Y Foundation Care support. Just to say. I'm referring specifically to the Aruba 8360 16Y2C v1 bundle (SKUs: JL702A/JL703A) but, I believe, the Aruba 8360 16Y2C v2 bundle (SKUs: JL702C/JL703C) would be near that price zone.
Original Message:
Sent: Jun 13, 2022 07:21 AM
From: Unknown User
Subject: iSCSI and packet buffer

Well, we have Dell storage and Dell servers, and Dell's switch requirements clearly say that switches must have 32 MB packet buffer. Any Aruba switch for less than 80€K does not seem to meet that requirement, thus would leave our environment 'unsupported' from Dell side (even if it might work). Whereas with the Dell switch it will be supported, and Dell will have to fix it if it doesn't work.

Easy decision then.
Original Message:
Sent: Jun 13, 2022 06:03 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Original Message:
Sent: 6/13/2022 11:11:00 AM
From: Stephan van Helden
Subject: RE: iSCSI and packet buffer

Ah, I knew that I missed something. They have "storage switches" in addition to the now Aruba-branded "network switches." Still, all these models have only 16 MB packet buffer - and Dell's statement is quite clear: Less than 32 MB packet buffer -> unsupported.
Original Message:
Sent: Jun 13, 2022 10:58 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

The DELL one looks - pardon - "half of a switch" (at least speaking in terms of number of front ports) more or less like the HPE M-series SN2100M or the HPE M-series SN2010M switch series (If I recall they have 16MB Packet Buffer but very low latency)...oh...not an easy decision IMHO.

QuickSpecs here and here.
Original Message:
Sent: Jun 13, 2022 10:41 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hey, indeed that looks the cheapest Aruba solution. 8325 price was so off (for our purposes) that I didn't even check 8360. Still, I couldn't find JL703A cheaper than 20K on quick googling, but will ask quote from our favorite vendor.

Dell switch is 7K though. 
Original Message:
Sent: Jun 13, 2022 10:25 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Well, to be honest - if we look at the number of offered/required ports - the Aruba 8360 16Y2C model (bundled with: Power Supplies and Fans Kit) seems to fit perfectly: it offers 16 ports 1/10/25 GbE (SFP/SFP+/SFP28) + 2 ports 40/100 GbE (QSFP+/QSFP28) and I doubt it will cost you more than 10/12 kEuro (VAT excluded) so quite far from the 80 kEuro you wrote about the Aruba 8325, at least I've seen that price range few times now (about 10kEuro VAT excluded) and the examined offers included also two SFP28 Transceivers, one Universal Rack Mounting kit, the Air Duct kit, a SFP28 DAC cable and also the 3Y Foundation Care support. Just to say. I'm referring specifically to the Aruba 8360 16Y2C v1 bundle (SKUs: JL702A/JL703A) but, I believe, the Aruba 8360 16Y2C v2 bundle (SKUs: JL702C/JL703C) would be near that price zone.
Original Message:
Sent: Jun 13, 2022 07:21 AM
From: Unknown User
Subject: iSCSI and packet buffer

Well, we have Dell storage and Dell servers, and Dell's switch requirements clearly say that switches must have 32 MB packet buffer. Any Aruba switch for less than 80€K does not seem to meet that requirement, thus would leave our environment 'unsupported' from Dell side (even if it might work). Whereas with the Dell switch it will be supported, and Dell will have to fix it if it doesn't work.

Easy decision then.
Original Message:
Sent: Jun 13, 2022 06:03 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Storage will be connected with 25G, current servers with 10G, the servers will eventually be replaced and the new ones will get 25G.

Dell requires 9 MB packet buffer for pure 10G or 32 MB packet buffer if any 25G is involved. I have no real doubts that it would work with 8 MB, but it would be officially "unsupported" and for obvious reasons we want to avoid that.
Original Message:
Sent: Jun 13, 2022 12:06 PM
From: Davide Poletto
Subject: iSCSI and packet buffer

Are you really going to connect all of your new and legacy DELL Servers to your DELL Storage leveraging the 25Gbps ports speed (strictly end-to-end) or what else? sorry to insist but I strongly believe that the 32MB packet buffer requirement is tied to, particularly, the usage of higher speed on access ports such as the 25Gbps instead of the more common 10Gbps...

Original Message:
Sent: 6/13/2022 11:11:00 AM
From: Stephan van Helden
Subject: RE: iSCSI and packet buffer

Ah, I knew that I missed something. They have "storage switches" in addition to the now Aruba-branded "network switches." Still, all these models have only 16 MB packet buffer - and Dell's statement is quite clear: Less than 32 MB packet buffer -> unsupported.
Original Message:
Sent: Jun 13, 2022 10:58 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

The DELL one looks - pardon - "half of a switch" (at least speaking in terms of number of front ports) more or less like the HPE M-series SN2100M or the HPE M-series SN2010M switch series (If I recall they have 16MB Packet Buffer but very low latency)...oh...not an easy decision IMHO.

QuickSpecs here and here.
Original Message:
Sent: Jun 13, 2022 10:41 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hey, indeed that looks the cheapest Aruba solution. 8325 price was so off (for our purposes) that I didn't even check 8360. Still, I couldn't find JL703A cheaper than 20K on quick googling, but will ask quote from our favorite vendor.

Dell switch is 7K though. 
Original Message:
Sent: Jun 13, 2022 10:25 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Well, to be honest - if we look at the number of offered/required ports - the Aruba 8360 16Y2C model (bundled with: Power Supplies and Fans Kit) seems to fit perfectly: it offers 16 ports 1/10/25 GbE (SFP/SFP+/SFP28) + 2 ports 40/100 GbE (QSFP+/QSFP28) and I doubt it will cost you more than 10/12 kEuro (VAT excluded) so quite far from the 80 kEuro you wrote about the Aruba 8325, at least I've seen that price range few times now (about 10kEuro VAT excluded) and the examined offers included also two SFP28 Transceivers, one Universal Rack Mounting kit, the Air Duct kit, a SFP28 DAC cable and also the 3Y Foundation Care support. Just to say. I'm referring specifically to the Aruba 8360 16Y2C v1 bundle (SKUs: JL702A/JL703A) but, I believe, the Aruba 8360 16Y2C v2 bundle (SKUs: JL702C/JL703C) would be near that price zone.
Original Message:
Sent: Jun 13, 2022 07:21 AM
From: Unknown User
Subject: iSCSI and packet buffer

Well, we have Dell storage and Dell servers, and Dell's switch requirements clearly say that switches must have 32 MB packet buffer. Any Aruba switch for less than 80€K does not seem to meet that requirement, thus would leave our environment 'unsupported' from Dell side (even if it might work). Whereas with the Dell switch it will be supported, and Dell will have to fix it if it doesn't work.

Easy decision then.
Original Message:
Sent: Jun 13, 2022 06:03 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Just to chime in on this, while the rest of my network is Aruba CX (8325s and 6200s), I have a pair of Dell S5212s as storage switches in production for about two and a half years now with non-Dell storage hardware, and they've been rock solid.  I wouldn't hesitate to buy them again, though I will say that what you're being quoted for them is 3-6x what I paid for them in late 2019 (but I guess this is the world we live in these days).

The CLI is a little bit clunky and the OS has some small irritants (the biggest one is that, like the Aruba 8325, you have to set entire port groups to 10g rather than individual interfaces, and unlike Aruba, the interface context acts like breakouts from there) but realistically, a storage switch probably isn't going to do anything more than basic layer 2 stuff, so the config is pretty short and you don't touch them, regardless.
Original Message:
Sent: Jun 13, 2022 01:58 PM
From: Unknown User
Subject: iSCSI and packet buffer

Storage will be connected with 25G, current servers with 10G, the servers will eventually be replaced and the new ones will get 25G.

Dell requires 9 MB packet buffer for pure 10G or 32 MB packet buffer if any 25G is involved. I have no real doubts that it would work with 8 MB, but it would be officially "unsupported" and for obvious reasons we want to avoid that.
Original Message:
Sent: Jun 13, 2022 12:06 PM
From: Davide Poletto
Subject: iSCSI and packet buffer

Are you really going to connect all of your new and legacy DELL Servers to your DELL Storage leveraging the 25Gbps ports speed (strictly end-to-end) or what else? sorry to insist but I strongly believe that the 32MB packet buffer requirement is tied to, particularly, the usage of higher speed on access ports such as the 25Gbps instead of the more common 10Gbps...

Original Message:
Sent: 6/13/2022 11:11:00 AM
From: Stephan van Helden
Subject: RE: iSCSI and packet buffer

Ah, I knew that I missed something. They have "storage switches" in addition to the now Aruba-branded "network switches." Still, all these models have only 16 MB packet buffer - and Dell's statement is quite clear: Less than 32 MB packet buffer -> unsupported.
Original Message:
Sent: Jun 13, 2022 10:58 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

The DELL one looks - pardon - "half of a switch" (at least speaking in terms of number of front ports) more or less like the HPE M-series SN2100M or the HPE M-series SN2010M switch series (If I recall they have 16MB Packet Buffer but very low latency)...oh...not an easy decision IMHO.

QuickSpecs here and here.
Original Message:
Sent: Jun 13, 2022 10:41 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hey, indeed that looks the cheapest Aruba solution. 8325 price was so off (for our purposes) that I didn't even check 8360. Still, I couldn't find JL703A cheaper than 20K on quick googling, but will ask quote from our favorite vendor.

Dell switch is 7K though. 
Original Message:
Sent: Jun 13, 2022 10:25 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Well, to be honest - if we look at the number of offered/required ports - the Aruba 8360 16Y2C model (bundled with: Power Supplies and Fans Kit) seems to fit perfectly: it offers 16 ports 1/10/25 GbE (SFP/SFP+/SFP28) + 2 ports 40/100 GbE (QSFP+/QSFP28) and I doubt it will cost you more than 10/12 kEuro (VAT excluded) so quite far from the 80 kEuro you wrote about the Aruba 8325, at least I've seen that price range few times now (about 10kEuro VAT excluded) and the examined offers included also two SFP28 Transceivers, one Universal Rack Mounting kit, the Air Duct kit, a SFP28 DAC cable and also the 3Y Foundation Care support. Just to say. I'm referring specifically to the Aruba 8360 16Y2C v1 bundle (SKUs: JL702A/JL703A) but, I believe, the Aruba 8360 16Y2C v2 bundle (SKUs: JL702C/JL703C) would be near that price zone.
Original Message:
Sent: Jun 13, 2022 07:21 AM
From: Unknown User
Subject: iSCSI and packet buffer

Well, we have Dell storage and Dell servers, and Dell's switch requirements clearly say that switches must have 32 MB packet buffer. Any Aruba switch for less than 80€K does not seem to meet that requirement, thus would leave our environment 'unsupported' from Dell side (even if it might work). Whereas with the Dell switch it will be supported, and Dell will have to fix it if it doesn't work.

Easy decision then.
Original Message:
Sent: Jun 13, 2022 06:03 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Well, I personally don't like any "vendor lock-in technique" (in this case I believe we are quite far from that scenario...since the "32MB Packet Buffer size" requirement is vendor agnostic and, probably, the true challenge is more about the switch(es) price other than about the vendor name)...in my case...speaking about the Ethernet switching part...having an all-Aruba ecosystem I'll push to deploy a solution using Aruba switches (considering that keeping the Ethernet network tied to a particular switch class can leverage the know-how about the running OS, in this  case ArubaOS-CX)...clearly deployed switches need to fulfill requested parameters and budget.

I have some experience about mixed Vendors environment (on the Ethernet side: mostly IBM Power Systems servers and DELL PowerEdge servers...all deployed with Aruba 83xx series) and I never had issues although I've to admit that the Storage area (FC SAN) is entirely made with IBM hardware but that choice was done only considering the benefit of having an unique IBM HW/SW support partner (and, not secondarily, also considering the required "Compatibility Matrix" by IBM, a thing that - on IBM environment - is quite sensible especially when some software solutions build on their servers are validated against the full HW/SW stack <- on the Ethernet part IBM is relatively loose and you can have more fun).

If the OP likes to deploy DELL Networking among other existing/future DELL hardware (Servers and Storage) there is no choice to speak about.
Original Message:
Sent: Jun 13, 2022 05:37 PM
From: John Kupski
Subject: iSCSI and packet buffer

Just to chime in on this, while the rest of my network is Aruba CX (8325s and 6200s), I have a pair of Dell S5212s as storage switches in production for about two and a half years now with non-Dell storage hardware, and they've been rock solid.  I wouldn't hesitate to buy them again, though I will say that what you're being quoted for them is 3-6x what I paid for them in late 2019 (but I guess this is the world we live in these days).

The CLI is a little bit clunky and the OS has some small irritants (the biggest one is that, like the Aruba 8325, you have to set entire port groups to 10g rather than individual interfaces, and unlike Aruba, the interface context acts like breakouts from there) but realistically, a storage switch probably isn't going to do anything more than basic layer 2 stuff, so the config is pretty short and you don't touch them, regardless.
Original Message:
Sent: Jun 13, 2022 01:58 PM
From: Unknown User
Subject: iSCSI and packet buffer

Storage will be connected with 25G, current servers with 10G, the servers will eventually be replaced and the new ones will get 25G.

Dell requires 9 MB packet buffer for pure 10G or 32 MB packet buffer if any 25G is involved. I have no real doubts that it would work with 8 MB, but it would be officially "unsupported" and for obvious reasons we want to avoid that.
Original Message:
Sent: Jun 13, 2022 12:06 PM
From: Davide Poletto
Subject: iSCSI and packet buffer

Are you really going to connect all of your new and legacy DELL Servers to your DELL Storage leveraging the 25Gbps ports speed (strictly end-to-end) or what else? sorry to insist but I strongly believe that the 32MB packet buffer requirement is tied to, particularly, the usage of higher speed on access ports such as the 25Gbps instead of the more common 10Gbps...

Original Message:
Sent: 6/13/2022 11:11:00 AM
From: Stephan van Helden
Subject: RE: iSCSI and packet buffer

Ah, I knew that I missed something. They have "storage switches" in addition to the now Aruba-branded "network switches." Still, all these models have only 16 MB packet buffer - and Dell's statement is quite clear: Less than 32 MB packet buffer -> unsupported.
Original Message:
Sent: Jun 13, 2022 10:58 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

The DELL one looks - pardon - "half of a switch" (at least speaking in terms of number of front ports) more or less like the HPE M-series SN2100M or the HPE M-series SN2010M switch series (If I recall they have 16MB Packet Buffer but very low latency)...oh...not an easy decision IMHO.

QuickSpecs here and here.
Original Message:
Sent: Jun 13, 2022 10:41 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hey, indeed that looks the cheapest Aruba solution. 8325 price was so off (for our purposes) that I didn't even check 8360. Still, I couldn't find JL703A cheaper than 20K on quick googling, but will ask quote from our favorite vendor.

Dell switch is 7K though. 
Original Message:
Sent: Jun 13, 2022 10:25 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Well, to be honest - if we look at the number of offered/required ports - the Aruba 8360 16Y2C model (bundled with: Power Supplies and Fans Kit) seems to fit perfectly: it offers 16 ports 1/10/25 GbE (SFP/SFP+/SFP28) + 2 ports 40/100 GbE (QSFP+/QSFP28) and I doubt it will cost you more than 10/12 kEuro (VAT excluded) so quite far from the 80 kEuro you wrote about the Aruba 8325, at least I've seen that price range few times now (about 10kEuro VAT excluded) and the examined offers included also two SFP28 Transceivers, one Universal Rack Mounting kit, the Air Duct kit, a SFP28 DAC cable and also the 3Y Foundation Care support. Just to say. I'm referring specifically to the Aruba 8360 16Y2C v1 bundle (SKUs: JL702A/JL703A) but, I believe, the Aruba 8360 16Y2C v2 bundle (SKUs: JL702C/JL703C) would be near that price zone.
Original Message:
Sent: Jun 13, 2022 07:21 AM
From: Unknown User
Subject: iSCSI and packet buffer

Well, we have Dell storage and Dell servers, and Dell's switch requirements clearly say that switches must have 32 MB packet buffer. Any Aruba switch for less than 80€K does not seem to meet that requirement, thus would leave our environment 'unsupported' from Dell side (even if it might work). Whereas with the Dell switch it will be supported, and Dell will have to fix it if it doesn't work.

Easy decision then.
Original Message:
Sent: Jun 13, 2022 06:03 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Sorry, the Aruba 8320 owns "only" 16MB of Buffer Memory for the internal packet processor (so it's less deeper than 8325 and 8360v1/8360v2). In any case those ones are all DC Switches, the same can't be told about Aruba 6300 (which is more devoted to Campus/Access sides).

Back to your line of reasoning: it's a matter of "price budget" but, keep in mind, it's also a matter of having the right switches for the right duty...I don't know what DELL switch series you're referring to for a like-for-like comparison (considering price/performance) but, in the end, it's up to you ensure your specific network deployment - generally speaking - fits the bill fulfilling all your requirements (not excluded the budget one).
Original Message:
Sent: Jun 13, 2022 05:44 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks! Y, then it seems that CX 6000 series is out of the question. And even CX 8320 has only 16 MB packet buffer if I read the datasheet correctly.

Looks like we need to get a Dell switch then. Btw, I too don't like iSCSI, it's just so that the existing hosts have iSCSI cards for legacy reasons and we don't want to touch them.
Original Message:
Sent: Jun 13, 2022 05:34 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

Hi Stephan, not an expert about iSCSI (I work with FC SAN Fabrics) but it seems that, in your scenario, all boils down to what ports' speeds you have to connect with.

If I read this DELL Knowledge Base Article correctly, if you're going to connect a iSCSI enabled DELL PowerStore with a 10Gbps network infrastructure then at least a 9MB of Shared Packet Buffer is required (which, as far as I understand, it means that 9MB would be enough), higher speeds require generally deeper packet buffers (thus larger size): 25Gbps seems to require at least 32MB of Shared Packet Buffer.

At this point the real question would be: is really the Aruba 6300 Switch series suitable for the Ethernet based Storage Network I want to setup considering the specific iSCSI implementation I've in mind?

To stay on the safe side (and thinking you're going to connect a lot of servers at the speed of 25Gbps) I would say I'm going to use Aruba CX 83xx Switch series (like the 8320, 8325 or 8360 <- all of them provide (a) 32MB Packet Buffer and (b) 10/25Gbps speeds on most port with 40/100Gbps speeds on few ports generally used for uplinks or VSX ISL) but those switches families require a totally different budget compared to the Aruba 6300.
Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

For lossless ethernet, the Aruba CX 8325 CX 10000 & CX 8360 series will be the right fit for an iSCSI lossless flow also supporting a 32MB packet buffer. These switches support the DCB suite of protocols with lossless pools which are a perfect for iSCSI traffic flows to ensure traffic is not lost.

If the above switches are out of  scope through cost, applying appropriate QoS settings with DWRR on the 6300 to ensure iSCSI packets are not dropped might be a fit subject to testing.  8MB shared buffer(or even 9MB shared buffer with Dell switches) is the limiting factor so fairness to other flows is a consideration (i.e.  not too  aggressive with DWRR weights). Some caveats, if using TCP with iSCSI and packets are dropped; this will impact performance. (which may not be an issue.)

8MB v 9MB shared packet buffer across different switches is I think mainly a minor issue as this buffer space is shared for all packets across all ports and can be minimalized use QoS for the appropriate flow(S) .

Best /recommended option from aruba is the the 8325/10000/8360 series if true lossless ethernet working is required. 
.
regards, Steve

------------------------------
Steve Bartlett
------------------------------

Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

Thanks. We'll check the 8360, others are out of scope pricewise. (Dell switches would be 15K € for two items including all cables and onsite installation.)

Tweaking 6300 settings is not an option because it would be unsupported.
Original Message:
Sent: Jun 14, 2022 07:31 AM
From: Steve Bartlett
Subject: iSCSI and packet buffer

For lossless ethernet, the Aruba CX 8325 CX 10000 & CX 8360 series will be the right fit for an iSCSI lossless flow also supporting a 32MB packet buffer. These switches support the DCB suite of protocols with lossless pools which are a perfect for iSCSI traffic flows to ensure traffic is not lost.

If the above switches are out of  scope through cost, applying appropriate QoS settings with DWRR on the 6300 to ensure iSCSI packets are not dropped might be a fit subject to testing.  8MB shared buffer(or even 9MB shared buffer with Dell switches) is the limiting factor so fairness to other flows is a consideration (i.e.  not too  aggressive with DWRR weights). Some caveats, if using TCP with iSCSI and packets are dropped; this will impact performance. (which may not be an issue.)

8MB v 9MB shared packet buffer across different switches is I think mainly a minor issue as this buffer space is shared for all packets across all ports and can be minimalized use QoS for the appropriate flow(S) .

Best /recommended option from aruba is the the 8325/10000/8360 series if true lossless ethernet working is required. 
.
regards, Steve

------------------------------
Steve Bartlett

Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

A side note: Best practices report that two "Storage" (dedicated, I add) switches should be deployed between Servers and iSCSI Storage (and that pair of switches should not necessarily be interconnected together to form a virtual switch using techniques like VSF, VSX or IRF), that's to mimic what normally happen on typical SAN Fabrics deployments where Multipath is used and Fabrics are decoupled (so you deal with two physically separated Fabrics: SAN Fabric A and SAN Fabric B and thus involved physical paths between each Server and its Storage are separated)...otherwise the single "Storage" switch approach, if used, will became a SPoF...and, if its duties/roles are also shared (Frontend to Clients mixed with Backend to Storage), the impact in case of fault is doubled (Frontend and Backend will be concurrently impacted).

At double the cost, the two "Storage" dedicated switches approach also help to mitigate the issue (on those switches) of prioritizing/optimizing the iSCSI traffic over the other types (since there should be only that type).


Original Message:
Sent: Jun 14, 2022 07:43 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks. We'll check the 8360, others are out of scope pricewise. (Dell switches would be 15K € for two items including all cables and onsite installation.)

Tweaking 6300 settings is not an option because it would be unsupported.
Original Message:
Sent: Jun 14, 2022 07:31 AM
From: Steve Bartlett
Subject: iSCSI and packet buffer

For lossless ethernet, the Aruba CX 8325 CX 10000 & CX 8360 series will be the right fit for an iSCSI lossless flow also supporting a 32MB packet buffer. These switches support the DCB suite of protocols with lossless pools which are a perfect for iSCSI traffic flows to ensure traffic is not lost.

If the above switches are out of  scope through cost, applying appropriate QoS settings with DWRR on the 6300 to ensure iSCSI packets are not dropped might be a fit subject to testing.  8MB shared buffer(or even 9MB shared buffer with Dell switches) is the limiting factor so fairness to other flows is a consideration (i.e.  not too  aggressive with DWRR weights). Some caveats, if using TCP with iSCSI and packets are dropped; this will impact performance. (which may not be an issue.)

8MB v 9MB shared packet buffer across different switches is I think mainly a minor issue as this buffer space is shared for all packets across all ports and can be minimalized use QoS for the appropriate flow(S) .

Best /recommended option from aruba is the the 8325/10000/8360 series if true lossless ethernet working is required. 
.
regards, Steve

------------------------------
Steve Bartlett

Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

I know, in typical iSCSI deployment you'd have two totally independent switches. However, Dell seems to demand that ports on same iSCSI subnet connect to different switches, and that would require these switches to be connected.

https://www.reddit.com/r/storage/comments/qne1d3/dell_emc_powerstore_cannot_be_integrated_in/

https://www.dell.com/support/manuals/en-us/powerstore-40u-rack/pwrstrt-ntwkg/node-to-top-of-rack-(tor)-switch-connectivity?guid=guid-685a0bed-9d3d-49e4-9e03-cd59602923da&lang=en-us

However, there are also documents with contradicting information:

https://www.dell.com/support/manuals/en-us/powerstore-500t/pwrstr-hcg/single-appliance-cluster?guid=guid-7692a2b0-95a7-44d5-9faa-6501f76e6886&lang=en-us

We'll have to clarify this with Dell (we'll book their installation service anyway). But it doesn't do harm if the switches can be connected.
Original Message:
Sent: Jun 14, 2022 09:17 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

A side note: Best practices report that two "Storage" (dedicated, I add) switches should be deployed between Servers and iSCSI Storage (and that pair of switches should not necessarily be interconnected together to form a virtual switch using techniques like VSF, VSX or IRF), that's to mimic what normally happen on typical SAN Fabrics deployments where Multipath is used and Fabrics are decoupled (so you deal with two physically separated Fabrics: SAN Fabric A and SAN Fabric B and thus involved physical paths between each Server and its Storage are separated)...otherwise the single "Storage" switch approach, if used, will became a SPoF...and, if its duties/roles are also shared (Frontend to Clients mixed with Backend to Storage), the impact in case of fault is doubled (Frontend and Backend will be concurrently impacted).

At double the cost, the two "Storage" dedicated switches approach also help to mitigate the issue (on those switches) of prioritizing/optimizing the iSCSI traffic over the other types (since there should be only that type).


Original Message:
Sent: Jun 14, 2022 07:43 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks. We'll check the 8360, others are out of scope pricewise. (Dell switches would be 15K € for two items including all cables and onsite installation.)

Tweaking 6300 settings is not an option because it would be unsupported.
Original Message:
Sent: Jun 14, 2022 07:31 AM
From: Steve Bartlett
Subject: iSCSI and packet buffer

For lossless ethernet, the Aruba CX 8325 CX 10000 & CX 8360 series will be the right fit for an iSCSI lossless flow also supporting a 32MB packet buffer. These switches support the DCB suite of protocols with lossless pools which are a perfect for iSCSI traffic flows to ensure traffic is not lost.

If the above switches are out of  scope through cost, applying appropriate QoS settings with DWRR on the 6300 to ensure iSCSI packets are not dropped might be a fit subject to testing.  8MB shared buffer(or even 9MB shared buffer with Dell switches) is the limiting factor so fairness to other flows is a consideration (i.e.  not too  aggressive with DWRR weights). Some caveats, if using TCP with iSCSI and packets are dropped; this will impact performance. (which may not be an issue.)

8MB v 9MB shared packet buffer across different switches is I think mainly a minor issue as this buffer space is shared for all packets across all ports and can be minimalized use QoS for the appropriate flow(S) .

Best /recommended option from aruba is the the 8325/10000/8360 series if true lossless ethernet working is required. 
.
regards, Steve

------------------------------
Steve Bartlett

Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?

With Aruba 8360 (v1 or v2, it doesn't matter...but, at this point, better considering the v2) you will not have any issue because you can eventually go with the VSX approach if necessary.

Original Message:
Sent: Jun 14, 2022 09:30 AM
From: Unknown User
Subject: iSCSI and packet buffer

I know, in typical iSCSI deployment you'd have two totally independent switches. However, Dell seems to demand that ports on same iSCSI subnet connect to different switches, and that would require these switches to be connected.

https://www.reddit.com/r/storage/comments/qne1d3/dell_emc_powerstore_cannot_be_integrated_in/

https://www.dell.com/support/manuals/en-us/powerstore-40u-rack/pwrstrt-ntwkg/node-to-top-of-rack-(tor)-switch-connectivity?guid=guid-685a0bed-9d3d-49e4-9e03-cd59602923da&lang=en-us

However, there are also documents with contradicting information:c

https://www.dell.com/support/manuals/en-us/powerstore-500t/pwrstr-hcg/single-appliance-cluster?guid=guid-7692a2b0-95a7-44d5-9faa-6501f76e6886&lang=en-us

We'll have to clarify this with Dell (we'll book their installation service anyway). But it doesn't do harm if the switches can be connected.
Original Message:
Sent: Jun 14, 2022 09:17 AM
From: Davide Poletto
Subject: iSCSI and packet buffer

A side note: Best practices report that two "Storage" (dedicated, I add) switches should be deployed between Servers and iSCSI Storage (and that pair of switches should not necessarily be interconnected together to form a virtual switch using techniques like VSF, VSX or IRF), that's to mimic what normally happen on typical SAN Fabrics deployments where Multipath is used and Fabrics are decoupled (so you deal with two physically separated Fabrics: SAN Fabric A and SAN Fabric B and thus involved physical paths between each Server and its Storage are separated)...otherwise the single "Storage" switch approach, if used, will became a SPoF...and, if its duties/roles are also shared (Frontend to Clients mixed with Backend to Storage), the impact in case of fault is doubled (Frontend and Backend will be concurrently impacted).

At double the cost, the two "Storage" dedicated switches approach also help to mitigate the issue (on those switches) of prioritizing/optimizing the iSCSI traffic over the other types (since there should be only that type).


Original Message:
Sent: Jun 14, 2022 07:43 AM
From: Unknown User
Subject: iSCSI and packet buffer

Thanks. We'll check the 8360, others are out of scope pricewise. (Dell switches would be 15K € for two items including all cables and onsite installation.)

Tweaking 6300 settings is not an option because it would be unsupported.
Original Message:
Sent: Jun 14, 2022 07:31 AM
From: Steve Bartlett
Subject: iSCSI and packet buffer

For lossless ethernet, the Aruba CX 8325 CX 10000 & CX 8360 series will be the right fit for an iSCSI lossless flow also supporting a 32MB packet buffer. These switches support the DCB suite of protocols with lossless pools which are a perfect for iSCSI traffic flows to ensure traffic is not lost.

If the above switches are out of  scope through cost, applying appropriate QoS settings with DWRR on the 6300 to ensure iSCSI packets are not dropped might be a fit subject to testing.  8MB shared buffer(or even 9MB shared buffer with Dell switches) is the limiting factor so fairness to other flows is a consideration (i.e.  not too  aggressive with DWRR weights). Some caveats, if using TCP with iSCSI and packets are dropped; this will impact performance. (which may not be an issue.)

8MB v 9MB shared packet buffer across different switches is I think mainly a minor issue as this buffer space is shared for all packets across all ports and can be minimalized use QoS for the appropriate flow(S) .

Best /recommended option from aruba is the the 8325/10000/8360 series if true lossless ethernet working is required. 
.
regards, Steve

------------------------------
Steve Bartlett

Original Message:
Sent: Jun 13, 2022 01:35 AM
From: Unknown User
Subject: iSCSI and packet buffer

Hi, we're about to replace our SAN, probably with Dell PowerStore. In the end we'll need two switches for iSCSI, each with 2 x 25 GbE and 4 x 10 GbE. 

As Aruba is our standard switch supplier, I wanted to use Aruba CX 6300 for that purpose. Now, Dell insists that the switch must have 32 MB packet buffer (or 9 MB if using 10 GbE only) - "to avoid packet loss large buffer is needed. If even just one packet is lost everything need to be retransmitted and that really kills performance."

Aruba CX 6300 has only 8 MB packet buffer though. Seems like the only Aruba switches with 32 MB are the ones with 48 100-GbE ports.

How do you see this? Do you think Dell made up the 9 MB requirement to sell their own switches? Or are Aruba switches simply not designed for iSCSI?