MaxOutstandingR2T iSCSI parameter and its influence on performance in case of high latency links. Let's consider we have a 1Gbps network between initiator and target with 10ms latency (a good near distance WAN/Internet to another building in the same town). We want to send backup from the initiator to a tape or tape library on the target. We are limited to send only 1 write command at time, because our tape doesn't allow more. We will send 2MB of data in each command. Our initiator and target negotiated typical values InitialR2T No, ImmediateData Yes, FirstBurstLength 65536, MaxBurstLength 262144. Other parameters don't matter for our task, except MaxOutstandingR2T. Let we can run 2 pieces of iSCSI target software on the target: one supporting only MaxOutstandingR2T 1 and another one supporting MaxOutstandingR2T >1. The first target negotiated MaxOutstandingR2T 1, the second one - 16. 10ms means that a 1 byte packet send from the initiator reaches the target in 5ms time. Then 5ms in the opposite direction. 1Gbps bandwidth means that 64K of data transferred from the initiator to the target in about 0.5ms. The maximum throuput we can have with 1Gbps link is about 120MB/s. For sake of simplification we will suppose the targets will process requests and the initiator will process responses quick enough, so we can ignore additional latency the processing on both sides adds. Since each R2T request must contain less than MaxBurstLength data, we need ((2MB - 64K)/256K) = 8 R2T requests to send. The first 64K will be sent as immediate/unsolicited data without explicit R2T request. 1. The first target with MaxOutstandingR2T 1. MaxOutstandingR2T 1 means that only one R2T request can be active on time. I.e., the next request for data transfer can be sent after the previous one completed and all the data received. Thus, on the first target each command will be completed on time: 5 + 0.5 + (5 + 5 + 256K/64K * 0.5) * 7 + (5 + 5 + (256K - 64K)/64K * 0.5) + 5 = 106ms, i.e. 9 IOPS, which is 18MB/s. Remember, we have 120MB/s link. 2. The second target with MaxOutstandingR2T 16. With MaxOutstandingR2T 16 the second target can send all R2T requests at once and the first R2T can be sent before immediate/unsolicited data received. Thus, on the second target each command will be completed on time: 5 + 5 + 5 + 256K/64K * 0.5 * 7 + (256K-64K)/64K * 0.5 + 5 = 35.5ms, i.e. 28 IOPS, which is 56MB/s. Thus, the second target with MaxOutstandingR2T 16 will perform on 56/18 = 311% better that the first target with MaxOutstandingR2T 1.