************************************************************************* ATM Forum Document Number: ATM_Forum/97-0857 ************************************************************************* Title: Effect of VS/VD on Switch Buffer Requirements ************************************************************************* Abstract: In this contribution, we present simulation results on the effect of Virtual Source/Virtual Destination(VS/VD) on switch buffer requirements. Experiments with different WAN/satellite configurations show that the ABR buffer requirements in the switch are bounded by the previous loop's delay-bandwidth product. ************************************************************************* Source: Xiangrong Cai, Rohit Goyal, Raj Jain, Shiv Kalyanaraman, Sonia Fahmy, and Bobby Vandalore Department of CIS, The Ohio State University (and NASA) 395 Dreese lab, 2015 Neil Ave, Columbus, OH 43210-1277 Phone: 614-292-3989,Fax:614-292-2911,Email:{cai,jain}@cis.ohio-state.edu Sastri Kota Lockheed Martin Telecommunications/Astrolink 1272 Borregas Avenue, Bldg B/551 O/GB - 70 Sunnyvale, CA 94089 Email: sastri.kota@lmco.com Pradeep Samudra Broadband Network Lab Samsung Electronics Co. Ltd. Samsung Telecom America, Inc. 1130 E Arapaho, Richardson, TX 75081 Email: psamudra@telecom.sna.samsung.com ************************************************************************* Date: September 1997 ************************************************************************* Distribution: ATM Forum Technical Working Group Members (AF-TM) ************************************************************************* Notice: This contribution has been prepared to assist the ATM Forum. It is offered to the Forum as a basis for discussion and is not a binding proposal on the part of any of the contributing organizations. The statements are subject to change in form and content after further study. Specifically, the contributors reserve the right to add to, amend or modify the statements contained herein. ************************************************************************* 1. Introduction --------------- In ABR traffic management the control loop is an end-to-end loop. All the resource management (RM) cells are sent from the source to the destination and then returned back to the source. Therefore, it may take as long as one round trip time (RTT) for these RM cells to affect the available cell rate (ACR) of the source. In the ATM Forum Traffic Management Specification Version 4.0 [1], Virtual Source/Virtual Destination (VS/VD) is proposed to allow the control loop to be segmented into several shorter control loops. A VS works like a real source in the sense that it controls the transmission rate of the virtual circuit (VC) just as a real source does. Similarly, a VD works like a real destination. A single switch can be both a VD for the previous control loop and a VS for the next control loop. As a result, the end-to-end control is replaced by segment-by-segment control. In our previous study [4], it was shown that VS/VD can reduce the response time during the first round trip and improve convergence time. In this study, we extend our work on VS/VD by studying its effect on switch buffer requirements. This contribution is organized as follows. Section 2 briefly describes the switch and VS/VD algorithms implemented. Then, we describe the simulation set-up, which includes the switch algorithm parameters, VS/VD options, and different configurations for our simulation. Finally, we present simulation results and their analysis in section 4. 2. Design and Implementation of VS/VD ------------------------------------- The simulations are based on our switch algorithm, Explicit Rate Indication for Congestion Avoidance (ERICA+), and our implementation of VS/VD. For detailed information about the algorithm, see [2], [3] and [4]. The ERICA+ algorithm allocates only a fraction of the total available capacity and the remaining capacity is used to drain queues. The fraction allocated is a function of the target queuing delay at the switch. Whenever the real queuing delay exceeds the target queuing delay, ERICA+ reduces the allocated ABR capacity. This reserves more bandwidth to drain the queue. If the actual queuing delay is smaller than this parameter, ERICA+ increases the allocated ABR capacity, which, in turn, increases the link utilization. A number of design alternatives for VS/VD were studied in [4]. After an extensive simulation study, it was concluded that the best design is one in which the VC's rate is measured at the output of the per-class queue, in the next loop, the total link input rate is measured at the input of the per-class queue in the next loop. Also, when a link becomes congested, the switch reduces the flow in both the next and the previous control loops. The rate allocation is recomputed both when an FRM is received on the previous loop and when a BRM is received on the next loop. This is the design that we used in the simulations reported here. 3. Simulation Set-up -------------------- VS/VD has little effect on the performance of LAN configurations. Therefore, in this study we consider only WAN configurations. The use of satellite links highlights the effect of feedback delays. Because of the large delay-bandwidth paths, the queues in the switches can be large. Normally, the satellite (ground) switches are designed with large buffers to allow for the large link delay. Other switches may not have that large buffers. We, therefore, purposely chose configurations in which the switch connected to the satellite links was not the bottleneck. We used three different such configurations. Configuration 1: ---------------- In this configuration, there are 5 sources sending to 5 destinations. The connection from each source to its corresponding destination crosses 3 switches as shown in Figure 1. LINK2 is the bottleneck because its bandwidth is 45 Mbps while all other links are 155 Mbps. The one-way link latencies are as follows: L1 (Src to SW1) = 275ms, LINK1 = 5ms, LINK2 = 5ms, L2 (SW3 to Dest) = 5us ----- ------ |Src 1|\ /|Dest 1| ----- \ / ------ . \ --- --- --- / . . ---| | LINK1 | | LINK2 | |--- . . L1 |SW1|-------|SW2|=======|SW3| L2 . . ---| | | | | |--- . / --- --- --- \ ----- / <---- Satellite Link \ ------ |Src 5|/ |Dest 5| ----- ------ Figure 1: Five sources/three switches configuration ----------------------------------------------------- The purpose of this configuration is to see whether the bottleneck queues are proportional to the total round-trip or only to the previous hop and whether VS/VD makes any difference. Configuration 2: ---------------- Here, we also have 5 connections, but each connection goes through 4 switches. The connections are bottlenecked at LINK3. See Figure 2. The one-way link latencies are as follows: L1 (Src to SW1) =5us, LINK1=275ms, LINK2=5ms, LINK3=5ms, L2 (SW4 to Dest)=5us LINK3 is a 45.0 Mbps link, while all other links are 155.52 Mbps. ----- ------ |Src 1|\ /|Dest 1| ----- \ / ------ . \ --- --- --- --- / . . ---| |LINK1| |LINK2| |LINK3| |--- . . L1 |SW1|-----|SW2|-----|SW3|=====|SW4|L2 . . ---| | | | | | | | |--- . / --- | --- --- --- \ ----- / | \ ------ |Src 5|/ L---> Satellite Link |Dest 5| ----- ------ Figure 2: Five sources/four switches configuration ----------------------------------------------------- Configuration 3: ---------------- This is the same as configuration 2. The only difference is that the latency of the link between SW2 and SW3 (LINK2) is 50 us instead of 5 ms. This helps study the senstivity to the previous loop's delay. Parameter Settings: ------------------- The following parameter values are used: -- All link bandwidths are 155.52 Mbps unless specified; -- All sources are infinite TCP sources; -- All buffers are infinite. There is no cell loss during the simulation. This enables us to measure the maximum queue length and assess buffer sizes at each hop; -- Peak Cell Rate is 155.52 Mbps; -- Initial Cell Rate(ICR) for TCP source configurations is 10 Mbps; -- All simulations run for 15 seconds; -- TCP timer granularity is 100 ms; -- TCP window size is 34,000 with a scale factor of 8, i.e., the real window size is 34,000 * 256. This ensures that the 155.52 Mbps satellite link pipe can be fully filled; -- The traffic is unidirectional. The sources send data. The destinations send only acknowledgments. 4. Simulation Results --------------------- The simulation results are shown in the following tables. In each table, the first column shows the options used in the simulation. We test two combinations for every configuration: ERICA+ without VS/VD, and ERICA+ with VS/VD. The remaining columns show the maximum queue size in the switches and the sources. Note that VS/VD requires per- VC queues and so there are several queues in the switch. The queue size reported here is the sum of all queues at the switch. Table 1, Table 2, and Table 3 are the simulation results for configuration 1, configuration 2, and configuration 3, respectively. Table 1: Maximum Queue Sizes of Configuration 1 =========================================================== | | Max Switch1 | Max Switch2 | Max Source | | Options | Queue Size | Queue Size | Queue Size | | | (cells) | (cells) | (cells) | =========================================================== |ERICA+ w/o VS/VD| 3,267 | 61,615 | 73,000 | |------------------------------------------------------------ |ERICA+ w. VS/VD| 7,100 | 4,700 |70,000~160,000| =========================================================== Table 2: Maximum Queue Sizes of of Configuration 2 =================================================================== | |Max Switch1|Max Switch2|Max Switch3| Max Source | | Options |Queue Size |Queue Size |Queue Size | Queue Size | | | (cells) | (cells) | (cells) | (cells) | =================================================================== |ERICA+ w/o VS/VD| 433 | 2 | 61,094 | 74,000 | |-------------------------------------------------------------------- |ERICA+ w. VS/VD| 95 | 10,200 | 9,000 |73,500~154,000| =================================================================== Table 3: Maximum Queue Sizes of Configuration 3 =============================================================== | |Max Switch1|Max Switch2|Max Switch3|Max Source| | Options |Queue Size |Queue Size |Queue Size |Queue Size| | | (cells) | (cells) | (cells) | (cells) | =============================================================== |ERICA+ w/o VS/VD| 384 | 2 | 61,157 | 29,300 | |---------------------------------------------------------------- |ERICA+ w. VS/VD| 113 | 8,000 | 8,800 | 33,000 | =============================================================== Note that in the non-VS/VD case, the bottlenecked switches, which are Switch2 in table 1 and Switch3 in table 2 and 3, have queue sizes of about 61,000 cells. For five sources with an ICR of 10 Mbps, and a bottleneck capacity of 45 Mbps, the input/output bandwidth difference is (5 * 10) - 45 = 5 Mbps. The feedback delay is 560 ms. Thus, the product of the feedback delay and the input/output bandwidth difference is 5 Mbps * 560 ms, which is 6,600 cells. The observed value, which is 61,000 cells, is 9 times as big as this. For the VS/VD case, the previous switches before the bottlenecked switches have large queue size. In table 1, it is 7,100 cells, this is about one time of the previous loop delay (550ms) times input/output bandwidth difference (5 Mbps), which is 6,500 cells. In table 2 and table 3, the bottlenecked switches have 10,200 cells and 8,000 cells, these are between 1 and 2 times of the above product. The only difference between configuration 2 and 3 is the latency of LINK2. The former is 5ms, the latter is 5us. This is showed in table 2 and 3 also. Since the feedback delay of configuration 2 is 10ms larger than configuration 3, the queue sizes at the switches following LINK2 in table 2 are a little bigger than those of table 3 in both non-VS/VD and VS/VD case. Observe that VS/VD moves part of the queue to the edge of the network. In table 1, the source queue size increases from 73,000 in the non-VS/VD case to more than 70,000 (from 70,000 to 160,000) in the VS/VD case. In table 2, the source queue size increases from 74,000 to more that 73,500 (from 73,500 to 154,000). And in table 3, it increase from 29,300 to 33,000. From the above simulation results, it appears that for a non-VS/VD switch, the maximum ABR queue size is a constant factor multiplied by the product of the feedback delay and link bandwidth. Since the feedback delay changes with different connections, this size also varies. This makes it difficult to determine the buffer size of a switch, because depending on its connections, the maximum buffer requirements are different. For a VS/VD switch, however, the queue size is bounded by the product of the previous loop delay and its bandwidth. This is because, with VS/VD, a bottlenecked switch can provide feedback on its congestion status to the previous switch in just the previous loop delay. When the previous switch receives this information, it reduces its transmission rate. Therefore, the queue size in the bottleneck switch is bounded. However, the previous switch becomes a new bottleneck. By doing the same thing as the first bottleneck switch, this switch can also pass this congestion information to its previous switch. As a result, the queue is propagated backwards. Every time it goes back one step, the degree of congestion is reduced. If the network cannot entirely manage the congestion, part of the queue is propagated to the edge of the network. Since the previous hop delay and the link bandwidth of a switch are known when it is connected to a network, the bound of the buffer requirements of this switch is local and fixed. Generally, the buffer requirements of a VS/VD switch are bounded by the maximum of the product of its previous loop delay and link bandwidth, and the product of its next loop delay and link bandwidth. The former is used to avoid cell loss, as explained above, while the latter is used to make full use of the output link bandwidth. An example of this situation is when ABR traffic is sharing a link with a higher priority VBR burst. When the VBR goes off, the ABR buffer should have enough cells to utilize the available bandwidth. 5. Conclusion ------------- In this contribution, we show by simulations that without VS/VD, the bottlenecked switch queue size can be approximately as large as the product of the link bandwidth and feedback delay. Therefore, the switch queue size is a function of the network diameter. With VS/VD, this large queue size is propagated along the previous switches. Normally, the maximum queue size at a switch is the product of its previous loop delay and the difference between its input bandwidth and the bottleneck bandwidth. It appears that with VS/VD, the maximum buffer requirements of a switch are bounded by the link bandwidth times previous/next loop delay. Thus, the buffer requirements of a VS/VD switch can be easily predicted. 6. References ------------- [1] ATM Forum Specification "ATM Traffic Management Specification Version 4.0," April 1996, ftp://ftp.atmforum.com/pub/approved-specs/af-tm-0056.000.ps [2] Raj Jain, Shiv Kalyanaraman, Rohit Goyal, Sonia Fahmy, and Ram Viswanathan "ERICA Switch Algorithm: A Complete Description," AF-TM 96-1172, August 1996 http://www.cis.ohio-state.edu/~jain/atmf/atm96-1172.ps [3] Raj Jain, Sonia Fahmy, Shivkumar Kalyanaraman, and Rohit Goyal "ABR Switch Algorithm Testing: A Case Study with ERICA," ATM Forum/96-1267, October 1996 http://www.cis.ohio-state.edu/~jain/atmf/atm96-1267.ps [4] Shivkumar Kalyanaranman, Raj Jain, Jianping Jiang, Rohit Goyal, and Sonia Fahmy "Virtual Source/Virtual Destination(VS/VD): Design Considerations" ATM Forum/96-1759, December 1996 http://www.cis.ohio-state.edu/~jain/atmf/atm96-1759.ps