************************************************************************ ATM Forum Document Number: ATM Forum/97-0613 ************************************************************************ Specification - Baseline Text ************************************************************************ Abstract: Improved text for throughput fairness and frame loss ratio measurement for throughput section of the baseline text is presented. ************************************************************************ Source: Gojko Babic, Arjan Durresi, Raj Jain, Justin Dolske. The Ohio State University Department of CIS Columbus, OH 43210-1277 Phone: 614-292-3989, Fax: 614-292-2911, Email: Jain@ACM.Org The presentation of this contribution at the ATM Forum is sponsored by NASA. ************************************************************************ Date: July 1997 ************************************************************************ Distribution: ATM Forum Technical Working Group Members (AF-TEST, AF-TM) ************************************************************************ Notice: This contribution has been prepared to assist the ATM Forum. It is offered to the Forum as a basis for discussion and is not a binding proposal on the part of any of the contributing organizations. The statements are subject to change in form and content after further study. Specifically, the contributors reserve the right to add to, amend or modify the statements contained herein. ************************************************************************ This contribution is a resubmission of a part of our contribution 97-0426 submitted in April. In this, we explain the proposed changes to the section 3.3 (throughput fairness) and 3.4 (frame loss ratio)of the baseline text. As instructed in the last meeting, this contribution has three parts. In the first part, we describe the changes. The second part contains the proposed text and the third part shows the changes from current baseline. [The third part is included only in the postscript version.] *********************************************************************** A postcript version of this contribution icluding all figures and tables has been uploaded to the ATM Forum ftp server in the incoming directory. It may be moved from there to the atm97 directory. The postcript version is also available on our web page as: http://www.cse.wustl.edu/~jain/atmf/a97-0613.htm A1. Explanations of the changes in the section "3.3. Throughput Fairness". a. Old 3.3.1. is split into new 3.3.1. and 3.3.2. in order to reach a clearer explanation. b. 3.3.1.-Peak-throughput fairness and full-load throughput fairness introduced as necessary in describing the throughput fairness of the switch plus some editorial changes of parts from old 3.3.1. c. 3.3.2.- New paragraph which includes parts of old 3.3.1. plus editorial changes d. Old 3.3.2. removed as unnecessary because load level and traffic pattern issues are discussed in the throughput measurement section 3.1.4. e. 3.3.3-Description of measurement procedures as necessary for the measurement process (new); f. 3.3.4. - Minor changes from old 3.3.3 g. Old 3.3.4 removed because the background traffic is for further study. h. 3.3.5.- Minor appropriate changes made. Part of it is covered now by Section 3.3.1 i. Old 3.3.6. removed as not needed. A2. Explanations of the changes in the section "3.4. Frame Loss Ratio". j. 3.4.1. - Minor editorial changes k. 3.4.2. - Unchanged l. Old 3.4.3 removed because the traffic pattern issue is the same as in throughput measurements. m. New 3.4.3 describes the measurement procedure n. 3.4.4. Removed mean and /or standard deviation since there is only one sample of the three frame-level throughput metrics. o. 3.4.5. - Minor appropriate changes made p. Old 3.4.6. removed as not needed. B1. Revised text. 3.3. THROUGHPUT FAIRNESS 3.3.1. Definition There are two throughput fairness metrics that are of interest to users: * Peak-throughput fairness: this is the fairness at a frame load for the peak throughput. * Full-load throughput fairness: This is the fairness at a frame load for the full-load throughput. Given n virtual circuits sharing a system (a single switch or a network of switches) and contending for the resources, throughput fairness indicates how far the actual individual allocations are from the ideal allocations. In the simplest case for a total throughput T, the ideal allocation should be T/n. We consider that in the most general case, the ideal allocation is defined by max-min allocation and that allocation is to be used. If the actual measured throughputs of n virtual circuits are found to be{T1, T2, ..., Tn}, where the ideal throughputs should be {t1 , t2 , ..., tn }, then the throughput fairness of the system under test is quantified by the "fairness index" computed as follows: Fairness index = sum(xi)^2 / (n x sum(xi^2)) where: xi = Ti/ti is the relative allocation to ith VC. Note that fairness index is not limited to throughput. It can be applied to other metrics, such as latency. However, extreme unfairness in latency is expected show up as unfairness in throughput and vice versa.Therefore, it is not required to quantify fairness of latency. 3.3.2. Units This fairness index is dimension-less. The units used to measure the throughput (bits/sec, cells/sec, or frames/sec) do not affect its value.In addition, the fairness index has the following desirable properties: * It is a normalized measure that ranges between zero and one. The maximum fairness is 100% and the minimum 0%. This makes it intuitive to interpret and present. * If all xi's are equal, the allocation is fair and the fairness index is one. * If n-k of n xi's are zero, while the remaining k xi's are equal and non-zero, the fairness index is k/n. Thus, a system which allocates all its capacity to 80% of VCs has a fairness index of 0.8 and so on. 3.3.3. Measurement procedures To measure a peak throughput fairness, the peak throughput for the given SUT has to be first obtained as described in 3.1.4. An experiment for peak throughput fairness is performed by generating the input load corresponding to the peak throughput and recording throughput for each foreground virtual circuit. The experiment is repeated p times. Here p is a parameter and its default value is 30. To measure a full throughput fairness, the full-load throughput for the given SUT has to be first obtained as described in 3.1.4. Then experiments for full-load throughput fairness are performed similarly to peak throughput fairness experiments. 3.3.4. Statistical Variations Let Fi be the fairness for the ith throughput experiment, then the mean fairness is computed as follows: Mean Fairness = sum(Fi) / p 3.3.5. Reporting Results Values of the mean fairness for peak and lossless throughput (with indication of a number of experiments) are reported along with a detailed description of the SUT, foreground traffic characteristics, and background traffic characteristics (if any), as defined in 3.1.8. B2. Revised text. 3.4 Frame loss ratio 3.4.1 Definition Frame loss ratio is defined as the fraction of frames that are not forwarded by a system under test (SUT) due to lack of resources. Partially delivered frames are considered lost. Frame loss ratio = (Input frame count - output frame count)/(input frame count) There are two frame loss ratio metrics that are of interest to a user: * Peak-throughput frame loss ratio:- This is the frame loss ratio at a frame load for the peak throughput. * Full-load frame loss ratio:- This is the frame loss ratio at a frame load for the full-load throughput. 3.4.2. Units The frame loss ration is expressed as a fraction of input frames. 3.4.3. Measurement Procedures The frame loss ratio metric is related to the throughput: Frame Loss Ratio = (Input Rate - Throughput)/Input Rate Thus, no additional experiments are required for frame loss ratios. These can be derived from tests performed for throughput measurements. 3.4.4. Statistical Variations Since there is only one sample for any of the three frame-level throughput metrics, there is no need for calculation of the means and/or standard deviations of frame loss ratio. 3.4.5. Reporting Results Values of the frame loss ratios for peak and lossless are reported along with a detailed description of the SUT, foreground traffic characteristics, and background traffic characteristics (if any), as defined in 3.1.8.