************************************************************ ATM Forum Document Number: ATM_Forum/97-1088 ************************************************************ Title: Modifications to the performance testing baseline text ************************************************************ Abstract: Several minor modifications are presented to get it ready for straw vote ballot. ************************************************************ Source: Gojko Babic, Raj Jain, Arjan Durresi, Justin Dolske, The Ohio State University Department of CIS Columbus, OH 43210-1277 Phone: 614-292-3989, Fax: 614-292-2911, Email: Jain@ACM.Org The presentation of this contribution at the ATM Forum is sponsored by NASA Lewis Research Center. ************************************************************ Date: December 1997 ************************************************************ Distribution: ATM Forum Technical Working Group Members (AF- TEST, AF-TM) ************************************************************ Notice: This contribution has been prepared to assist the ATM Forum. It is offered to the Forum as a basis for discussion and is not a binding proposal on the part of any of the contributing organizations. The statements are subject to change in form and content after further study. Specifically, the contributors reserve the right to add to, amend or modify the statements contained herein. ************************************************************ A PostScript version of this contribution has been uploaded to the ATM Forum. It is also available on our web page: http://www.cse.wustl.edu/~jain/atmf/a97-1088.htm A. Changes in Section 3.1. Throughput 3.1.4. Measurement Procedures Add the following paragraph at the end of the subsection: Before conducting the tests, it is recommended that the port clocks be synchronized or locked together; otherwise, an unstable delay may be observed. In case of instability, one solution is to reduce the maximum load to slightly below 100%. In this case, the load used should be reported. Some switches have independent clocks for each module or port. If these clocks run at different rates, it is possible for the input and output rates to be slightly different. In some cases, the queue length (buffers in the switches) may increase continuously resulting in instability. This change, suggested by Bob Mandeville of ENL France, fixes this problem of unsynchronized clocks, which has been observed in actual measurement experiments. 3.1.5. Foreground Traffic a. Delete the following two sentences: Use of the 1-to-(n1) multicast connection configuration for the foreground traffic is under study. Use of the n-to-(n1) multicast connection configuration for the foreground traffic is under study. The details of the foreground traffic characteristics were added in July 97 meeting and so these sentences are no longer required. b. Add the following paragraph after the paragraph ending with the sentence "See Figure 3.2f": Note that a generalization of 1-to-(n-1) multicast and n- to-(n-1) multicast is m-to-(n-1) multicast with 1 m n. This is to be consistent with the discussion on n-to-n straight and n-to-(n-1) full cross configurations, which are special cases of n-to-m partial cross. 3.1.6. Background traffic The current text of this subsection is: Higher priority traffic (like VBR or CBR) can act as background traffic for experiments. Further details of measurements with background traffic using multiple service classes simultaneously are under study. Until then, all testing will be done without any background traffic. Replace the above text with the following paragraph: In connection configurations with multiple VCCs, it is possible to use some VCCs for foreground traffic and the others for background traffic. One particularly interesting case is when one VCC of the n-to-n straight configuration is used for foreground and the remaining n- 1 VCCs are used for background. This will help study the effect of background traffic on the quality of service of the foreground traffic. The background and the foreground traffics can be of the same or different classes. Further details of measurements with background traffic are under study. The special case added above, pointed out by Juha Heinen of Telecom Finland is of immediate interest in verifying if the minimum cell rate (MCR) guaranty to an ABR or GFR VCC is being met. Other more general cases are for further study. 3.1.7. Guidelines For Scaleable Test Configurations Add the following paragraph at the end of the subsection: In the case of unicast, it may not be possible to overload a port with only one generator. Using two generators in scaleable configurations may exhibit different behavior, such as overloading, that may not show up with one generator. This addition is introduced to indicate the well known limitation of one-generator scaleable configurations and was suggested by Bob Mandeville of ENL France. B. Changes in Section 3.2. Frame Latency 3.2.6 Background Traffic Delete the following sentence: Use of the 1-to-(n1) multicast and n-to-(n1) multicast connection configuration for the background traffic is under study. The details of the background traffic characteristics were added in July 97 meeting and so this sentence is no longer required. Sections 3.2.6 through 3.2.8: Throughput these three subsections interchange w and n. This will make the use of n consistent with rest of the document. In the rest of the document, n is the number of ports in the switch. In these three subsections, n is used for the equivalent number of ports for background traffic and w is used for the total number of ports. C. Changes in Section 3.6. Call Establishment Latency 3.6.1. Definition a. Remove the following footnote: Applies only if cells of Setup and Connect messages are contiguous at the input port. This footnote was included at the early stages of MIMO latency definition. At that time, MIMO latency was defined only for contiguous frames at input. MIMO latency definition has now been generalised and so the footnote is no longer needed. b. Remove the paragraph, which reads: Recall that the MIMO latency for a frame is defined as the minimum of last-bit-in-to-last-bit-out (LILO) and the difference of first-bit-in-to-last-bit-out (FILO) and normal frame output time (NFOT). MIMO Latency = Min{LILO, FILO-NFOT} The MIMO latency definition has changed. The above paragraph reflects old definition. There is no need to repeat the definition here again. 3.6.3. Configurations Add the following two paragraphs at the end: It has been shown that the values of traffic contract and quality of service parameters may influence the processing time of Setup and Connect messages. Values of those parameters for which measurements should be performed are for further study. Measurement can be performed with or without background traffic. Further details of measurements with background traffic are under study. This section already talks about the influence of PNNI group hierarchy on the call establishment latency. The above changes add other aspects, which are known to affect the latency. 3.6.4. Statistical Variations Remove the following two sentences: Each time a different node pair is selected randomly as the source and destination end system. For a singe n-port switch it is expected that all n ports are equally probable candidates to be source and destination end-systems. Measurements have shown that results for two pairs of source and destination end-systems are not statistically different as long as both source end systems are connected to the same switch and both destination end systems are connected to the same switch. There is no need to introduce additional complexity required by random selection. D. Changes in Section 3.7. Application Goodput This section should be removed completely. Application goodput and the frame loss ratio are related as follows: Frame loss ratio = 1 Application goodput Thus, the application goodput metric does not provide any new information.