CSE 503S Performance Evaluation Study

Students enrolled in CSE 530S will complete a performance analysis study as part of their final project. The goal of this analysis is to measure the performance of your server. These experiments do not need to be performed on your Creative Project. Instead, it is recommended that you focus the performance analysis on the lower-level applications (such as web server and database performance).

Reading

The following article will contain information that will help you with this assignment.

• Tools for Performance Evaluation in Linux

Project

You may work alone or with a partner on this assignment.

Requirements

• Choose two types of experiments to perform on your web server. The types of experiments you should run are largely dependent on the type web application you are developing. Ask the instructor if you are unclear on what types of experiments to perform. Below is a small list of potential experiments to consider.
  • Apache web server performance evaluation on different AWS instance types [1] [2][3]
  • MySQL Performance evaluation on different AWS instance types
  • Apache and Nginx comparison on the same AWS instance type [4]
  • MongoDB performance evaluation on different instance types [5] [6]
  • Node.js and Apache+PHP comparison on the same AWS instance type
  • Apache Web Server performance with and without optimizations and tuning [7][8]
  • PHP performance with and without caching [9]
  • Elastic Load Balancing (ELB) evaluation using several smaller instances vs a single larger instance [10][11]

• For each experiment collect data for a range of settings/inputs/configurations and at least two types of data.
  • An experiment using Apache Benchmark (ab) would be interested in the requests/sec and time per request
  • When evaluating a MySQL database, one could measure both reads/sec and writes/sec to a database

• Create a report that contains a description of your web app, metrics you are interested in, experimental setup, and results
  • Provide a discussion of the evaluation and recommendations for deployment. If you are testing on different instance types discuss the cost benefits of a larger instance. (Is an instance that costs 10x that of a smaller instance worth the additional costs?)

Grading

This assignment is worth 60 points in total.

________

1. Two Experimental Evaluations (20 Points Per Experiment, 40 Points Total):
   • Experimental setup clearly explained (5 pts)
   • Experimental results contain multiple data points ( 5 pts)
   • At least two different types of data are recorded (6 pts)
   • A well-formatted graph (not a table) of the experimental results for each type of data recorded (4 pts). I expect to see four graphs in total for your report. Two graphs per experiment. If you have questions about what is expected here, please ask the instructor

1. Write-up (20 Points):
   • Detailed discussion of results (10 pts)
   • What were the potential bottlenecks of the system? (5 pts)
   • What type of instance or configuration would you recommend? (5 pts)