CSE 503S Performance Evaluation Study

Students enrolled in CSE 530S will complete a performance analysis study as part of their final project. The goal of this analysis is to measure the performance of your web application. An additional goal is to increase the performance of your web application after we have completed a baseline measurement.

Reading

The following article will contain information that will help you with this assignment.

• Performance Evaluation in Linux and AWS

Project

You may work alone or with a partner on this assignment.

Requirements

• Choose two types of experiments to perform on your web server. The types of experiments you should run are largely dependent on type web application you are developing. Ask the instructor if you are unclear on what types of experiments you should perform. Below is a small list of potential experiments to perform.
  • Apache web server performance evaluation on different AWS instance types [1] [2] [3][4]
  • MySQL Performance evaluation on different AWS instance types[5]
  • Apache and Nginx comparison on the same AWS instance type[6]
  • MongoDB performance on different instance types [7] [8]
  • Node and Apache+PHP comparison on the same AWS instance type [9]
  • Apache Web Server performance with and without optimizations and tuning[10][11][12]
  • PHP performance with and without caching[13]
  • Elastic Load Balancing (ELB) evaluation using several smaller instances vs a single larger instance[14][15]

• For each experiment collect data for a range of settings/inputs/configurations (at least 5 data points) and two types of data.
  • For example if I was using Apache Benchmark (ab) to test my web server I would be interested in the pages/sec and the mean time per request
  • For a mySQL database evaluation I could measure both and reads/sec and writes/sec to a database

• Create a report that contains a description of your app, the metrics you are interested in, the baseline and improved performance.
  • Provide a discussion of the evaluation and recommendations for deployment. If you are testing on different instance types discuss the cost benefits of a larger instance. (Is an instance that costs 10x that of a smaller instance worth it?)

Grading

This assignment is worth 60 points in total.

________

1. Two Experimental Evaluations (20 Points Per Experiment, 40 Points Total):
   • Experimental setup clearly explained (5 pts)
   • Tests performed with multiple data points (at least 5 data points) ( 5 pts)
   • At least two different types of data recorded (6 pts)
   • Visualization of experimental results (4 pts)

1. Write-up (20 Points):
   • Detailed discussion of results (10 pts)
   • What were the bottlenecks of the system? (5 pts)
   • What type of instance or configuration would you recommend? (5 pts)