Difference between revisions of "CSE 503S Performance Evaluation Study"

From CSE330 Wiki
Jump to navigationJump to search
 
(27 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
Students enrolled in CSE 530S will complete a performance analysis study as part of their final project.  The goal of this analysis is to measure the performance of your web application. An additional goal is to increase the performance of your web application after we have completed a baseline measurement.
+
Students enrolled in CSE 530S will complete a performance analysis study as part of their final project.  The goal of this analysis is to measure the performance of your server. These experiments do not need to be performed on your Creative Project. Instead, it is recommended that you focus the performance analysis on the lower-level applications (such as web server and database performance).
  
 
== Reading ==
 
== Reading ==
Line 5: Line 5:
 
The following article will contain information that will help you with this assignment.
 
The following article will contain information that will help you with this assignment.
  
−
* [[Performance Evaluation in Linux and AWS]]
+
* [[Tools for Performance Evaluation in Linux]]
  
 
== Project ==
 
== Project ==
Line 13: Line 13:
 
==== Requirements ====
 
==== Requirements ====
  
−
* Choose at least two types of experiments to perform on your web server. The types of experiments you should run are largely dependent on type web application you are developing.  Ask the instructor if you are unclear on what types of experiments you should perform. Below is a small list of potential experiments to perform.
+
* Choose two types of experiments to perform on your web server. The types of experiments you should run are largely dependent on the type web application you are developing.  Ask the instructor if you are unclear on what types of experiments to perform. Below is a small list of potential experiments to consider.
−
** Apache web server performance evaluation on different AWS instance types [http://www.appneta.com/blog/web-app-performance-apache-bench/] [http://infoheap.com/ab-apache-bench-load-testing/] [https://www.joedog.org/siege-manual/][https://kalamuna.atlassian.net/wiki/display/KALA/Testing+With+Apache+Benchmark+and+Siege]
+
** Apache web server performance evaluation on different AWS instance types [http://infoheap.com/ab-apache-bench-load-testing/] [https://www.joedog.org/siege-manual/][https://kalamuna.atlassian.net/wiki/spaces/KALA/pages/16023587/Load+Testing+With+Apache+Benchmark+and+Siege]
−
**MySQL Performance evaluation on different AWS instance types[http://wiki.mikejung.biz/Sysbench]
+
**MySQL Performance evaluation on different AWS instance types
−
**Apache and Nginx comparison on the same AWS instance type[https://www.garron.me/en/go2linux/how-benchmark-stress-your-apache-nginx-or-iis-server.html]
+
**Apache and Nginx comparison on the same AWS instance type [https://www.garron.me/en/go2linux/how-benchmark-stress-your-apache-nginx-or-iis-server.html]
−
**MongoDB performance on different instance types [https://www.percona.com/blog/2015/07/14/mongodb-benchmark-sysbench-mongodb-io-bound-workload-comparison/]
+
**MongoDB performance evaluation on different instance types [https://www.percona.com/blog/2015/07/14/mongodb-benchmark-sysbench-mongodb-io-bound-workload-comparison/] [https://www.mongodb.com/blog/post/performance-testing-mongodb-30-part-1-throughput-improvements-measured-ycsb]
−
**Node and Apache+PHP comparison on the same AWS instance type [http://www.hostingadvice.com/blog/comparing-node-js-vs-php-performance/]
+
**Node.js and Apache+PHP comparison on the same AWS instance type
−
**Apache Web Server performance with and without optimizations and tuning[https://www.digitalocean.com/community/tutorials/how-to-configure-apache-content-caching-on-ubuntu-14-04][https://doc.owncloud.org/server/8.0/admin_manual/configuration_server/performance_tuning.html][http://lunarpages.com/uptime/5-ways-optimize-apache-performance]
+
**Apache Web Server performance with and without optimizations and tuning [https://www.digitalocean.com/community/tutorials/how-to-configure-apache-content-caching-on-ubuntu-14-04][https://doc.owncloud.org/server/8.0/admin_manual/configuration_server/performance_tuning.html]
−
**PHP performance with and without caching[http://www.sitepoint.com/understanding-opcache/]
+
**PHP performance with and without caching [http://www.sitepoint.com/understanding-opcache/]
−
**Elastic Load Balancing (ELB) evaluation using several smaller instances vs a single larger instance[https://aws.amazon.com/elasticloadbalancing/][http://harish11g.blogspot.com/2012/07/aws-elastic-load-balancing-elb-amazon.html]
+
**Elastic Load Balancing (ELB) evaluation using several smaller instances vs a single larger instance [https://aws.amazon.com/elasticloadbalancing/][http://harish11g.blogspot.com/2012/07/aws-elastic-load-balancing-elb-amazon.html]
  
−
** Benchmark your current ec2 instance's web server's performance using the tool ab from a separate ec2 instance (m4.large)
+
*For each experiment collect data for a range of settings/inputs/configurations and at least two types of data.
−
+
**An experiment using Apache Benchmark (ab) would be interested in the requests/sec and time per request
−
* Graph to performance of your web server, specifically the requests per second and time per request
+
**When evaluating a MySQL database, one could measure both reads/sec and writes/sec to a database
  
−
* Select two additional metrics that are important for your web application and measure them as well to establish a base line performance (see list of examples)
+
* Create a report that contains a description of your web app, metrics you are interested in, experimental setup, and results
−
 
+
** Provide a discussion of the evaluation and recommendations for deployment. If you are testing on different instance types discuss the cost benefits of a larger instance. (Is an instance that costs 10x that of a smaller instance worth the additional costs?)
−
* Determine three improvements you will make on your web server to improve these key metrics. (one of them may be to use a faster instance)
 
−
 
 
−
* Benchmark the performance of the improved system
 
−
 
 
−
* Graph the performance of the improved
 
−
 
 
−
* Create a report that contains a description of your app, the metrics you are interested in, the baseline and improved performance.
 
−
 
 
−
* Provide a discussion of the proposed improvements and the benefits it offered. (is an instance that costs 10x worth it?)
 
  
 
== Grading ==
 
== Grading ==
Line 44: Line 35:
 
________
 
________
  
−
# '''Baseline performance (20 Points):'''
+
# '''Two Experimental Evaluations (20 Points Per Experiment, 40 Points Total):'''
−
#* ab benchmark executed results of website
+
#* Experimental setup clearly explained  (5 pts)
−
#* measurements of two other initial metrics
+
#* Experimental results contain multiple data points ( 5 pts)
−
#* graph of initial performance
+
#* At least two different types of data are recorded (6 pts)
−
# '''Upgraded performance (20 Points):  
+
#* A well-formatted graph (not a table) of the experimental results for each type of data recorded (4 pts). I expect to see four graphs in total for your report. Two graphs per experiment. If you have questions about what is expected here, please ask the instructor
−
#* Benchmark each improvement individually
+
 
−
#* Graph the results
+
# '''Write-up (20 Points):  
−
# '''Performance Analysis Report (20 Points)
+
#* Detailed discussion of results  (10 pts)
 +
#* What were the potential bottlenecks of the system?  (5 pts)
 +
#* What type of instance or configuration would you recommend? (5 pts)

Latest revision as of 15:45, 23 August 2024

Students enrolled in CSE 530S will complete a performance analysis study as part of their final project. The goal of this analysis is to measure the performance of your server. These experiments do not need to be performed on your Creative Project. Instead, it is recommended that you focus the performance analysis on the lower-level applications (such as web server and database performance).

Reading

The following article will contain information that will help you with this assignment.

Project

You may work alone or with a partner on this assignment.

Requirements

  • Choose two types of experiments to perform on your web server. The types of experiments you should run are largely dependent on the type web application you are developing. Ask the instructor if you are unclear on what types of experiments to perform. Below is a small list of potential experiments to consider.
    • Apache web server performance evaluation on different AWS instance types [1] [2][3]
    • MySQL Performance evaluation on different AWS instance types
    • Apache and Nginx comparison on the same AWS instance type [4]
    • MongoDB performance evaluation on different instance types [5] [6]
    • Node.js and Apache+PHP comparison on the same AWS instance type
    • Apache Web Server performance with and without optimizations and tuning [7][8]
    • PHP performance with and without caching [9]
    • Elastic Load Balancing (ELB) evaluation using several smaller instances vs a single larger instance [10][11]
  • For each experiment collect data for a range of settings/inputs/configurations and at least two types of data.
    • An experiment using Apache Benchmark (ab) would be interested in the requests/sec and time per request
    • When evaluating a MySQL database, one could measure both reads/sec and writes/sec to a database
  • Create a report that contains a description of your web app, metrics you are interested in, experimental setup, and results
    • Provide a discussion of the evaluation and recommendations for deployment. If you are testing on different instance types discuss the cost benefits of a larger instance. (Is an instance that costs 10x that of a smaller instance worth the additional costs?)

Grading

This assignment is worth 60 points in total.

________

  1. Two Experimental Evaluations (20 Points Per Experiment, 40 Points Total):
    • Experimental setup clearly explained (5 pts)
    • Experimental results contain multiple data points ( 5 pts)
    • At least two different types of data are recorded (6 pts)
    • A well-formatted graph (not a table) of the experimental results for each type of data recorded (4 pts). I expect to see four graphs in total for your report. Two graphs per experiment. If you have questions about what is expected here, please ask the instructor
  1. Write-up (20 Points):
    • Detailed discussion of results (10 pts)
    • What were the potential bottlenecks of the system? (5 pts)
    • What type of instance or configuration would you recommend? (5 pts)
Retrieved from "http://classes.engineering.wustl.edu/cse330/index.php?title=CSE_503S_Performance_Evaluation_Study&oldid=2927"