Difference between revisions of "CSE 503S Performance Evaluation Study"
From CSE330 Wiki
Jump to navigationJump to search| (22 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | Students enrolled in CSE 530S will complete a performance analysis study as part of their final project. The goal of this analysis is to measure the performance of your | + | Students enrolled in CSE 530S will complete a performance analysis study as part of their final project. The goal of this analysis is to measure the performance of your server. These experiments do not need to be performed on your Creative Project. Instead, it is recommended that you focus the performance analysis on the lower-level applications (such as web server and database performance). |
== Reading == | == Reading == | ||
| Line 5: | Line 5: | ||
The following article will contain information that will help you with this assignment. | The following article will contain information that will help you with this assignment. | ||
| − | * [[Performance Evaluation in Linux | + | * [[Tools for Performance Evaluation in Linux]] |
== Project == | == Project == | ||
| Line 13: | Line 13: | ||
==== Requirements ==== | ==== Requirements ==== | ||
| − | * Choose two types of experiments to perform on your web server. The types of experiments you should run are largely dependent on type web application you are developing. Ask the instructor if you are unclear on what types of experiments | + | * Choose two types of experiments to perform on your web server. The types of experiments you should run are largely dependent on the type web application you are developing. Ask the instructor if you are unclear on what types of experiments to perform. Below is a small list of potential experiments to consider. |
| − | ** Apache web server performance evaluation on different AWS instance types | + | ** Apache web server performance evaluation on different AWS instance types [http://infoheap.com/ab-apache-bench-load-testing/] [https://www.joedog.org/siege-manual/][https://kalamuna.atlassian.net/wiki/spaces/KALA/pages/16023587/Load+Testing+With+Apache+Benchmark+and+Siege] |
| − | **MySQL Performance evaluation on different AWS instance types | + | **MySQL Performance evaluation on different AWS instance types |
| − | **Apache and Nginx comparison on the same AWS instance type[https://www.garron.me/en/go2linux/how-benchmark-stress-your-apache-nginx-or-iis-server.html] | + | **Apache and Nginx comparison on the same AWS instance type [https://www.garron.me/en/go2linux/how-benchmark-stress-your-apache-nginx-or-iis-server.html] |
| − | **MongoDB performance on different instance types [https://www.percona.com/blog/2015/07/14/mongodb-benchmark-sysbench-mongodb-io-bound-workload-comparison/] [https://www.mongodb.com/blog/post/performance-testing-mongodb-30-part-1-throughput-improvements-measured-ycsb] | + | **MongoDB performance evaluation on different instance types [https://www.percona.com/blog/2015/07/14/mongodb-benchmark-sysbench-mongodb-io-bound-workload-comparison/] [https://www.mongodb.com/blog/post/performance-testing-mongodb-30-part-1-throughput-improvements-measured-ycsb] |
| − | **Node and Apache+PHP comparison on the same AWS instance type | + | **Node.js and Apache+PHP comparison on the same AWS instance type |
| − | **Apache Web Server performance with and without optimizations and tuning[https://www.digitalocean.com/community/tutorials/how-to-configure-apache-content-caching-on-ubuntu-14-04][https://doc.owncloud.org/server/8.0/admin_manual/configuration_server/performance_tuning.html | + | **Apache Web Server performance with and without optimizations and tuning [https://www.digitalocean.com/community/tutorials/how-to-configure-apache-content-caching-on-ubuntu-14-04][https://doc.owncloud.org/server/8.0/admin_manual/configuration_server/performance_tuning.html] |
| − | **PHP performance with and without caching[http://www.sitepoint.com/understanding-opcache/] | + | **PHP performance with and without caching [http://www.sitepoint.com/understanding-opcache/] |
| − | **Elastic Load Balancing (ELB) evaluation using several smaller instances vs a single larger instance[https://aws.amazon.com/elasticloadbalancing/][http://harish11g.blogspot.com/2012/07/aws-elastic-load-balancing-elb-amazon.html] | + | **Elastic Load Balancing (ELB) evaluation using several smaller instances vs a single larger instance [https://aws.amazon.com/elasticloadbalancing/][http://harish11g.blogspot.com/2012/07/aws-elastic-load-balancing-elb-amazon.html] |
| − | *For each experiment collect data for a range of settings/inputs/configurations | + | *For each experiment collect data for a range of settings/inputs/configurations and at least two types of data. |
| − | ** | + | **An experiment using Apache Benchmark (ab) would be interested in the requests/sec and time per request |
| − | ** | + | **When evaluating a MySQL database, one could measure both reads/sec and writes/sec to a database |
| − | * Create a report that contains a description of your app, | + | * Create a report that contains a description of your web app, metrics you are interested in, experimental setup, and results |
| − | ** Provide a discussion of the evaluation and recommendations for deployment. If you are testing on different instance types discuss the cost benefits of a larger instance. (Is an instance that costs 10x that of a smaller instance worth | + | ** Provide a discussion of the evaluation and recommendations for deployment. If you are testing on different instance types discuss the cost benefits of a larger instance. (Is an instance that costs 10x that of a smaller instance worth the additional costs?) |
== Grading == | == Grading == | ||
| Line 37: | Line 37: | ||
# '''Two Experimental Evaluations (20 Points Per Experiment, 40 Points Total):''' | # '''Two Experimental Evaluations (20 Points Per Experiment, 40 Points Total):''' | ||
#* Experimental setup clearly explained (5 pts) | #* Experimental setup clearly explained (5 pts) | ||
| − | #* | + | #* Experimental results contain multiple data points ( 5 pts) |
| − | #* At least two different types of data recorded (6 pts) | + | #* At least two different types of data are recorded (6 pts) |
| − | #* | + | #* A well-formatted graph (not a table) of the experimental results for each type of data recorded (4 pts). I expect to see four graphs in total for your report. Two graphs per experiment. If you have questions about what is expected here, please ask the instructor |
# '''Write-up (20 Points): | # '''Write-up (20 Points): | ||
#* Detailed discussion of results (10 pts) | #* Detailed discussion of results (10 pts) | ||
| − | #* What were the bottlenecks of the system? (5 pts) | + | #* What were the potential bottlenecks of the system? (5 pts) |
#* What type of instance or configuration would you recommend? (5 pts) | #* What type of instance or configuration would you recommend? (5 pts) | ||
Latest revision as of 15:45, 23 August 2024
Students enrolled in CSE 530S will complete a performance analysis study as part of their final project. The goal of this analysis is to measure the performance of your server. These experiments do not need to be performed on your Creative Project. Instead, it is recommended that you focus the performance analysis on the lower-level applications (such as web server and database performance).
Contents
Reading
The following article will contain information that will help you with this assignment.
Project
You may work alone or with a partner on this assignment.
Requirements
- Choose two types of experiments to perform on your web server. The types of experiments you should run are largely dependent on the type web application you are developing. Ask the instructor if you are unclear on what types of experiments to perform. Below is a small list of potential experiments to consider.
- Apache web server performance evaluation on different AWS instance types [1] [2][3]
- MySQL Performance evaluation on different AWS instance types
- Apache and Nginx comparison on the same AWS instance type [4]
- MongoDB performance evaluation on different instance types [5] [6]
- Node.js and Apache+PHP comparison on the same AWS instance type
- Apache Web Server performance with and without optimizations and tuning [7][8]
- PHP performance with and without caching [9]
- Elastic Load Balancing (ELB) evaluation using several smaller instances vs a single larger instance [10][11]
- For each experiment collect data for a range of settings/inputs/configurations and at least two types of data.
- An experiment using Apache Benchmark (ab) would be interested in the requests/sec and time per request
- When evaluating a MySQL database, one could measure both reads/sec and writes/sec to a database
- Create a report that contains a description of your web app, metrics you are interested in, experimental setup, and results
- Provide a discussion of the evaluation and recommendations for deployment. If you are testing on different instance types discuss the cost benefits of a larger instance. (Is an instance that costs 10x that of a smaller instance worth the additional costs?)
Grading
This assignment is worth 60 points in total.
________
- Two Experimental Evaluations (20 Points Per Experiment, 40 Points Total):
- Experimental setup clearly explained (5 pts)
- Experimental results contain multiple data points ( 5 pts)
- At least two different types of data are recorded (6 pts)
- A well-formatted graph (not a table) of the experimental results for each type of data recorded (4 pts). I expect to see four graphs in total for your report. Two graphs per experiment. If you have questions about what is expected here, please ask the instructor
- Write-up (20 Points):
- Detailed discussion of results (10 pts)
- What were the potential bottlenecks of the system? (5 pts)
- What type of instance or configuration would you recommend? (5 pts)
