Performance

Learning Outcomes
Resources
Lab
Assignment

Learning Outcomes

describe the impact of user location on application performance
address the location issue by locating services close to users
describe the impact of load on application performance
mitigate the impact of load using a web server tuned for static content where appropriate
mitigate the impact of load using either vertical or horizontal scaling
discuss the pros and cons of vertical vs horizontal scaling
explain how a content delivery network further improves the performance of web applications

Resources

Lab

Video walkthrough of this lab Part I - Distance and Vertical Scaling Video walkthrough of this lab Part II - Horizontal Scaling

Configure VM to Automatically Start Application on Boot

Assuming your VM is currently working and serving pages.

From the Google Cloud Shell, SSH back into the running instance:

gcloud compute ssh lab1 --zone YOUR_ZONE_GOES_HERE

Determine the command required to setup a startup script:

pm2 startup

The output of this command is the command you need to run to configure the system to start your application on boot. Copy and paste it exactly to the command prompt and run it.

Save the current configuration of running applications:

pm2 save

Log out of the instance (CTRL-D).

Impact of Service Location on Performance

Create an Image of Your VM

We are going to create an image of the VM so we can create similar instances later in this lab in other zones without having to install everything from scratch.

Go to the Google Cloud Platform Console.

Go to “COMPUTE | Compute Engine | VM Instances”

Check the box next to your instance. Click on the stop button.

Once the instance has stopped, go to “”, go to “COMPUTE | Compute Engine | STORAGE | Images”. Click on the “[+] CREATE IMAGE” button. For “Name”, enter “myapp-v1”. For “Source disk”, select “lab1”. Click on the “Create” button at the bottom of the page.

Resize Your VM

While the instance is stopped, we are going to resize it to the smallest possible size that has a dedicated core, rather than a shared core so that it will not impact our performance measurements.

Go to “COMPUTE | Compute Engine | VM Instances”.

Click on the “lab1” link to view the instance details. Click on the “EDIT” button. Under “Machine configuration | Machine type”, choose “Custom” and then use the sliders to choose 1 vCPU and 1 GB of memory. Click on the “SAVE” button at the bottom of the page.

Return to the page with the list of instances. Check the box next to your instance. Click on the “START/RESUME” button.

Once the instance has started, confirm the application still works in the browser. In Chrome, open “More tools | Developer tools” and go to the “Network” tool. Do a hard reload on the page a couple of times and note the page load times. If your application is running a zone close to you, the load time should easily be 100ms or less.

Start a VM in a Distant Zone

Go to “COMPUTE | Compute Engine | VM Instances”.

Click on the button to start a new instance.

For “Name”, enter “lab1-singapore”.

For “Region”, select “australia-southeast1 (Sydney)”.

For “Machine configuration | Series”, choose “N1”.

For “Machine configuration | Machine type”, choose “Custom” and then use the sliders to choose 1 vCPU and 1 GB of memory.

Under “Boot disk”, click on the “Change” button; choose “Custom images”; Under “Image”, select “myapp-v1”; and then click on the “Select” button at the bottom of the page.

Click the “Management, security, disks, networking, sole tenancy” link to expand the available options. Click on the “Networking” tab. For “Network tags”, enter “http-server-8080”.

Click on the “Create” button.

Test the application using the external IP address for the new instance in Singapore. In Chrome, open “More tools | Developer tools” and go to the “Network” tool. Do a hard reload on the page a couple of times and note the page load times. The times should be much longer than for the instance close to you

Benchmark Using “ab” Tool

Create a 3rd instance “benchmark” in the same zone as your original instance with a dedicated single core and 1 GB of memory using the standard Debian 10 boot disk.

gcloud compute ssh benchmark --zone YOUR_ZONE_GOES_HERE

Install the apache2-utils package:

sudo apt update
sudo apt install apache2-utils

Benchmark the original instance:

ab -n 100 http://NNN.NNN.NNN.NNN:8080/

using the IP address of the original instance. Note the mean time per request.

Benchmark the instance in Singapore:

ab -n 100 http://SSS.SSS.SSS.SSS:8080/

using the IP address of the instance in Singapore. Note the mean time per request.

Record the result of this ab command for the assignment.

Log out of the benchmark instance (CTRL-D).

Delete the Singapore instance by going to “COMPUTE | Compute Engine | VM Instances”. Check the box next to the Singapore instance. Click on the delete button.

Using a Static Web Server To Improve Performance

Node.js is pretty efficient but it still serves dynamically generated pages using an interpreted language. If you are only serving static content, you can improve performance by using a web server tuned for static content like NGINX.

Create a new instance “nginx” in the same zone as your original instance with a dedicated single core and 1 GB of memory using the standard Debian 10 boot disk. Also, under “Firewall” be sure to check “Allow HTTP traffic”

gcloud compute ssh nginx --zone YOUR_ZONE_GOES_HERE

Install the nginx package:

sudo apt update
sudo apt install nginx

Confirm that NGINX is working by visiting the home page for the instance in a new tab in your browser.

Log out of the “nginx” instance (CTRL-D).

gcloud compute ssh benchmark --zone YOUR_ZONE_GOES_HERE

Benchmark the “nginx” instance:

ab -n 100 http://GGG.GGG.GGG.GGG/

using the IP address of the “nginx” instance. Note the mean time per request.

Record the result of this ab command for the assignment.

Log out of the benchmark instance (CTRL-D).

Delete the “nginx” instance.

Scaling Applications Vertically

Add a CPU Bound Route to your Application

We are going to start by adding a CPU bound route to our application so we can benchmark more accurately. The performance of our simple index page is dominated by network latency and doesn’t have much to do with generating the page itself, so we are going to create a page that is dominated by the time to actually generate the page.

Create a new view, myapp/views/cpu-bound.ejs, with the following content:

<!DOCTYPE html>
<html>
  <head>
    <title><%= title %></title>
    <link rel='stylesheet' href='/stylesheets/style.css' />
  </head>
  <body>
    <h1><%= title %></h1>
    <p>Welcome to <%= title %></p>
    <table>
        <tr>
            <th>Variable</th>
            <th>Value</th>
        </tr>
        <tr>
            <td>Time (seconds)</td>
            <td><%= diff[0] %></td>
        </tr>
        <tr>
            <td>Time (nanoseconds)</td>
            <td><%= diff[1] %></td>
        </tr>
        <tr>
            <td>Count</td>
            <td><%= count %></td>
        </tr>
        <tr>
            <td>PID</td>
            <td><%= pid %></td>
        </tr>
        <tr>
            <td>hostname</td>
            <td><%= hostname %></td>
        </tr>
    </table>
  </body>
</html>

Create the corresponding route. All this route does is generate 30,000,000 random values and count the number that are above 0.5. Create a new route, myapp/routes/cpu-bound.js, with the following content:

const os = require('os');
var express = require('express');
var router = express.Router();

/* GET home page. */
router.get('/', function(req, res, next) {
    const pid = process.pid;
    const hostname = os.hostname();
    const start = process.hrtime();
    var i, count=0;
    for ( i=0; i<30000000; i++ ) {
        if ( Math.random() > 0.5 ) count++;
    }
    const diff = process.hrtime(start);
    res.render('cpu-bound', { title: 'CPU Bound', count, diff, pid, hostname });
});

module.exports = router;

Modify myapp/app.js to incorporate the new route. Modify myapp/views/index.js so there is a link to the new route on the home page.

Test the changes locally.

Stage, commit, and push the changes.

Verify that the new route is working on your VM.

gcloud compute ssh benchmark --zone YOUR_ZONE_GOES_HERE

Benchmark the new route with 16 requests:

ab -n 16 http://NNN.NNN.NNN.NNN:8080/cpu-bound

Note the values for “Requests per second” and “Time per request”.

Benchmark the new route with 16 requests but a concurrency level of 8:

ab -c 8 -n 16 http://NNN.NNN.NNN.NNN:8080/cpu-bound

Note the values for “Requests per second” and “Time per request”.

Scale Your Application Vertically

Go to “COMPUTE | Compute Engine | VM Instances”.

Stop the application instance.

Edit the machine type to be “n1-highcpu-8”.

Restart the instance. NOTE: The IP address has likely changed.

Test the application in the browser.

In another terminal SSH into the application instance.

Delete the application:

pm2 delete www

Change to the bin directory of the application:

cd myapp/bin

Restart the application using all 8 available vCPUs (substituting your actual values for YOUR_ACCESS_TOKEN and YOUR_WEBHOOK_SECRET):

GITLAB_PRIVATE_TOKEN=YOUR_ACCESS_TOKEN WEBHOOK_TOKEN=YOUR_WEBHOOK_SECRET PORT=8080 DEBUG=myapp:* pm2 start www -i max

The output should clearly indicate that there are 8 instances of the application running.

Log out of application instance.

Back in the terminal on the benchmark instance, benchmark the new route with 16 requests but a concurrency level of 8:

ab -c 8 -n 16 http://NNN.NNN.NNN.NNN:8080/cpu-bound

Note the values for “Requests per second” and “Time per request”.

Record the result of this ab command for the assignment.

Log out of the benchmark instance.

THE 8 CPU INSTANCE IS EXPENSIVE. BE SURE TO SCALE IT BACK DOWN TO 1 vCPU and 1 GB OF MEMORY.

Scaling Horizontally

Add a Health Check Page to Your Application

Create a new page, myapp/public/health-check.html, with the following content:

<!DOCTYPE html>
<html>
  <head>
    <title>Health Check</title>
  </head>
  <body>
    <h1>Health Check</h1>
    <p>I'm OK.</p>
  </body>
</html>

Test locally. Once it is working, stage, commit, and push the changes. Verify that the new page is working on the application VM.

Stop the application instance and create a new image, “myapp-v2”.

Create an Instance Template

Go to “COMPUTE | Compute Engine | Instance templates”.

Click on the “CREATE INSTANCE TEMPLATE” button.

For “Name”, enter “myapp-template”.

For “Machine configuration”, set up for an “N1” series with a custom 1 vCPU and 1 GB memory.

For “Boot disk”, select the “myapp-v2” image you created in the previous step.

Click the “Management, security, disks, networking, sole tenancy” link to expand the available options. Click on the “Networking” tab. For “Network tags”, enter “http-server-8080”. For “External IP”, select “None”.

Click on the “Create” button.

Create a Health Check Resource

Go to “COMPUTE | Compute Engine | Health checks”.

Click on the “CREATE A HEALTH CHECK BUTTON”.

For “Name”, enter “myapp-health-check”.

For “Scope”, select “Global”.

For “Protocol”, select “HTTP”.

For “Port”, enter “8080”.

For “Request path”, enter “/health-check.html”.

For “Check interval”, enter “10”.

For “Unhealthy threshold”, enter “3”.

Click on the “CREATE” button.

Create an Instance Group

Go to “COMPUTE | Compute Engine | Instance groups”.

Click on the “CREATE INSTANCE GROUP” button.

Make sure you are using the “New managed instance group (stateless)” option.

For “Name”, enter “myapp-group”.

For “Location”, select “Single zone”.

For “Region”, choose “us-west1 (Oregon)” or the region closest to you.

For “Instance template”, select “myapp-template”.

Scroll down to find the “Delete auto-scaling configuration” button and click it. Confirm the delete.

For “Number of instances”, enter “8”.

For “Health check”, select “mypp-health-check”.

Click on the “Create” button.

Create a Load Balancer

Go to “NETWORKING | Network services | Load balancing”.

Click on the “Create load balancer” button.

Choose “Start configuration” for “HTTP(S) Load Balancing”.

Choose “From Internet to my VMs” and “Continue”.

For “Name”, enter “myapp-lb”.

Select “Backend configuration”. From the drop-down menu, choose “CREATE A BACKEND SERVICE”.

For “Name”, enter “myapp-be”.
For “Backend type”, select “Instance group”.
Leave “Protocol”, “Named port”, and “Timeout” as is.
In the “Backends | New backend” section, for “Instance group”, select “myapp-group”.
For “Port numbers”, enter “8080”.
For “Balancing mode”, select “Rate”.
For “Maximum RPS”, enter “1”.
Click on the “DONE” link.
For “Health check”, select “myapp-health-check”
For “Logging”, uncheck “Enable logging”.
Click on the “CREATE” button.

Select “Host and path rules”.

Accept the default settings.

Select “Frontend configuration”.

For “Name”, enter “myapp-fe”.
For “Protocol”, select “HTTP”.
For “Network Service Tier”, select “Premium”.
For “IP vresion”, select “IPv4”.
For “IP address”, select “Ephemeral”.
For “Port”, select “80”.
Click on the “DONE” link.

Click on the main “CREATE” button.

It may take several minutes for the load balancer to become functional, be patient.

Click on the “myapp-lb” link.

You will see the IP:Port of the frontend. You can paste it in a new browser tab to test the application. It may take several more minutes to become functional; continue to be patient. If you test the address and see errors, wait a few minutes and try again.

Once it is working, go to the /cpu-bound URL. Refresh the page several times. The hostname should change occasionally if the load balancing is working.

Benchmark your Load balancer

Go back to you Cloud Shell. SSH into the benchamark instance. Benchmark the load balancer:

ab -c 8 -n 16 http://NNN.NNN.NNN.NNN:8080/cpu-bound

Note the values for “Requests per second” and “Time per request”.

Record the result of this ab command for the assignment.

Clean up

THE LOAD BALANCER AND THE INSTANCE GROUP ARE EXPENSIVE. BE SURE TO DELETE THEM WHEN YOU HAVE COMPLETED THIS EXERCISE

Assignment

In a text document, record the ab results for:

Sydney, Australia
NGINX
Vertical Scaling
Horizontal Scaling