🚀 AWS - Optimizing Web App Performance with AWS CloudFront, New Relic Monitoring, and Terraform Deployment

🚀 AWS - Optimizing Web App Performance with AWS CloudFront, New Relic Monitoring, and Terraform Deployment

🚀 Introduction

In today's rapidly evolving digital landscape, deploying web applications efficiently and ensuring optimal performance is essential for delivering a seamless user experience. This project aims to address these critical aspects by leveraging Infrastructure as Code (IAC) through Terraform, harnessing the content delivery power of AWS CloudFront, and implementing robust monitoring using New Relic.

🎯 Objective

The objective of this post is to bring and spread a knowledge about #aws cloud services, how to ? and where to consume the #aws services to solve the real world business challenges

🚀 Use Case:

Manual Deployment
Terraform (IaC) Deployment
New Relic Integration

This project aims to address these critical aspects by leveraging Infrastructure as Code (IAC) through Terraform, harnessing the content delivery power of AWS CloudFront, and implementing robust monitoring using New Relic.

Automated deployment of web applications is at the heart of this initiative, as it not only streamlines the provisioning of cloud infrastructure but also lays the foundation for future scalability across multiple cloud platforms, free from vendor lock-in constraints. By utilizing Terraform, a cloud-agnostic open-source tool, we ensure that our infrastructure can adapt to evolving requirements seamlessly.

The project also emphasizes the acceleration of web application performance through AWS CloudFront, a content delivery network (CDN) service. By strategically caching content at edge locations around the world, CloudFront enhances the user experience, minimizing latency and ensuring swift content delivery.

To maintain a vigilant eye on the performance and health of our web application, we integrate New Relic monitoring. This comprehensive tool provides us with invaluable insights into infrastructure visibility and response monitoring. With the aid of AWS Lambda functions, we securely send logs from our AWS S3 bucket to New Relic for real-time analysis.

This project not only underscores the importance of automated deployment and performance optimization but also provides a detailed, step-by-step guide for implementing these solutions. Whether you're new to these technologies or a seasoned professional, this project equips you with the knowledge and tools to harness the full potential of Terraform, CloudFront, and New Relic for your web applications.

🛠️ Solution Diagram:

Image description

The web application should deploy with the following criteria’s & requirements:

  1. Requirement 1:

    • The web application should be deployed within the AWS Cloud platform.
  2. Requirement 2:

    • The web application should be accessed securely.
  3. Requirement 3:

    • The viewer’s experience should be seamless without any delays while accessing the web content and pages.
  4. Requirement 4:

    • The web application should be accessible from around the world with a custom domain: www.prafect.link.
  5. Requirement 5:

    • The web application logs should be stored in object storage and then sent to third-party monitoring solutions.
  6. Requirement 6:

    • The web application should be deployed in an automated way without any vendor locking tools.

Solution:
Let’s discuss and analyze the solution for the above requirements:

  1. Solution 1: AWS S3

    • From the AWS Cloud platform, there is a service called S3, which provides a service to deploy a static web application.
  2. Solution 2: AWS ACM

    • AWS ACM is the solution that provides a wildcard certificate through which the web application can be secured with SSL/TLS communication.
  3. Solution 3: AWS Cloudfront

    • AWS Cloudfront is the content delivery network service through which the web app content will be cached on edge locations nearby the viewers' proximity to improve the user's experience.
  4. Solution 4: AWS Route 53

    • AWS Route 53 provides a DNS service through which the web application can have its own custom domain: www.prafect.link.
  5. Solution 5: AWS S3

    • AWS S3 provides a service to store the logs and push the logs to an external monitoring tool. New Relic can be used, where S3 logs will be pushed to New Relic using a Lambda function.
  6. Solution 6: Terraform

    • Terraform is the solution to provision the resources and infrastructure in an Infrastructure as Code (IAC) way. It's a cloud-agnostic open-source tool that can further scale up based on the requirements for multi-cloud.

Project Cost Estimation:
(Note: This cost is not any actual cost; it's just an estimation based on high-level requirements. Prices may vary based on adding and removing services based on requirements.)
Ref: AWS Pricing Calculator

Image description

Host a Static Website on AWS: Services Costs

Tools & Technologies Covered:

  • AWS Cloud

  • AWS S3

  • AWS Certificate Manager

  • AWS Cloudfront

  • AWS Route 53

  • AWS Lambda

  • AWS CloudWatch

  • New Relic (Monitoring)

  • Terraform (Infrastructure as Code)

  • Visual Studio Code (IDE)

  • GitHub

Ref: AWS Services Documentation:
AWS S3: https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-bucket-intro.html
AWS
Cloudfront:
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.SimpleDistribution.html

Pre-requisite:

  1. AWS Free Tier Account

  2. AWS IAM User created with programmatic access

  3. AWS Route53 hosted domain

  4. Visual Studio Code configured

  5. Latest Terraform version installed

  6. GitHub Account

  7. Gitbash installed on desktop

  8. New Relic account set up

  9. Web application source code ready

AGENDA:
To achieve the goal of explaining the user how to configure each and every service within the console to make them familiar with the ins and outs of all service components, and then to present the automated way in which the entire web application resources are provisioned within a few minutes using Terraform (IaC).

a. Manual way web application configuration and optimizing web speed to accelerate web performance.

b. Automated way using Terraform to deploy the complete web application.


Image description

MANUAL DEPLOYMENT

1. Set up Web Application:

  • Step 1: Register a Custom Domain with Route 53

  • Step 2: Create Two Buckets

  • Step 3: Configure Your Root Domain Bucket for Website Hosting

  • Step 4: Configure Your Subdomain Bucket for Website Redirect

  • Step 5: Configure Logging for Website Traffic

  • Step 6: Upload Index and Website Content

  • Step 7: Upload an Error Document

  • Step 8: Edit S3 Block Public Access Settings

  • Step 9: Attach a Bucket Policy

  • Step 10: Test Your Domain Endpoint

  • Step 11: Add Alias Records for Your Domain and Subdomain

  • Step 12: Test the Website

2. Perform Latency Test without Cloudfront:

  • a. Test from Canada

  • b. Test from London

  • c. Test from Singapore

3. Configure Certificate Manager to Make the Website Secure:

  • a. Request a Certificate

  • b. Validate - Add CNAME Records to Route53

  • c. Redirection Bucket – Change the Request Server from HTTP to HTTPS

4. Accelerate Web Performance Using Cloudfront:

  • a. Create Cloudfront Distribution

  • b. Validate that Cloudfront Domain is Displayed on the Web

  • c. Point Cloudfront Endpoint with Route53 Custom Domain

  • d. Measure the Latency Test with Cloudfront

    • i. Test from Canada

    • ii. Test from London

    • iii. Test from Singapore

Implementation in Action:

  1. Set up Web application

Step 1: Register a custom domain with Route 53

Image description

Image description

Image description

Image description

Step 2: Create two buckets

Main bucket: prafect.link
Second bucket: www.prafect.link

Image description

Step 3: Upload index and website content
Step 4: Upload an error document

Go to local terminal
AWS Configure : provide credentials
Go to source code directory: webfiles
aws s3 sync . s3://prafect.link

Image description

Image description

Step 5: Configure your root domain bucket for website hosting
Enable Static web

Image description

Step 6: Configure your subdomain bucket for website redirect
Bucket redirection: www.prafect.link

Image description

Image description

Step 7: Edit S3 Block Public Access settings
Buck Policy:

Image description

Step 9: Test your domain endpoint

Image description

Verify that “index.html” is displayed the web page
Access web page using endpoint
S3 web endpoint
URL : http://prafect.link.s3-website-us-east-1.amazonaws.com
index.html

Image description

Verify that “error.html” is displayed the error page
Access “error.html”

Image description

Step 10: Add alias records for your domain and subdomain
Add Custom domain with Route53
Verify that hosted zone exist

Image description

Image description

Create A record to add custom domain to web application
Prafect.link
Point out alias to AWS S3 website Endpoint

Image description

Create second alias for www.prafect.link

Image description

Image description

Step 11: Test the website
Access the web application using custom domain
www.prafect.link

Image description

  1. Perform Latency test without Cloudfront a. Test from Canada b. Test from London c. Test from Singapore

Image description

Measure the performance of the web application
Certainly, here is the performance measurement presented in a table format:

OriginRegionPageDNS (ms)Connection (ms)Response (ms)
CanadaUS-east-1Home Page189.39102.67
Contact Us48.25154.24
Portfolio49.70123.31
LondonHome Page322.29
Contact Us347.94
Portfolio533.79
SingaporeHome Page789.50
Contact Us762.48
Portfolio792.65

Validate response header that it is coming from Amazon S3

Image description

Test 1: Access from Canada
Home Page

Image description

About US

Image description

Portfolio

Image description

Switch the location using VPN other than Canada to test the latency

Country : London

Image description

HomePage

Image description

Contact Us

Image description

Portfolio

Image description

Country: Singapore
HomePage

Image description

About Us

Image description

Portfolio

Image description

Image description

3.Configure the Certificate Manager
How to secure Web application?
1.Request a certificate
Certificate Manager
SSL Certificate
Request certificate

Image description

Image description

Status: Pending Validation

Image description

2.Create records in Route53

Image description

3.Go to Route53 and validate that two CNAME records created

Image description

SSL certificate issued successfully

Image description

Verify that web application is access using secure https
http://www.prafect.link : not secure

Image description

  1. Do one change on redirection bucket and redirect to “https” Change from “http” to “https”

Image description

Image description

5.Accelerate the web performance using Cloudfront

a. Create cloudfront distribution
b. Validate that cloudfront domain is displayed on the web
c. Point the cloudfront endpoint with the Route53 custom domain
d. Measure the Latency test with Cloudfront
i. Test from Canada
ii. Test from London
iii. Test from Singapore

Image description

Image description

Create Cloudfront distribution
Origin domain: Provide AWS S3 web endpoint:
http://prafect.link.s3-website-us-east-1.amazonaws.com

Image description

Image description

Use only North America and Europe

Image description

Image description

Image description

Image description

Image description

Image description

Cloudfront deployment in progress

Image description

Cloudfront deployed

Image description

Custom origin

Image description

Validate that cloudfront domain is correctly deployed

Image description

Access web application using cloudfront url
URL: https://d3hffwkpocdftx.cloudfront.net

Image description

Security Checkpoint:

How to Secure AWS S3 web end point and restrict to access the web application allowing access through only Cloudfront web endpoint ?

Go to S3 Bucket policy
Go to Origin
Edit Origin

Image description

Image description

Replace this Bucket policy with cloudfront policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::
prafect.link"
}
] }

{
"Version": "2008-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"Statement": [
{
"Sid": "AllowCloudFrontServicePrincipal",
"Effect": "Allow",
"Principal": {
"Service": "cloudfront.amazonaws.com"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::prafect.link/*",
"Condition
": {
"StringEquals": {
"AWS:SourceArn": "arn:aws:cloudfront::914141388779:distribution/EU0BLI2K9XWK3"
}
}
}
]
}

Restrict to access from S3 web endpoint and allow only from cloudfront distribution endpoint

Image description

Image description

Access Web url:
Expected: it should block the request using S3 endpoint : PASSED

Image description

Only allowed using cloudfront endpoint

Image description

Point out Cloudfront URL to Route53

Image description

Replace the s3 web endpoint with cloudfront endpoint
Delete the record sets

Image description

Create New record set for Prafect.link
Routing Policy: Simple Routing

Image description

Create New record set for www.Prafect.link

Image description

Record sets created successfully

Image description

Measure the performance of the web application
Perform Latency test

Image description

Image description

Image description

OriginRegionCloudfrontPageDNSConnectionSSLResponse
CanadaNo CloudfrontHome Page189.39 ms102.67 ms
With CloudfrontHome Page1.06 ms22.08 ms
No CloudfrontContact Us48.25 ms25.33 ms22.59 ms
With CloudfrontContact Us46.67 ms
No CloudfrontPortfolio49.70 ms123.31 ms
With CloudfrontPortfolio65.42 ms25.43 ms25.43 ms23.75 ms
LondonNo CloudfrontHome Page322.29 ms
With CloudfrontHome Page121.18 ms121.18 ms121.18 ms252.88 ms
No CloudfrontContact Us347.94 ms
With CloudfrontContact Us122.69 ms122.69 ms125.31 ms
No CloudfrontPortfolio533.79 ms
With CloudfrontPortfolio120.55 ms120.55 ms119.54 ms
SingaporeNo CloudfrontHome Page789.50 ms
With CloudfrontHome Page375.48 ms375.48 ms409.51 ms
No CloudfrontContact Us762.48 ms
With CloudfrontContact Us294.09 ms388.27 ms390.91 ms
No CloudfrontPortfolio792.65 ms
With CloudfrontPortfolio382.05 ms382.05 ms378.65 ms

Validate that web application is secure

Image description

Verify Header that the page is coming from cloudfront cache

Image description

Test from Canada
HomePage

Image description

About Us

Image description

Test from London

Image description

Home Page

Image description

Contact Us

Image description

Portfolio

Image description

Test from Singapore

Image description

Home Page

Image description

Contact Us

Image description

Portfolio

Image description

Optimize the Cloudfront performance
Point DNS to United Kingdom
Change the Routing Policy: Geolocation
Location: United Kingdom

Image description

Keep alive time

Image description

Change the timeout to 60

Image description

Cloudfront speed increase tips:
Cloudfront – Keep alive time need to increate: 60
DNS TTL time – need to increase: 60
SSL time need to increase

Image description

Cloudwatch

Image description

Image description

NEW RELIC

INTEGRATION

Objective: The objective is to monitor Cloudfront logs using the external monitoring tool New Relic and integrate it with AWS S3. This will be achieved by implementing comprehensive monitoring through an AWS Lambda function. The Lambda function will be configured to trigger and send Cloudfront logs from an AWS S3 bucket to New Relic, providing infrastructure visibility and response monitoring.

Technology & Tools: New Relic, AWS S3, AWS Lambda


New relic

Image description

Select AWS

Image description

Image description

Standard logs
https://docs.newrelic.com/docs/logs/forward-logs/cloudfront-web-logs/#enable-standard-logging

Create logs bucket in S3

Image description

Enable standard logging
Go to Cloudfront Enable standard logging

Image description

Go to this link: https://serverlessrepo.aws.amazon.com/applications
Search
: newrelic

Image description

Click :Deploy

Image description

1.Scroll to the Application settings and enter your New Relic license key.

Image description

Logtype: built in parsing rulesets : https://docs.newrelic.com/docs/logs/ui-data/built-log-parsing-rules/
Deployment
in progress

Image description

Lambda deployment complete

Image description

Create Lambda trigger

Image description

Add Trigger

Image description

Image description

Trigger added successfully

Image description

Go back to newrelic
If configure correct it will look like this

Image description

Image description

View logs

Image description

Image description

Image description

Image description

Objective: Automating Web Application Performance Enhancement

Objective: The objective is to automate the provisioning of services and configure CloudFront to improve the performance of a web application.

Technology & Tools: AWS S3, Terraform, CloudFront

Terraform
Terraform project source:
GitHub: https://github.com/prafulpatel16/terraform-aws-tests.git
Project
directory: gocloud-test/10-tf-static-web-complete/

WHAT IS TERRAFORM ?

HashiCorp Terraform is an infrastructure as code tool that lets you define both cloud and on-prem resources in human-readable configuration files that you can version, reuse, and share. You can then use a consistent workflow to provision and manage all of your infrastructure throughout its lifecycle. Terraform can manage low-level components like compute, storage, and networking resources, as well as high-level components like DNS entries and SaaS features.

How does Terraform work?

HashiCorp and the Terraform community have already written thousands of providers to manage many different types of resources and services. You can find all publicly available providers on the Terraform Registry, including Amazon Web Services (AWS), Azure, Google Cloud Platform (GCP), Kubernetes, Helm, GitHub, Splunk, DataDog, and many more.

The core Terraform workflow consists of three stages:

Definition: You define resources, which may span multiple cloud providers and services. For example, you might create a configuration to deploy an application on virtual machines in a Virtual Private Cloud (VPC) network with security groups and a load balancer.

Planning: Terraform creates an execution plan describing the infrastructure it will create, update, or destroy based on the existing infrastructure and your configuration.

Application: On approval, Terraform performs the proposed operations in the correct order, respecting any resource dependencies. For example, if you update the properties of a VPC and change the number of virtual machines in that VPC, Terraform will recreate the VPC before scaling the virtual machines.

Implementation Phase:

Phase 1: AWS Cloud Configuration via Terminal

  1. Go to GitBash terminal and execute the command "aws configure."

  2. Provide the IAM user's secret key and secret access key.

  3. Set the AWS region to "us-east-1."

  4. Specify the output type as "json."

Phase 2: Create Terraform Project and Define Terraform Files and Folders

  1. Open Visual Studio Code (VS Code) and create a new folder named "tf-static-web."

  2. Organize the file structure for the Terraform project: a. Create a new file "01-providers.tf." b. Create a new file "02-main.tf." c. Create a new file "03-variables.tf." d. Create a new file "04-outputs.tf." e. Create a new file "terraform.tfvars." f. Create a folder for the web application source code, named "webfiles," and upload the web application files into it.

  3. Save all the created files within VS Code.

Phase 3: Configure and Write Terraform Resources into Respective Files

  1. Open the "01-providers.tf" file and define the Terraform and provider blocks.

  2. Open the "02-main.tf" file and write all the necessary S3 resources.

  3. Open the "03-variables.tf" file and define the variables for dynamic access.

  4. Open the "04-outputs.tf" file and specify the required outputs.

  5. Open the "terraform.tfvars" file and configure variables for dynamic access that will override default variable values.

Phase 4: Terraform fmt, Terraform validate, Terraform plan

  1. Go to the Git Bash terminal and execute the following Terraform commands:

  2. Run "terraform init."

  3. Execute "terraform fmt."

  4. Validate the configuration with "terraform validate."

  5. Apply the configuration with "terraform apply."

  6. Observe that the "terraform.tfstate" is stored locally.

Phase 5: Verify that Resources Are Created in AWS Cloud

  1. Access the AWS S3 console and confirm that the bucket has been created.

  2. Verify that the web files have been uploaded to the bucket.

  3. Confirm that the necessary permissions have been configured.

  4. Ensure that the web endpoint has been created.

Phase 6: Verify that the Web Application Is Accessible Successfully

  1. In the terminal, examine the output values.

  2. Copy the endpoint URL and access it from a web browser.

  3. Confirm that the "index.html" displays the web page.

  4. Validate that the "error.html" is displayed as the error page.

AWS Configuration:

Go to terminal and type
$aws configure

Terraform file structure:

Image description

Phase 2: Create Terraform Project and Define Terraform Files and Folders

  1. Open Visual Studio Code (VS Code) and create a new folder named "tf-static-web."

  2. Organize the file structure for the Terraform project: a. Create a new file "01-providers.tf." b. Create a new file "02-main.tf." c. Create a new file "03-variables.tf." d. Create a new file "04-outputs.tf." e. Create a new file "terraform.tfvars." f. Create a folder for the web application source code, named "webfiles," and upload the web application files into it.

Image description

Phase 3: Configure and Write Terraform Resources into Respective Files

  1. Open the "01-providers.tf" file and define the Terraform and provider blocks.

  2. Open the "02-main.tf" file and write all the necessary S3 resources.

  3. Open the "03-variables.tf" file and define the variables for dynamic access.

  4. Open the "04-outputs.tf" file and specify the required outputs.

  5. Open the "terraform.tfvars" file and configure variables for dynamic access that will override default variable values.

01_Providers.tf

  • Terraform block

  • Providers block

  • Terraform block
    Providers file contains terraform block which includes a “backend” configurations for state file storage and also “required_provisioners” contains a source and version of the terraform provider.

  • Providers block

Image description

This block contains a cloud providers information about region and profile on which the cloud services needs to be provisioned.

02_main.tf

Image description

Image description

Image description

03_variables.tf

Image description

04_outputs.tf

Image description

05_terraform.tfvars.tf

Image description

Webfiles

Image description

AWS configured successfully

Phase 4: Terraform fmt, terraform validate, terraform plan

1) Go to git bash terminal and apply the terraform commands
2) Terraform init
3) Terraform fmt
4) Terraform validate
5) Terraform apply
6) Observe that “terraform.tfstate” is stored locally

Terraform init

Image description

Terraform fmt

Image description

Terraform validate

Image description

Image description

Terraform plan

Image description

Terraform apply

Image description

static-web-statefile
Terraform.tfstate

Image description

Phase 5: Verify that resources are created In AWS cloud

1) Go to AWS S3 and verify that bucket is created
2) Verify that web files are uploaded into bucket
3) Verify that permission has been created

Terraform destroy

Image description

Image description

4) Verify that web endpoint is created

Phase 6: Verify that web application accessible successfully

1) Go to terminal and observe the outputs value
2) Copy the endpoint url and access from web browser
3) Verify that “index.html” is displayed the web page
4) Verify that “error.html” is displayed the error page

Terraform destroy

Conclusion:

This project successfully optimized web application performance with AWS CloudFront, New Relic, and Terraform. We deployed a secure, globally accessible web app using Terraform's infrastructure as code (IAC). By fine-tuning latency, implementing SSL/TLS, and integrating CloudFront, we significantly improved user experience. Leveraging AWS Lambda, we monitored CloudFront logs and pushed data to New Relic, enhancing infrastructure visibility. This project demonstrates the power of cloud-native solutions and monitoring tools in delivering fast, reliable web applications.

Congratulations!!!! 🔥🚀

Let's Stay Connected:

  1. 🌐 Website: Visit my Website

  2. 💼 LinkedIn: Connect with me on LinkedIn

  3. 🐙 GitHub: Check out my GitHub

  4. 🎬 YouTube: Subscribe to my YouTube Channel

  5. ✍️ Medium: Read my articles on Medium

  6. 📝 Dev.to: Follow me on Dev.to

AWS #CloudEngineering #CloudComputing #AmazonWebServices #AWSArchitecture #DevOps #CloudSolutions #CloudSecurity #InfrastructureAsCode #AWSCertification #Serverless #AWSCommunity #TechBlogs #CloudExperts #CloudMigration #CloudOps #AWSJobs #TechIndustry #CareerInTech #InnovationInCloud #devops #cloudengineerjobs #devopsjobs #azure #gcp #oci #cloudjobs, #kubernetes