Amazon Web Services: “How to create High Availability Architecture by integrating S3 & CloudFront with EC2 attaching EBS Volume using AWS CLI 2?”
In today’s world, The number of Cloud-based applications is increasing rapidly along with consumers. To ensure optimal operational performance, the implementation of services and resources is required which is defined by The High Availability Architecture approach.
In AWS, This approach can be achieved with the Integration of S3 & CloudFront with EC2. So, Let’s continue to “Creating High Availability Architecture in AWS”. Using this approach, we will be able to create High Availability Architecture which will provide A Web Server running on EC2, storing web pages or codes on EBS Volume, fetching a static file from CloudFront distribution, and storing the static file on S3 Bucket.
The steps we need to follow are
- Creating EC2 Instance attaching EBS Volume and configuring Web Server.
- Creating S3 Bucket and uploading a file using AWS CLI2.
- Creating CloudFront Distribution of S3 Bucket file.
Creating EC2 Instance attaching EBS Volume and configuring Web Server
First, we need to launch an Operating System or Instance using EC2 Service. In my case, I am using RHEL 8 (Red Hat Enterprise Linux version 8).
Launching EC2 Instance
To launch EC2 Instance, we need to follow the below steps
Step 1) In AWS EC2 Service Dashboard, we need to click on the “Launch instances” button at the top right corner.
Step 2) After that, In the “Choose AMI” section, we need to select Red Hat Enterprise Linux 8–64 bit (in my case, I am selecting RHEL 8, you can select any of your choice or requirement).
Step 3) After that, In the “Choose Instance Type” section, we need to select “t2.micro”. (in my case, I am selecting t2.micro, you can select any of your choice or requirement).
Step 4) After that, In the “Configure Instance” section, we need to input only 1 in the “Number of instances” and let other configurations remain as default. (in your case, you can configure your instance according to your requirement).
Step 5) After that, In the “Add Storage” section, we need to set storage size and let other configurations remain as default. (in my case, I am using only 10 GiB of Disk Size, you can use any size according to your requirement).
Step 6) After that, In the “Add Tags” section, It is recommended to add a tag of your instance name in the key-value pair using Name as Key and Your instance name as Value. (This is optional but it’s a best practice to give your instance a name).
Step 7) After that, In the “Configure Security Group” section, we need to assign a security group to our instance. Either we can create a new security group or choose an existing one. In the Security Group, we need to add a Rule to allow All Traffic from Anywhere. (In my case, I am allowing all the traffic from all the sources but it is not recommended to allow everyone to access your instance).
Step 8) Finally, In the “Review” section, we should review all the configurations which we’ve made, and after we need to click on “Launch”.
Step 9) After launching, we need to create or select the key pair. (It is used to connect instances remotely).
After Successful creation of Instance, you will see the following output
To Know How to launch an instance using AWS CLI 2, read the article here.
Creating EBS Volume
To create EBS Volume, we need to follow the below steps
Step 1) In AWS EC2 Service Dashboard, we need to click on the “Volumes” under the Elastic Block Storage option at the left sidebar.
Step 2) After that, We need to click on “Create Volume” at the top left.
Step 3) In the “Create Volume” section, we need to enter the size of EBS Volume and select the same region where the EC2 Instance is running and let other configurations remain as default. (in my case, I am using 1 GiB of Size for EBS Volume). You can add a Name Tag for your Volume (This is optional but recommended).
Step 4) Finally, click on “Create Volume”. It will create the EBS Volume.
Attaching EBS Volume with EC2 Instance
To Attach EBS Volume with EC2 Instance, we need to follow the below steps
Step 1) In AWS EC2 Service Dashboard, we need to click on the “Volumes” under the Elastic Block Storage option at the left sidebar.
Step 2) After that, we need to select the EBS Volume which was created independently earlier by selecting the “Attach Volume” option from the Action menu.
Step 3) In the “Attach Volume” section, we need to select the instance from with volume is going to be attached and then click on “Attach”. This will attach the EBS Volume to the EC2 Instance.
Configuring Web Server in RHEL 8
Web Server is a server software that serves web pages to the clients. There are different kinds of Web Server software are available in the market. One of them is “httpd” also known as Apache Tomcat Server by Apache Software Foundation. Read more about this here.
To configure Web Server in RHEL 8, we need to follow the below steps
Step 1) In RHEL 8, we need to switch to the root account first if not switched by using this command
sudo su - rootStep 2) Next, to install the “httpd” package for Web / HTTP Server, we need to run this command
yum install httpdThe output will be
Last metadata expiration check: 0:13:25 ago on Mon 02 Nov 2020 10:27:12 AM UTC.
Dependencies resolved.
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
httpd x86_64 2.4.37-21.module+el8.2.0+5008+cca404a3
rhel-8-appstream-rhui-rpms 1.4 M
Installing dependencies:
apr x86_64 1.6.3-9.el8 rhel-8-appstream-rhui-rpms 125 k
apr-util x86_64 1.6.1-6.el8 rhel-8-appstream-rhui-rpms 105 k
httpd-filesystem noarch 2.4.37-21.module+el8.2.0+5008+cca404a3
rhel-8-appstream-rhui-rpms 36 k
httpd-tools x86_64 2.4.37-21.module+el8.2.0+5008+cca404a3
rhel-8-appstream-rhui-rpms 103 k
mailcap noarch 2.1.48-3.el8 rhel-8-baseos-rhui-rpms 39 k
mod_http2 x86_64 1.11.3-3.module+el8.2.0+7758+84b4ca3e.1
rhel-8-appstream-rhui-rpms 156 k
redhat-logos-httpd noarch 81.1-1.el8 rhel-8-baseos-rhui-rpms 26 k
Installing weak dependencies:
apr-util-bdb x86_64 1.6.1-6.el8 rhel-8-appstream-rhui-rpms 25 k
apr-util-openssl x86_64 1.6.1-6.el8 rhel-8-appstream-rhui-rpms 27 kTransaction Summary
================================================================================
Install 10 PackagesTotal download size: 2.0 M
Installed size: 5.5 M
Is this ok [y/N]: y
Downloading Packages:
(1/10): apr-util-1.6.1-6.el8.x86_64.rpm 917 kB/s | 105 kB 00:00
(2/10): apr-util-openssl-1.6.1-6.el8.x86_64.rpm 214 kB/s | 27 kB 00:00
(3/10): apr-util-bdb-1.6.1-6.el8.x86_64.rpm 193 kB/s | 25 kB 00:00
(4/10): httpd-2.4.37-21.module+el8.2.0+5008+cca 12 MB/s | 1.4 MB 00:00
(5/10): httpd-filesystem-2.4.37-21.module+el8.2 344 kB/s | 36 kB 00:00
(6/10): apr-1.6.3-9.el8.x86_64.rpm 1.2 MB/s | 125 kB 00:00
(7/10): mod_http2-1.11.3-3.module+el8.2.0+7758+ 1.6 MB/s | 156 kB 00:00
(8/10): httpd-tools-2.4.37-21.module+el8.2.0+50 851 kB/s | 103 kB 00:00
(9/10): mailcap-2.1.48-3.el8.noarch.rpm 328 kB/s | 39 kB 00:00
(10/10): redhat-logos-httpd-81.1-1.el8.noarch.r 269 kB/s | 26 kB 00:00
--------------------------------------------------------------------------------
Total 4.1 MB/s | 2.0 MB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : apr-1.6.3-9.el8.x86_64 1/10
Running scriptlet: apr-1.6.3-9.el8.x86_64 1/10
Installing : apr-util-openssl-1.6.1-6.el8.x86_64 2/10
Installing : apr-util-1.6.1-6.el8.x86_64 3/10
Running scriptlet: apr-util-1.6.1-6.el8.x86_64 3/10
Installing : apr-util-bdb-1.6.1-6.el8.x86_64 4/10
Installing : httpd-tools-2.4.37-21.module+el8.2.0+5008+cca404a3 5/10
Installing : redhat-logos-httpd-81.1-1.el8.noarch 6/10
Installing : mailcap-2.1.48-3.el8.noarch 7/10
Running scriptlet: httpd-filesystem-2.4.37-21.module+el8.2.0+5008+cca 8/10
Installing : httpd-filesystem-2.4.37-21.module+el8.2.0+5008+cca 8/10
Installing : mod_http2-1.11.3-3.module+el8.2.0+7758+84b4ca3e.1. 9/10
Installing : httpd-2.4.37-21.module+el8.2.0+5008+cca404a3.x86_6 10/10
Running scriptlet: httpd-2.4.37-21.module+el8.2.0+5008+cca404a3.x86_6 10/10
Verifying : apr-util-bdb-1.6.1-6.el8.x86_64 1/10
Verifying : apr-util-openssl-1.6.1-6.el8.x86_64 2/10
Verifying : apr-util-1.6.1-6.el8.x86_64 3/10
Verifying : httpd-2.4.37-21.module+el8.2.0+5008+cca404a3.x86_6 4/10
Verifying : httpd-filesystem-2.4.37-21.module+el8.2.0+5008+cca 5/10
Verifying : apr-1.6.3-9.el8.x86_64 6/10
Verifying : httpd-tools-2.4.37-21.module+el8.2.0+5008+cca404a3 7/10
Verifying : mod_http2-1.11.3-3.module+el8.2.0+7758+84b4ca3e.1. 8/10
Verifying : mailcap-2.1.48-3.el8.noarch 9/10
Verifying : redhat-logos-httpd-81.1-1.el8.noarch 10/10Installed:
apr-1.6.3-9.el8.x86_64
apr-util-1.6.1-6.el8.x86_64
apr-util-bdb-1.6.1-6.el8.x86_64
apr-util-openssl-1.6.1-6.el8.x86_64
httpd-2.4.37-21.module+el8.2.0+5008+cca404a3.x86_64
httpd-filesystem-2.4.37-21.module+el8.2.0+5008+cca404a3.noarch
httpd-tools-2.4.37-21.module+el8.2.0+5008+cca404a3.x86_64
mailcap-2.1.48-3.el8.noarch
mod_http2-1.11.3-3.module+el8.2.0+7758+84b4ca3e.1.x86_64
redhat-logos-httpd-81.1-1.el8.noarchComplete!
Step 3) Finally, we need to run the following command to start the webserver
systemctl start httpdNow we can test it using the Private or Public IP of the system where we installed the httpd server.
Creating, Formatting & Mounting the Partition to httpd Web Root Directory
To perform this operation, we need to follow the below steps
Step 1) First, we need to run the following command to verify which disk is going to be used for the operation.
fdisk -lThis will show the following output
Disk /dev/xvda: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xd00a186fDevice Boot Start End Sectors Size Id Type
/dev/xvda1 2048 4095 2048 1M 83 Linux
/dev/xvda2 * 4096 20971486 20967391 10G 83 Linux
Disk /dev/xvdf: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
In this output, there is a disk /dev/xvdf which is going to be used to create a partition.
Step 2) To create a partition, we need to run the following command
fdisk /dev/xvdfThe output will be
Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x77521bed.Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-2097151, default 2048): 2048
Last sector, +sectors or +size{K,M,G,T,P} (2048-2097151, default 2097151): 2097151Created a new partition 1 of type 'Linux' and of size 1023 MiB.Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
In this output, there is two major commands are used: n and w
n command is used to create a new partition by inputting p and Partition number, First & Last sector as default.
w command is used to write or save the operations
Step 3) After that, to verify whether the partition is created or not, we need to run the following command
lsblkThe output will be
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 10G 0 disk
├─xvda1 202:1 0 1M 0 part
└─xvda2 202:2 0 10G 0 part /
xvdf 202:80 0 1G 0 disk
└─xvdf1 202:81 0 1023M 0 partIn this output, the created partition is labeled as xvdf1 of xvdf disk
Step 4) Now, to format the partition, we need to run the following command
mkfs.ext4 /dev/xvdf1The output will be
mke2fs 1.45.4 (23-Sep-2019)
Creating filesystem with 261888 4k blocks and 65536 inodes
Filesystem UUID: a5f49d11-9a98-45cf-bb89-f3ffaf9da65e
Superblock backups stored on blocks:
32768, 98304, 163840, 229376Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
Step 5) Finally, we need to run the following command to mount the partition to the /var/www/html directory.
mount /dev/xvdf1 /var/www/html/And again, we need to run the lsblk command to verify
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 10G 0 disk
├─xvda1 202:1 0 1M 0 part
└─xvda2 202:2 0 10G 0 part /
xvdf 202:80 0 1G 0 disk
└─xvdf1 202:81 0 1023M 0 part /var/www/htmlNow, the conclusion of this operation is all the web pages or codes will be saved in case of root disk failure, because the files will be saved in EBS Volume instead of Root Volume.
Creating S3 Bucket and uploading a file using AWS CLI2
Amazon S3 or Amazon Simple Storage Service is a service that provides object storage through a web service interface.
To Create S3 Bucket and upload a file using AWS CLI2, we need to follow the below steps
Before doing this operation, make sure you have already installed and logged into the AWS CLI 2. To know more about AWS CLI 2 Configuration, check out the post here.
Step 1) In Terminal or Command Prompt, we need to run the following command to create S3 Bucket
aws s3api create-bucket --bucket shobhitnov2020 --region us-east-2 --create-bucket-configuration LocationConstraint=us-east-2The output will be
{
"Location": "http://shobhitnov2020.s3.amazonaws.com/"
}In this command, shobhitnov2020 is the bucket name and it should be unique because every bucket has its own unique URL. “us-east-2” is the region name. (in your case you can use any region and unique bucket name).
Step 2) After successful creation of bucket, Now upload the file using the following command
aws s3 sync "C:\Users\Shobhit Sharma\Desktop\Image" s3://shobhitnov2020In this command, I’ve used the Image directory located in my Desktop folder and selected the shobhitnov2020 S3 Bucket. Inside the Image directory, I’ve placed a single file called “Tech_Dec_2.png” inside the folder.
The output will be
upload: Desktop\Image\Tech_Dec_2.png to s3://shobhitnov2020/Tech_Dec_2.pngNow, the file is successfully uploaded, we can access it using the following bucket Public URL.
http://shobhitnov2020.s3.amazonaws.com/Tech_Dec_2.png
Additional Step) To make an image file public, go to S3 Dashboard and select the created bucket and then in the Overview Section, click on the “Make Public” button.
Creating CloudFront Distribution of S3 Bucket file
Amazon CloudFront is a content delivery network offered by Amazon Web Services. Content delivery networks provide a globally-distributed network of proxy servers that cache content, such as web videos or other bulky media, more locally to consumers, thus improving access speed for downloading the content. (Wikipedia)
To create CloudFront Distribution of S3 Bucket file, we need to follow the below steps
Step 1) To create a distribution, we need to run the following command
aws cloudfront create-distribution --origin-domain-name shobhitnov2020.s3.amazonaws.com --default-root-object Tech_Dec_2.pngIn this command, shobhitnov2020.s3.amazonaws.com is an S3 Bucket file name and Tech_Dec_2.png is a static image file name.
The output will be
{
"Location": "https://cloudfront.amazonaws.com/2019-03-26/distribution/EQ2B61SDUUK2W",
"ETag": "E1MOWMZQ6DDVL0",
"Distribution": {
"Id": "EQ2B61SDUUK2W",
"ARN": "arn:aws:cloudfront::934204869270:distribution/EQ2B61SDUUK2W",
"Status": "InProgress",
"LastModifiedTime": "2020-11-02T12:01:17.154000+00:00",
"InProgressInvalidationBatches": 0,
"DomainName": "d34szijkpzlcjk.cloudfront.net",
"ActiveTrustedSigners": {
"Enabled": false,
"Quantity": 0
},
"DistributionConfig": {
"CallerReference": "cli-1604318475-444914",
"Aliases": {
"Quantity": 0
},
"DefaultRootObject": "Tech_Dec_2.png",
"Origins": {
"Quantity": 1,
"Items": [
{
"Id": "shobhitnov2020.s3.amazonaws.com-1604318475-140513",
"DomainName": "shobhitnov2020.s3.amazonaws.com",
"OriginPath": "",
"CustomHeaders": {
"Quantity": 0
},
"S3OriginConfig": {
"OriginAccessIdentity": ""
}
}
]
},
"OriginGroups": {
"Quantity": 0
},
"DefaultCacheBehavior": {
"TargetOriginId": "shobhitnov2020.s3.amazonaws.com-1604318475-140513",
"ForwardedValues": {
"QueryString": false,
"Cookies": {
"Forward": "none"
},
"Headers": {
"Quantity": 0
},
"QueryStringCacheKeys": {
"Quantity": 0
}
},
"TrustedSigners": {
"Enabled": false,
"Quantity": 0
},
"ViewerProtocolPolicy": "allow-all",
"MinTTL": 0,
"AllowedMethods": {
"Quantity": 2,
"Items": [
"HEAD",
"GET"
],
"CachedMethods": {
"Quantity": 2,
"Items": [
"HEAD",
"GET"
]
}
},
"SmoothStreaming": false,
"DefaultTTL": 86400,
"MaxTTL": 31536000,
"Compress": false,
"LambdaFunctionAssociations": {
"Quantity": 0
},
"FieldLevelEncryptionId": ""
},
"CacheBehaviors": {
"Quantity": 0
},
"CustomErrorResponses": {
"Quantity": 0
},
"Comment": "",
"Logging": {
"Enabled": false,
"IncludeCookies": false,
"Bucket": "",
"Prefix": ""
},
"PriceClass": "PriceClass_All",
"Enabled": true,
"ViewerCertificate": {
"CloudFrontDefaultCertificate": true,
"MinimumProtocolVersion": "TLSv1",
"CertificateSource": "cloudfront"
},
"Restrictions": {
"GeoRestriction": {
"RestrictionType": "none",
"Quantity": 0
}
},
"WebACLId": "",
"HttpVersion": "http2",
"IsIPV6Enabled": true
}
}
}Now, we got the CloudFront CDN Cache URL for fast access of the file from the nearest Edge Location.
“DomainName”: “d34szijkpzlcjk.cloudfront.net”
Testing the integration of High Availability Architecture
To perform this operation, we need to follow the below steps
Step 1) In RHEL 8, To create a static HTML page, we need to run the following command
vi /var/www/html/shobhit.htmlThis vi command is used to create a file called “shobhit.html” in the root web directory “/var/www/html”.
This will open the vi editor and according to my requirement, I am creating the following static web page
<html>
<head><title>Shobhit Sharma's Page</title></head>
<body>
<h1>Welcome to the page of Shobhit Sharma</h1>
<br>
<h2>EightTwelve [0812]</h2>
<br>
<img height=50% src ="d34szijkpzlcjk.cloudfront.net">
</body>
</html>and Save it.
Step 2) Now, Open the webpage using a Public or Private URL (private URL only if you are using GUI of Linux).
The output will be
Conclusion
After the implementation of High Availability Architecture with Integration of EC2, S3 & CloudFront, we can achieve Security, Flexibility, and High Data Transfer Speed and Content Distribution to the nearest Edge Locations according to user’s region.
…
This article is written, edited, and published by Shobhit Sharma.
