What is automated backup in AWS?

Backing up your data regularly is critical for any company with business continuity in mind, whether the company is using an on-premises data center or a public cloud like AWS. Backups should be implemented properly and tested often, so that if a disaster occurs, the business doesn’t suffer.

Taking a backup of your infrastructure resources frequently is very important in order to be able to recover from a disaster. It’s important to schedule AWS backups on a timely basis, such as taking backup weekly or monthly on different availability zones. It’s one of the best practices that is followed by DevOps teams all over the world.

So, in this article, we’ll walk you through how to backup EC2 instances using cloud-native technologies such as EBS snapshots, AWS Backup, AWS Data Lifecycle Manager, Amazon EBS snapshots using CloudWatch event and AMIs— to achieve both cost-saving and time-saving benefits while maximizing your data availability.

This step-by-step article presents the various advantages and disadvantages worth considering when looking into automating your EC2 instance backups on AWS.

How to Backup EC2 Instances

When doing an EC2 instance backup, you are essentially backing up the Elastic Block Store (EBS) volume (which contains all the data on it) by creating a snapshot of it. You can also create a new Amazon Machine Image (AMI) from your existing instance or a snapshot, if needed.

This method, which was more commonly used in the past, preserves the configuration and deploys new copies of it. Both the EBS snapshot and the AMI are automatically stored on Amazon S3 which is known for being highly durable and reliable. (Learn more with this post: 3 Ways To Backup And Recover EC2 Windows Instances) Most of the recently introduced tools, including the ones discussed below, focus on EBS volumes alone.

4 Different Ways to Take EC2 Instance Backups

1) Automate Backups Using AWS Data Lifecycle Manager

In July of 2018, Amazon released Data Lifecycle Manager (DLM), a tool for automating EBS volumes. DLM allows you to use tag-based lifecycle policies to define various backup schedules.

To get started, open LifeCycle Manager under the EC2 instance service and start defining the policy.

Policy works by looking at the desired tags, so make sure your resources are tagged properly before starting. You can also pick the resource type. This is helpful when you only want to look for tagged EBS volumes, rather than finding all the tagged EC2 instances.

Further down, you define the backup schedule. Simply choose the starting time and how often the policy will run (you can choose between 2h and 24h). You will also need to pick the number of snapshots to be retained.

You can have additional tags added to your snapshots, either by checking the option to copy the tags from the original volume or by adding new ones yourself.

You can also exclude the snapshot of the root volume. This is a useful option if you only need additional data volumes to be backed up. For the role, you can keep the default.

It is possible to create the policy without immediately enabling it. If you do this, simply modify the policy later.

After you are done, you can see your newly created policy, as well its status.

If you want more details, you can look at the policy summary. It will tell you exactly how you have defined the backup schedule, and it will show the retention of the backups.

When the defined schedule triggers backups, you will see them under Snapshots.

2) Backup Automation Using AWS Backup

AWS Backup is a fully managed service that is used to automate backups on AWS (though it can be implemented on-premises as well, by using AWS Storage Gateway). Compared to Data Lifecycle Manager, it is a much more powerful tool, and it can serve as a centralized location for configuring and monitoring backups.

AWS Backup can be used not only for an EBS volume, but also for RDS databases, DynamoDB tables, Storage Gateway volumes, and even EFS file systems.

To start, open the AWS Backup service.

Click on Backup plans (or Manage Backup plans), and then Create Backup plan.

There, you have a couple of options.

You can start by using an existing plan and choosing an option such as “Daily 35-day retention” or “Daily Monthly 1yr Retention.” In this example, we are going to start from scratch, so pick Build a new plan.

After naming your plan, you will configure the backup rule—or, more specifically, a schedule that will be followed for creating backups. Make sure you set the desired backup window (if using a custom one, UTC is always used and can’t be changed) and the frequency of execution.

You can also create the life cycle settings and either transition the backups to Amazon Glacier cold storage or expire them completely.

Below that section is the one in which you pick the Backup vault. You can either use the default one or select a custom vault, if you’re looking for logical separation. As with the Data Lifecycle Manager, you can add tags to your backups if needed.

After you’re done and your plan is created, it is time to assign resources to it.

Resources can be assigned by using tags or by adding them based on the Resource ID.

With the plan created and the resources assigned, your AWS Backup is ready.

You can go back to the AWS Backup dashboard and actually see the backup (and also restore) jobs. The screenshot below shows that there is a backup job in progress.

After the backup has been completed, the dashboard will be updated accordingly, as shown below.

With AWS Backup, you can easily initiate the restore of your backups. This can be very convenient, especially in a disaster recovery situation when multiple volumes may need to be restored quickly.

3) Schedule Automated Amazon EBS Snapshots Using CloudWatch Events

The last backup method we will examine is EBS Snapshot creation using CloudWatch events. To implement this process, you will use a CloudWatch rule.

Click on Create rule, and pick Schedule under Event Source.

This will give you two options: you can either have this rule triggered using a fixed rate (every X minutes, hours, or days), or you can use a cron expression, which gives you much more flexibility. For example, you can have an event that triggers every Wednesday, Thursday, and Friday at 6pm. You will also see a preview of your next ten trigger dates. Do note that all times are presented in GMT.

After configuring the desired schedule, click on Add Target and chose “EC2 CreateSnapshot API call” from the list. Then, copy the volume ID of the volume you want to backup. You can leave the option to create a new role, unless you have one already.

Click on Configure details, name your rule, and give it description. After that, you are ready to go.

4) N2WS Backup & Recovery Tool

N2WS Backup & Recovery is a tool that was designed specifically for Amazon’s public cloud, and its purpose is to help you with backup and restore of your AWS resources. Using it, you can automate backups for one or more AWS accounts from a single pane of glass, simplifying the backup process and making it a great tool when dealing with disaster recovery, compliance requirements and storage savings.

N2WS Backup & Recovery is available as a service model that allows users to manage multiple AWS accounts and configure policies and schedules to take automated snapshot backups. It also has a Windows agent to consistently back up Windows applications without the need for maintenance windows. Furthermore, in a dynamic cloud environment, you need to be able to keep consistent backup policies across all your instances at any point in time.

To be most effective, your solution needs to be dynamic and automated when a server is terminated and a new instance needs to be launched.

Using EC2 instance tags, N2WS can automatically assign each one of these new instances the appropriate backup policy based on their purpose and your initial configuration. For more info, check out our previous article about tag-based continuous AWS cloud backup.

N2WS Backup & Recovery has been steadily improved since its global release, and the latest updates introduced many enhanced backup features to let you protect even more AWS services like EFS backups and have improved file level recovery and reporting. We’ve also worked hard to address the growing problem of cloud waste and unnecessary AWS spend.

N2WS’ latest versions have several features specifically designed to save on your monthly AWS bill such as Resource Control, an instance scheduler to start/stop instances on-demand for better control of your resources particularly when they are idle. Also, N2WS-enabled Amazon S3 repository lets customers archive data with longer term storage needs resulting in reducing their storage costs by up to 60%.

Automating AWS EC2 Backup is a great way to save time and ensure critical data is protected

AWS EC2 backup automation is a great way to save time on recurring tasks. It also helps you to confirm that desired tasks have indeed been done. Since having regular backups is an absolute must in any business environment, this critical process should be not hinge upon manual execution.

Depending on your cloud environment, one of the four different methods presented above for automating your AWS EC2 instance backups could be a better fit than the others. N2WS Backup & Recovery is worth exploring, since it is a powerful and useful tool, as it supports not only AWS EC2 instance backup, but a range of other AWS services such as RDS, Redshift, DynamoDB, Aurora, and EFS all under one single pane of glass.

You can manage your entire AWS backup and recovery operations from a dedicated N2WS instance within your secure AWS environment as you scale. You can check it out by spinning up a 30 day free trial here (no credit card needed). Bonus: your free trial automatically turns into our forever Free Edition.

As your organization grows, your data grows. And that growing data is generated from a growing number of endpoints (like desktops, laptops, servers, virtual machines, and other devices).

That’s why automating your backup solution isn’t just time-saving, it’s very cost effective, too. But, trying to leverage non-cloud-native solutions to backup AWS resources may not be effective in achieving this desired resource-saving benefit.

That’s because traditional backup software and methods are very centralized by nature —creating innate disadvantages, such as single points of failure, high-cost software licensing, and requiring dedicated hardware resources. So, in this article, we’ll walk through how to backup your EC2 instances using cloud-native technologies —EBS snapshots and AMIs— to achieve both cost-saving and time-saving benefits while maximizing your data availability.

2 Different Ways to Take EC2 Instance Backups

There are two ways to implement backups of your EC2 instances on AWS:

• If your instance is EBS-backed, you can create the snapshots of the EBS volume.
• Or, you can create an AMI of your instances as a backup solution.

Both the EBS snapshot and the AMI are automatically stored on Amazon S3 which is known for being highly durable and reliable. (Learn more with this post: 3 Ways To Backup And Recover EC2 Windows Instances)

Option 1: Manual EC2 Instance Backup by an EBS Snapshot

Before showing how to automate your instance backup, let’s first look at the different manual steps we should perform. Follow the steps below to implement a backup for an EC2 EBS volume:

• Open AWS Console.
• Click the “Instances” section in AWS console under the EC2 dashboard.

• • Select the instance for which you want to create a backup.
  • Under the description tab for that instance, you can see the details of that instance which will also show you the block devices. Clicking on a block device will show the volume ID

• • Click on EBS ID (volume ID), which will take you to the volume section under the EC2 dashboard. To create the snapshot click on the action button and select “Create Snapshot” option.

• • Enter a name and description for the AWS snapshot in the snapshot dialog as below. After creating a snapshot you can see your AWS snapshot in the snapshot section under the EC2 dashboard.

To restore the volume from an AWS snapshot:

• Search for the snapshot under the snapshot section using its description.
• Right-click and select the “Create Volume” option.
• Fill in the required details in the Create Volume dialog box and click the “Create” option.
• A volume with the same snapshot will be created and then you can attach the new volume to an EC2 Instance for further use.

Option 2: Manual Backup Using AMI

This solution can be used if your instance is not EBS backed up. To create the EC2 Instance backup, you can create the AMI of an Instance. Furthermore, by using the same AMI you can launch an instance in the same state.

• Search for the Instance for which you want to create the AMI. Select the “Create Image” option under Image after clicking on the action button.

• Follow the steps below once you have selected the “Create Image” option:
  • Enter the AMI details in the “Create Image” dialog box as below.
  • Enter the AMI Name and Description.
  • If you want your instance to stay in a running state without restarting while creating the AMI, then check the “No Reboot” option.
  • You can also customize the size and other options like “delete on termination”. The “delete on termination” option will delete the volume if the instance is terminated, otherwise, it will still be in an available state if the EC2 instance is terminated.

The Better Option: Automating the EC2 Backup

To automate the EC2 Backup, you will need to write a script to automate the above steps by using AWS’ API. Below is the step by step process which should be followed in the script:

• Get the list of instances.
• Connect to AWS through API to list the Amazon EBS volumes that are attached locally to the instance.
• List the snapshots of each volume.
• Assign a retention period to the snapshot.
• Create an AWS snapshot of each volume.
• Delete the AWS snapshot if it is older than the retention period.

By using AWS Command Line Interface (AWS CLI) you can write a shell script which will be used for automating the EBS volume backup. It’s recommended to install the AWS CLI if it has not already been installed. You can refer to this resource for details:AWS CLI Installation.

Commands to Install AWS CLI

curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
unzip awscli-bundle.zip
./awscli-bundle/install -b ~/bin/aws

After installing AWS CLI, configure it using the aws configure command

aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: ENTER

Creating Shell Script

Use the script below to copy code to snapshot.sh and set it as cron for automatic timely backup. You can find an explanation of script in comments.

#!/bin/bash
# Volume list file will have volume-id:Volume-name format
VOLUMES_LIST = /var/log/volumes-list
SNAPSHOT_INFO = /var/log/snapshot_info
DATE = `date +%Y-%m-%d`
REGION = "eu-west-1"
# Snapshots Retention Period for each volume snapshot
RETENTION=6
SNAP_CREATION = /var/log/snap_creation
SNAP_DELETION = /var/log/snap_deletion
EMAIL_LIST = [email protected]
echo "List of Snapshots Creation Status" > $SNAP_CREATION
echo "List of Snapshots Deletion Status" > $SNAP_DELETION
# Check whether the volumes list file is available or not?
if [ -f $VOLUMES_LIST ]; then
# Creating Snapshot for each volume using for loop
for VOL_INFO in `cat $VOLUMES_LIST`
do
# Getting the Volume ID and Volume Name into the Separate Variables.
VOL_ID = `echo $VOL_INFO | awk -F":" '{print $1}'`
VOL_NAME = `echo $VOL_INFO | awk -F":" '{print $2}'`
# Creating the Snapshot of the Volumes with Proper Description.
DESCRIPTION = "${VOL_NAME}_${DATE}"
/usr/local/bin/aws ec2 create-snapshot --volume-id $VOL_ID --description "$DESCRIPTION" --region $REGION &>> $SNAP_CREATION
done
else
echo "Volumes list file is not available : $VOLUMES_LIST Exiting." | mail -s "Snapshots Creation Status" $EMAIL_LIST
exit 1
fi
echo >> $SNAP_CREATION
echo >> $SNAP_CREATION
# Deleting the Snapshots which are 10 days old.
for VOL_INFO in `cat $VOLUMES_LIST`
do
# Getting the Volume ID and Volume Name into the Separate Variables.
VOL_ID = `echo $VOL_INFO | awk -F":" '{print $1}'`
VOL_NAME = `echo $VOL_INFO | awk -F":" '{print $2}'`
# Getting the Snapshot details of each volume.
/usr/local/bin/aws ec2 describe-snapshots --query Snapshots[*].[SnapshotId,VolumeId,Description,StartTime] --output text --filters "Name=status,Values=completed" "Name=volume-id,Values=$VOL_ID" | grep -v "CreateImage" > $SNAPSHOT_INFO
# Snapshots Retention Period Checking and if it crosses delete them.
while read SNAP_INFO
do
SNAP_ID=`echo $SNAP_INFO | awk '{print $1}'`
echo $SNAP_ID
SNAP_DATE=`echo $SNAP_INFO | awk '{print $4}' | awk -F"T" '{print $1}'`
echo $SNAP_DATE
# Getting the no.of days difference between a snapshot and present day.
RETENTION_DIFF = `echo $(($(($(date -d "$DATE" "+%s") - $(date -d "$SNAP_DATE" "+%s"))) / 86400))`
echo $RETENTION_DIFF
# Deleting the Snapshots which are older than the Retention Period
if [ $RETENTION -lt $RETENTION_DIFF ];
then
/usr/local/bin/aws ec2 delete-snapshot --snapshot-id $SNAP_ID --region $REGION --output text> /tmp/snap_del
echo DELETING $SNAP_INFO >> $SNAP_DELETION
fi
done < $SNAPSHOT_INFO
done
echo >> $SNAP_DELETION
# Merging the Snap Creation and Deletion Data
cat $SNAP_CREATION $SNAP_DELETION > /var/log/mail_report
# Sending the mail Update
cat /var/log/mail_report | mail -s "Volume Snapshots Status" $EMAIL_LIST

Follow the steps below for creating and running shell script:

• Create a script by the name of snapshot.sh using the command below.
• Set it as a cron in crontab.

# Edit Cron File
crontab -e

Taking a backup of your infrastructure resources frequently is very important in order to be able to recover from a disaster. It’s important to schedule AWS backups on a timely basis, such as taking backup weekly or monthly on different availability zones. It’s one of the best practices that is followed by DevOps teams all over the world.

Automating Instance Backup Using N2WS Backup & Recovery

While using in-house scripts can provide for a basic backup solution, it doesn’t make business sense for organizations to invest in a fully-featured in-house backup solution rather than focusing on their business-critical tasks. That’s why our founder Uri created N2WS Backup & Recovery. It’s an enterprise-class backup-recovery and disaster recovery solution designed for AWS EC2 covering all the essential backup and recovery features to ensure the robustness of your data as well as simplifying processes and saving precious DevOps time.

N2WS Backup & Recovery is available as a service model that allows users to manage multiple AWS accounts and configure policies and schedules to take automated snapshot backups. It also has a Windows agent to consistently back up Windows applications without the need for maintenance windows. Furthermore, in a dynamic cloud environment, you need to be able to keep consistent backup policies across all your instances at any point in time.

To be most effective, your solution needs to be dynamic and automated when a server is terminated and a new instance needs to be launched. Using EC2 tags, N2WS can automatically assign each one of these new instances the appropriate backup policy based on their purpose and your initial configuration. For more info, check out our previous article about tag-based continuous AWS cloud backup. And try N2WS Backup & Recovery for FREE!

Keep Reading:

• How to Copy EBS Snapshots to Amazon S3 and Save on Storage Costs
• 7 Ways to Save BIG on Your AWS Storage Bill
• 10 Tips for a Solid AWS Disaster Recovery Plan

PrevPreviousEphemeral Storage Mirror on an EBS Volume

NextHow to Connect an AWS EBS Volume to Another EC2 InstanceNext

More To Explore

Blog

The Best Storage Guide to AWS re:Invent 2022

Every year our global team meets up in Las Vegas (let’s not count that little 2020 blip) to meet the most amazing cloud community on

Microsoft Azure Cloud Services

NEW RELEASE: N2WS Backup & Recovery with new storage class Glacier Instant Retrieval – Overview & Demo Webinar

NEW N2WS v4.1.1 is generally available on the AWS Marketplace, now with support for new storage class S3 Glacier Instant Retrieval! N2WS v4.1.1 has a

What are automated backups?

What is automated backup? Automatic backup is primarily enabled through backup software that automates the entire backup process. It means organizations can back up files, folders and systems without any human intervention. Essentially, automated backups simplify backup procedures to drive speedy recoveries.

What is automated backup in RDS?

Automated Backups Amazon RDS automatically creates a storage volume snapshot of your DB instance, backing up the entire DB instance and not just individual databases. This backup occurs during a daily user-configurable 30 minute period known as the backup window.

What is the main difference between automated and manual RDS backups?

Automated backups are deleted when the DB instance is deleted. Only manually created DB Snapshots are retained after the DB Instance is deleted.

Why is automated backup important?

Automated backups are more reliable and less error-prone than manual backups. With automated backups, you can ensure consistent data protection, reduce recovery windows, and improve business continuity.