Of course we cannot always share details about our work with customers, but nevertheless it is nice to show our technical achievements and share some of our implemented solutions.
Setting up an EC2 instance in AWS is easy. But what about backups? How does that even work in the cloud? Or can you even avoid backups because everything in the cloud is stable?
To answer at least the last question: No, not everything is stable in the cloud (as often misleadingly advertised). Instances may crash (due to software bug or a hardware crash) and may never recover. Never heard of that? Read on AWS EC2 instance unreachable after reboot ([Errno 101]) and see for yourself.
So yes - if you are serious about your setups, you do need to back up your instances. In this article we take a closer look at how to create automatic daily snapshots of an EC2 instance (or more precisely: of the EBS volumes of EC2 instances) using AWS' Cloudwatch service.
To be able to access Cloudwatch and create Event Rules, the user needs certain additional IAM privileges. To bundle these together, a group "CloudWatchAdmins" can be created which contain the necessary policies:
Note: Although the policies starting with "CloudWatch" would suggest these are enough to create Event Rules, you're mistaken. The additional policies for ResourceGroups are needed (at least resource_groups:ListGroups is required).
With the relevant IAM permissions granted, open the Cloudwatch service. In the side-navigation locate and click on "Rules" (under Events). Then click on the blug "Create rule" icon (1).
In the Event Source select "Schedule" (2) and set the interval of this rule. Here a fixed rate of 1 Days was selected (daily interval).
As "Target" the "EC2 CreateSnapshot API call" was chosen (3). This triggers the CreateSnapshot function in the AWS API in the background. Last thing to do is to select the target EBS volume (4). If you're not sure which volume ID you need to select, navigate to AWS' EC2 service, click on the desired EC2 instance and click on the block devices. A popup should show the relevant EBS volume ID:
Once the EBS volume ID was chosen, a role for this event needs to be chosen. Usually the default (Cloudwatch creates a new role) should be sufficient.
Finally, click on the blue icon "Configure details" to continue.
Agreed, "details" is kind of overrated here. In this second step one can only define a name for this event rule and an additional description.
Note: The name field does not support whitespaces.
Finally, click on the blue icon "Create rule" to finish.
Without specific timing given in the rule schedule (in the example above, just a daily interval was defined), the event rule should be triggered immediately. This can be verified in AWS' EC2 service. In the navigation click on Snapshots (under Elastic Block Storage):
The pending snapshot task was automatically added by the Cloudwatch event (see volume ID).
24 hours later EC2 - EBS - Snapshots can once again be opened to verify if the daily backup was successfully triggered. And yes, it was:
Once again, check the Volume ID to verify.
It would have been nice if the description contained an information that Cloudwatch event rule XXX-daily-snapshot created that particular snapshot. AWS, if you read this, you know what to do :-).
Updated June 22nd 2020
Yes, there is an easier and more self-explanatory way how to create and schedule automatic snapshots of EBS Volumes! Read our follow up article how to create automatic snapshots (backups) of EC2 storage volumes using Data Lifecycle Manager.