Setup DataCenter to backup to Amazon S3 using AWS Storage Gateway

Do you ask yourself how do you get your business critical data offsite? There are just so many options: tape, portable disk (and discs), thousands of ‘cloud’ storages, and of course file sharing services (which are not backup, just FYI :) ).

Unfortunately, we can’t answer the question for you, because it depends on your RPOs, RTOs, and in general on the backup and restore requirements you have. But at least we can help you with giving you some information on one of the options out there.

Backup data to Amazon S3 using Amazon’s AWS Storage Gateway and NovaStor's DataCenter

Amazon is one of the biggest storage provider in the world. With S3 and Glacier everybody can easily book offsite storage for backup and archiving purposes. Today we want to explain how you can setup Amazon’s AWS Storage Gateway and use S3 as an offsite storage for NovaStor’s network backup software DataCenter. This blog article will give a first overview, the detailed setup can be found in our new whitepaper here.

What is Amazon S3?

‘S3’ stands for ‘Simple Storage Service’ and that’s exactly what Amazon S3 is, an offsite storage for all kind of data, accessed through a web service interface. The storage includes some amazing features that helps to meet company compliances:

• Cross-Region Replication
• Event Notifications
• Lifecycle Management
• Encryption
• Security and Access Management
• Flexible storage options

Because of the high-end technology used in their data centers, Amazon’s service is able to deliver HA and redundancy at its best. Transferring single files with a size of up to 5GB at once is not a problem, individual objects can have a size of up to 5TB. In case your complete data is too big to transfer it over a regular internet connection, Amazon’s offers include e.g. a dedicated connection between your and their data center. I don’t want to talk in depth about the functions, but here are some interesting facts about Amazon S3:

• The consistency model used for the US region is ‘eventual consistency’.
• Amazon S3 supports user authentication such as bucket policies or ACLs
• Up- and Download of data via SSL endpoints using the https protocol (further security mechanism available, e.g. SSE, SSE-C)
• Encrypt your data inside the buckets using SSE-S3, SSE-C, SSE-KMS (HIPAA/ HITECH, and FedRAMP compliant) encryption
• SSE encryption keys are unique for every object. This key is also encrypted with the master key that changes at least monthly. All keys are stored in separated storages.
• Using a VPC endpoint, allows to connect only to the S3 storage
• The checksum used to detect data corruption is Content-MD5 and cycling redundancy checks (CRCs)
• Amazon doesn’t access the data stored except for billing purposes or if required by law. Details regarding Amazon S3 SLA’s can be found here: http://aws.amazon.com/s3-sla/ or http://aws.amazon.com/agreement

You can find more information about features and functions here: http://aws.amazon.com/s3/
Just remember, every extra feature probably comes with extra costs!

In order to access the Amazon S3 storage, it has to be connected with the Amazon Storage Gateway. This service ensures that your data is transferred fully SSL encrypted. But the main function is the simple integration to every device. Therefore the Amazon Storage Gateway implements a standard iSCSI interface, either as a block disk device or VTL.

What is the AWS Storage Gateway?

The AWS Storage Gateway basically is a virtual machine. In order to connect it to your Amazon services it has to run on a virtual machine. A VMware ESXi v4.1 and newer or a Hyper-V 2008 and newer Hypervisor are sufficient. The service also needs

• 4 virtual processors assigned to the VM
• 7.5GB RAM
• 75GB disk space for the installation

Security and encryption wise the AWS Storage Gateway encrypts all data in-transit to and from AWS via SSL. All volume and snapshot data stored in AWS using Gateway-Stored Volumes/Gateway-Cached Volumes and all virtual tape data stored in AWS using Gateway-VTL is encrypted-at-rest using Advanced Encryption Standard (AES) 256, a secure symmetric-key encryption standard using 256-bit encryption keys. So you do not need to add other encryption options in order to make sure that the data that you put on the AWS Storage Gateway is encrypted.

There are 3 different ways to setup the AWS Storage Gateway, Gateway-Cached Volumes, Gateway-Stored Volumes, and Gateway-Virtual Tape Library. Below is a description of what each configuration does:

• Gateway-Cached Volumes: You can durably and inexpensively store your primary data in Amazon S3, and retain your frequently accessed data locally. Gateway-Cached Volumes provide substantial cost savings on primary storage, minimize the need to scale your storage on-premises, and provide low-latency access to your frequently accessed data. In addition to storing your primary data in Amazon S3 using Gateway-Cached Volumes, you can also take point-in-time snapshots of your Gateway-Cached volume data in Amazon S3, enabling you to make space-efficient versioned copies of your volumes for data protection and various data reuse needs.
  
  
• Gateway-Stored Volumes: In the event you need low-latency access to your entire data set, you can configure your gateway to store your primary data locally, and asynchronously back up point-in-time snapshots of this data to Amazon S3. Gateway-Stored volumes provide durable and inexpensive off-site backups that you can recover locally or from Amazon EC2 if, for example, you need replacement capacity for disaster recovery.
  
  
• Gateway-Virtual Tape Library (Gateway-VTL): With Gateway-VTL you can have a limitless collection of virtual tapes. Each virtual tape can be stored in a Virtual Tape Library backed by Amazon S3 or a Virtual Tape Shelf backed by Amazon Glacier. The Virtual Tape Library exposes an industry standard iSCSI interface which provides your backup application with on-line access to the virtual tapes. When you no longer require immediate or frequent access to data contained on a virtual tape, you can use your backup application to move it from its Virtual Tape Library to your Virtual Tape Shelf in order to further reduce your storage costs.

How to setup Amazon Storage Gateway

Continue to read in our Whitepaper ‘How to setup NovaStor DataCenter to backup data to Amazon S3 using Amazon’s AWS Storage Gateway’ that you can find here.

If you haven’t already, sign up for our email to receive information about the technology behind NovaStor DataCenter, NovaStor’s technology partners, Webinar invitations, and general network backup and restore knowledge.

More information about NovaStor DataCenter here.