S3 - Security & Encryption - In detail

Securing the buckets:

By default, the newly created buckets are private.
We can setup access control to the buckets using;
- Bucket Policies;
- ACL (Access Control Lists) ;

We can setup access log on the S3 buckets.

Encryption : 

Two Types;

- In Transit (SSO/TLS, just using HTTPS)
- At Rest (Server side and client side)
  - Server Side; S3 Managed Key - SSE S3
  - AWS Key Management Service, Managed Keys - SSE-KMS
  - Server Side encryption with customer provided keys - SSE - C
  - Client Side  encryption.

Content Delivery Network CDN - Hands On

As soon as we login to the AWS console, we can see the CloudFront under the Networking & Content Delivery section:

Click on CloudFront to see this screen below:

The first thing is to create a distribution.

Creating a simple web distribution:

From the auto populated sections of the S3 buckets or Elastic Load Balancers, we can choose one of the S3 buckets object. Thats the origin domain name.

Origin Path : Can be a folder with in an S3 bucket, so that its not pointed to the root of the bucket. We can leave the origin path. There can be multiple origins in the same distribution.

We can restrict the bucket access, by clicking on Yes under restrict access. This restrict the public access of the S3 url object and the object cannot be accepted by the public using the S3 url, but only from the CloudFront.

Origin Access Identity: This is another way of restricting buckets.

The purpose of each of the options can be read from the information icon beside the field.

Restrict Viewer Access: Restricts the users from accessing the files by securing it. Only the authorized users can access the files.

Once the required settings are made which are quite a few, we can now create a distribution and access the same object that we have accessed using url of the S3 bucket we can access using the CloudFront Url, which will load the object from the cache of the edge location from the second user accessing it.

The Unique domain of the distribution can be configured to be able to remembered by a human.

Upon successfully creating the Distribution we can click the Id link to open the distribution and view other options:

Under the Restrictions --> Geo Restriction we can enable this and select either white or black list
and shuffle the countries under it.

White list : want the cloudFront to allow to distribute the content.
Black list :  dont want the cloudFront to allow to distribute the content.

We can create custom error pages when users get any errors they land on this page.

We can delete the distribution, we can select the distribution and click on disable it, it will take some 15 mins, after that we can delete the destribution.

CloudFront - In Detail

What is CDN ?

CDN is Content Delivery Network. It is a system of distributed services, that delivers webpages and other web content to a user base based on the geographic locations of the users, the origin of the web page and and a content delivery server.

How the users would access without CDN :

Edge Location:

This is the location where the content is cached. This is separate to an AWS Region/AZ.
We can look at the different locations by going to the

Origin :

This is the origin of all the files that CDN will distribute. This can be an S3 bucket, an EC2 Instance, an Elastic Load Bucket or Route53.

Distribution :

This is a name given to the CDN which consists of a collection of Edge Locations.


CDN :
When the first user makes a request, the request goes to the edge location.
Edge location checks if the file is cached in that location or not.
If it doesnt, it pulls from S3, and caches it.
From the second user, the file is already cached at that location and can be much faster from the second user in that region.

 Amazon CloudFront can be used to deliver your entire website, including dynmic, static, streaming and interactive content using a global network of edge locations. Requests for the content are automatically routed to the nearest edge location, so the content is delivered with the best possible performance.

CloudFront is optimized to work with other amazon service like the S3, EC2, Load balancing, Route53.

CloudFront also works with any non AWS origin server, which stores the original, definitive versions of your files.

Two Types of Distributions;
Web distributions: Typically used for websites.
RTMP : used of media streaming(adobe flash)

Edge locations are not for read only.
We can set a TTL for the object to be cached at the edge location.
You will be charged to clear the cached object.

Lifecycle Mangement, IA S3 and Galcier - Hands On

Lifecycle Management:

This basically helps in maintaining the life cycle of the object by writing a rule for the object under management section, by adding a life cycle rule.

Life cycle rule: These rules help in manage the life cycle cost by transitioning from S3, after certain time into IA S3 and then to Glacier archiving the files which are least frequently used there by reducing the cost of the storage drastically.

Under the life cycle transition rules we can configure either on the current version or on the previous versions.

Object Created -- 30days later --> Transitioned to IA -- 30days later --> Transitioned to Glacier

-- 425 days later --> Expires

If the number of days in the glacier is less than 90 days and we want to expire it before completing 90 days in the glacier, we will be required to give an extra authorization saying we want to delete the object even though we are charged for 90 days(which logically doesn't make sense).
Glacier is designed to store an object at least for a minimum of 90 days.

Min 30 days after creation to IA S3, 60 days to Glacier and 61 days to Expire

AWS Global Infrastructure - INTRO

AWS Global Infrastructure:

Regions are physical stuffs that AWS runs on. (Regions, Availability zones and each locations)
Regions are places where AWS resources exist.
They keep constantly increasing.
Each region consists of two or more availability zones.
Availability zones are simply data centers (collection of data centers as well).

Edge Location is a content delivery location. They are CDN end points for CloudFront.

CDN : Content Delivery Network. 

Way to cache media files in the cloud. From the second user the cached filed is fetched from the edge location.

North America has the largest Edge Locations obviously because of the number of users.

Messaging - INTRO

SNS-Simple Notification Services

Simple Notification Services; this is a way of notifying either by email, text message.

SQS-Imp

This is a way of decoupling the applications. Its a queue system.

SES-Simple email service

This is a way of sending and receiving the emails using AWS.

Atrificial Intelligence - INTRO

Alexa -

Alexa is amazons voice service in the cloud, can  be communicated to Alexa using Echo.
In the back end it is talking to Lambda. Lex drives Alexa. Now there is no longer any need for Echo, this can be embedded in all kinds of devices.

Polly-

Polly is the most advanced text to speech service on the planet. Can use 47 voices and available in 24 languages. Text is converted in MP3. It uses synthesis speech mark up language and stores the MP3 file in S3 which stores the files as objects.

Machine Learning -

We give AWS a data set (name, married, age, occupation, etc) we tell it what the outcomes (bought a product or not ) amazon will use machine learning to analyze the data set and predict outcomes for future decisions. This allows to predict data based on the previous performance.

Rekognition -

This service will  recognize an uploaded picture if it has a human being (facial recognition) and also identifies the objects in the picture (mountain bike)

Desktop and App Streaming - INTRO

WorkSpaces-

Workspace is a way of having desktop in the cloud. The actual OS is running on the cloud.

AppStream 2.0 -

It is a way of streaming desktop applications to the users. The previous version is called AppStream 1.0 and it is retired but still sits on the AWS console.

Internet Of Things - INTRO

iOT -

This is a way of having millions of devices keeping track of them using the iOT gateway.
This service is new.

Business Productivity - INTRO

WorkDocs -

Used for storing important work documents in the cloud. In the back end it is using S3 but with all the security added.

Workmail-

This is like the exchange for AWS. A way of sending and receiving the emails.

Mobile Services - INTRO

Mobile Hub -

This lets to add, configure and design features for mobile apps which includes, user auth, data storage, push notifications, back end logic, content delivery and anlytics. Mobile has its own console for mobile apps.

Cognito -

It is used for sign up using gmail credentials. Used in IOS with AWS.
Instagram style app - IOS with AWS back end services. Take a photo stored in S3 write the metadata to DynamoDB, trigger Lambda to generate thumbnails of that photo.

Device Farm -

This service helps improve the android, IOS and fios apps  by quickly and securely testing them on hundreds of real smart phones.This helps test the physical devices using AWS data center, in other words this service is used for load testing.

Mobile Analytics - 

This service helps in analyzing the app usage data in a cost effective manner.

Pinpoint -

This is new service added to the mobile services list. This enables us to engage and understand with the application users. To understand what the users are doing with the app. Lets say we wanted to do a targeted marketing campaign this service helps do that.

Developer Tools - INTRO

CodeCommit -

It is a GitHub, GitHub is place to store code.

CodeBuild -

It is a way to compile the code.

CodeDeploy -

Way of deploying in an automated fashion.

CodePipeline -

Way of keeping versions of code.

Application Services - INTRO

Step Functions:

Announced in 2016. Visualizing whats going on inside the application basically what micro services it is using.

SWF - Simple Workflow Service

This is used in the amazon fulfillment center. A way of coordinating automated tasks and manual tasks.

API Gateway -

Door to create, publish, monitor, maintain and secure APIs at scale. Door for the Apps to access the back end data things like Lambda etc.

AppStream -

Its a way of streaming desktop applications to users.

Elastic Transcoder -

Used for changing the video format to suit all different devices based on resolution, size etc

Management Tools - INTRO

Cloud Watch -

SysOps certification. Used for monitor performance of AWS env, in particular EC2.

Cloud Formation - Most Imp

It is a way of turning physical infrastructure into code. Provision prod env using one script.

Cloud Trial -

Auditing AWS resources.

Opswork -

A way of automating deployments using shift.

Config -

Config manager automatically monitors environment and gives warnings if the config might break.

Trusted Advisor -

Designed by the AWS solutions architecture team. They make series of recommendations; Security, Cost optimization, series of recommendations which is automated by scanning the env.

Security and Identity - INTRO

IAM - Identity Access Management 

Fundamental component and comes up in all the certifications.
Used to sign in to AWS, setup permissions, groups etc.,

Inspector -

Its an agent installed on the virtual machine, and inspects the virtual machines and does security reporting.

Certificate Manager -

This gives free SSL certificates for the domain names.

Directory Service -

A way of using Microsoft AD with AWS.

WAF - Web Application Firewall -

This allows application level protection. Firewall gives network protection, this service gives application level protection. This is a security protocol.

Artifacts -

Place where we can get documentation.

Analytics - INTRO

Athena -

This basically allows SQL queries on S3. Turning Flat files into searchable data bases.

Elastic Map Reduce -

This is used for big data processing.
This is used for processing of large amounts of data (web indexing, log analysis etc ). Framework based on Hadoop, Apache Spark etc.

Cloud Search -

This is similar to Elastic. Search engine for website or application we can use cloud or elastic search.
This is based purely managed service by the Amazon.

Elastic Search -

This is service using open source.

Kinesis -

This is a way of streaming and analyzing real time data. We can capture and store terrabytes of data, things like analyzing market, social media tweets etc.,

Data Pipeline - 

Data Pipeline is a service to move data place to place.  Ex:- From S3 to DynamoDB.

Quick Sight -

This is a business analytics tools, used for creating rich dashboards for the data. Not in demand now.

Migration Services - INTRO

Snowball -

Snowball started as import and export. Send disk/disks and connect those disks and transfer the data to S3/EC2 etc. Snowball was introduced to avoid disks which comes in different shapes and sizes. It is a brief case sized appliance consists of only storage and load it with terra bytes of data and send back to Amazon. Recently introduced Snowball edge which instead of just being a storage application and added compute capacity like a data center.

DMS - Database Migration Services 

DMS allows on premis data base to cloud and into things like Redshift. Dont have to stay with the migrated data base. DMS will take care of the migration with no downtime.

SMS - Server Migration Services 

This does exactly like DMS but targets VMWare migrations. Can work on 50 concurrently.

Databases - INTRO

RDS - Relational Database Service

MySQL, SQL Server, Oracle, etc all come under this section.

DynamoDB - Non Relational Database Service

Heavily stressed in the developer associate course. This is a No SQL database and this can be scalable very vastly.

Redshift - Amazon warehousing solution

Queried only when required like big data stored in the warehouse. Queries are not directly run the production data base as it is going to get slowed down instead we transfer a copy on the redshift and query as and when required.
This is stressed mainly in the Big Data speciality certifications.

Elasticache -

This is a way of caching the data in the cloud. Usually data that is most frequently being used.

Storage - INTRO

Storage consists of four different components:

S3 - Simple Storage Service

Virtual Disk in the cloud to store objects (docs/files/text files/movies etc). Not to install a DB, an application, a computer game. This is an object based storage. For installing a DB, application or a computer game we need a block based storage. DropBox is one of the first start up to use S3.

Glacier -

Glacier is a place where we archive your files from S3. Glacier is used where we dont need instant access. It is extremely low cost. It takes around 3-4 hours to retrieve them.

EFS - Elastic File Service-

EFS is block based storage. This is a new service. EFS can be shared with multiple virtual machines.

Storage Gateway -

This is a way of connecting S3 to the on premise data center or to the head quarters. It is a virtual machine that is installed on prems, and you get a virtual machine image it then communicates with S3.

Compute - INTRO

EC2 - Elastic Cloud Compute:

EC2 in short is the virtual machines in the cloud. Virtual machines that run on AWS.

EC2 Container Service :

This is a highly scalable, highly performing container management service. This basically allows applications to run on a managed cluster of EC2 instances (i.e., virtual machines).
It eliminates the need for the users to install, operate and scale the cluster management infrastructure.

Elastic Beanstalk: 

Used to deploy the code onto AWS.  We will just have to upload the code onto Elastic Beanstalk.
Elastic Beanstalk will then check the code.

Lambda:

AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume - there is no charge when your code is not running. With Lambda, you can run code for virtually any type of application or backend service - all with zero administration.

LightSail: 

With a couple of clicks you can choose a configuration from a menu and launch a virtual machine preconfigured with SSD-based storage, DNS management, and a static IP addres

Networking and Content Delivery - INTRO

VPC :- Virtual Private Cloud

Availability Zone as mentioned in the previous post being physical data center, VPC is virtual data center.
The assets are deployed on the VPC. There are multiple VPC's in a region.
This is the most important topic for the certification exam. (Building a VPC from memory)

Route53:

Route 53 is amazon's DNS service.
DNS service looks up for the public IP address.
The domain names are registered using the Route 53 service of AWS.
The number 53 is the DNS port number.

Cloud Front: 

Cloud Front is part of the content delivery network. Consists of all the edge locations which will cache the assets, giving faster accessibility to the users.

Direct Connect : 

Direct Connect is a way of connecting your office or connecting the physical data centers to AWS, directly using a dedicated telephone line instead of going over the internet into AWS.

AWS Architect Associate Exam - Services that we will need to learn the most

1. AWS Global Infrastructure-Understand the difference between region and availability
2. Networking and Content Delivery
3. Compute
4. Databases
5. Security & Identity - Identity Access Management
6. Management Tools
7. Desktop & App Streaming
8. Messaging

AWS Basics

The First and the foremost requirement for the AWS developer is to get the certification as employers would be looking for employees with certification so that they fall under one of the categories of partner programming:

    Partner                          Associate Certificates                          Professional Certificates

1. Standard                                       2                                                              0
2. Advanced                                     4                                                              2
3. Premier                                        20                                                             8

The different certification exams that are available in the market currently:
1. Developer Associate
2. Solutions Architect Associate
3. Sysops Administrator Associate
4. Security Specialist
5. Big Data Speciality
6. Devops Pro
7. Advanced Networking Speciality
8. Solutions Architect Professional

The first being the easiest and the last one the hardest.

There is a very good reason to try to complete all the eight certifications, since he/she will be the first in the world as of today :)

Get Started:
Go to https://aws.amazon.com/free/ and get yourself a free account.

A Brief history of AWS:

Officially Launched in 2006. 

SQS and EC2 were some of the first services available.

2012 First Re Invent Conference.

2013 Officially launched the certification process.

8 Certifications of 3 are associate level, 2 professional level and 3 new speciality levels recently added.
In Comparison with AWS, Microsoft Azure and Google cloud are a comparable competitors.

Services to be focused:

Out of all the services available we need to know about these services to pass the exam (Solutions Architect Associate Certification):

• Messaging services.
• Desktop and App Streaming (Very High Level specifically around work space).
• Security and Identity Access Management (Very Important).
• Management Tools; only a little bit. 
• Storage, Databases, Networking & Content Delivery, Compute (Most Important)