Sunday, April 30, 2017

Introduction to Identity Access Management

Essentially IAM allows you to manage users and their level of access to the AWS console.
IAM is very important for administrating a companies AWS account in real life.

  • Centralised control of AWS account.
  • Shared access to your AWS account.
  • It gives granular permissions. (User John Smith to have access to have read only access to dynamoDB)
  • Identity Federation (including Active Directory, Facebook, Linkedin etc). Login to FB/Linkedin and use AWS resources. 
  • Multifactor authentication.
  • Provide temporary access for users/devices and services where necessary.
  • Allows you to set up your own password rotation policy (how often users have to change the password, also allows to have password policy like small case, upper case, numerical etc.,).
  • Integrates with many different AWS services.
  • Supports PCI DSS Compliance. PCI DSS Compliance is an order process that has to go thorough for online payments when you are handling the payments ourselves, if we are using any third party vendors like PayPal or any other service they take care of the PCI DSS Component.
  • IAM is global it doesn't require any region. So we cannot select any region as such.

Critical Terms :

  • Users - End Users (People using)
  • Groups - A collection of users under one set of permissions (Developers, Human Resources, Finance dept.,).
  • Roles - You create roles and can then assign them to AWS resources.
  • Policies - A document that defines one or more permissions. We create a policy document and assign it to a user/group/role.
Hands On:


  • The first time we login in to AWS console, we are actually logging in with the root account.
  • In order to access IAM, we have to select IAM under the Security, Identity and Compliance section.


  • We can see that the five sections out of which only the first one is complete and the rest are incomplete.
  • One other thing we can observe is that IAM doesnt have any location, it is specified as global.
  • So the users, groups and roles are available all across the world, doesnt matter on a location.
  • We can customize the sign-in link by clicking on customize and giving our own DNS name space, which is unique.
  • The number that comes up on the link is the AWS account number which can also be accessed from My Account.


Step 1: Activate MFA(MultiFactor Authentication) on your root account:
Root account is the email id that we used to login in to the AWS, which gives root access(highest power).

Managing a MFA can be done in two ways using the virtual device or a hardware device.

Google Authenticator app is generally used for authentication by scanning the QR code.

Download the app, scan the QR code and enter the activation key twice and thats it the virtual device is all set up for the root account.

Step 2:  Manage Users:
Click on manage users, give the names of the users.

Access Type:
Programmatic access is used for the developers where they can access AWS programmatically using command line or their code using access key id and secret access key.
Management console access is by using the AWS console.

The next step is to give permissions to the users that we have created either by copy users to a group(inheriting the permissions that the group has) or copy the permissions from an existing group or by attaching existing policies directly.

When you try to create a group, we can attach a policy from the existing policies.

In the final step of adding the users we can see the secret access key and the tokens for the users that can be used to access AWS programmatically.

For the users to login to console they will be needing the user name and the password.
These cannot be used vice versa.

Console - User Name and Password
Programmatic acess - token and access key

We can create new groups attach policies and add users to those groups.

We can even add individual permissions/policies to the users.

Step 5 : Apply an IAM password policy


Creating Roles:

AWS Service Roles; roles for different services. Select one of the services roles and then select the policy (which basically specifies what kind of access to which service)
Role for Cross-Account Access; which is for one service to be able to access another service.
Role for Identity Provider Access; used when external applications like FB, LinkedIn or anyother services have to talk to AWS.

Security Token Service (STS):-

 Grants users limited and temporary access to AWS resources. Users can come from three sources:
  • Federation (Active Directory)-> Uses Security Assertion Markup Language(SAML).
    -> Grants temporary access based off the users Active Directory credentials. Does not need to be a user in IAM.
    -> SSO allows users to log in to AWS console without assigning IAM credentials.
  • Federation with mobile Apps-> Use Facebook/Amazon/Google or other OpenID providers to log in.
  • Cross Account Access
    -> Lets users from on AWS account access resources in another.
Key Terms:
  1. Federation: combining or joining a list of users in one domain (such as IAM) with a list of users in another domain (such as Active Directory, Facecbook etc)
  2. Identity Broker: a service that allows you take an identity from point A and join it (federate it) to point B.
  3. Identity Store:  Services like AD, FB, Goole etc.,
  4. Identities : A user of a service like FB. etc..
 Process of using STS to connect to LDAP:
  1. Develop an Identity Broker to communicate with LDAP and AWS STS.
  2. Identity Broker always authenticates with LDAP first, then with AWS STS.
  3. Application then gets temporary access to AWS resources. 


Active Directory Federation:

->When someone is working from home and need to login to AWS console.
->They browse to a url and access using AD username and password.
->Browser then receives a SAML assertion in the form of an authentication response from ADFS.
->Browser posts the SAML assertion to the AWS sign-in endpoint for SAML. Behind the scenes,      sign-in uses the AssumeRoleWithSAML API to request temporary security credentials and then constructs a sign-in URL for the AWS Management Console.
-> Browser receives the sign-in URL and is redirected to the console.

User Always get authenticated against the AD first and then gets assigned a temporary URL for authentication to sign in to the AWS.

Web Identity Federation: - 
-> Uses call AssumeRoleWithWebIdentity (using FB, Google etc.,)
-> We basically sign in to either FB or google and then get a temporary token after getting authenticated against the same and continue to use AWS resources.








Saturday, April 29, 2017

AWS Certified Developer Important Topics:

-> Messaging
-> Security and Identity
-> Management Tools (Very High level)
-> Storage
-> Databases
-> Networking & Content Delivery
-> Compute
-> AWS Global Infrastructure

AWS Global Infrastructure:
  1. Broken down into three different things : Regions, AZ and Edge Locations.
  2. Regions: It is geographical area. It consists of two or more availability zones.
  3. AZ: It is simply a data center/centers (so it is made sure that they are have enough space between in case of natural calamity). 
  4. Edge Location: It is CDN end point for CloudFront. It is a way to cache media files on the first access. This is more like user dependent and nothing to do with Regions and AZ's.
Networking & Content Delivery:
  1. VPC: Virtual Private Cloud is virtual data center where assets are deployed. (Very Important).
  2. Route53: It is Amazons DNS service. Register domain names using Route53. 53 is the DNS port.
  3. CloudFront: It consists of Edge Locations which will cache files.
  4. DirectConnect: It is used to connect to AWS using dedicated line.
Compute:
  1. EC2: Elastic Cloud Compute, is virtual machines that run on the cloud.
  2. EC2 Container Services: It is a highly scalable and highly performing container management system. It allows to run applications on a managed cluster of EC2 instances. (Not in Developer Associate Exam).
  3. Elastic BeanStalk: Code can be uploaded to elastic beanstalk, this will look at the code (PHP, MySQL,etc) and provide the underlying infrastructure. (Important for Developer Exam).
  4. Lambda: It is actually called serverles, for EC2 allows you to login to the operating system, but here we will upload the code and the code will respond to events. (Still not coming up on the Developers Exam but it is very important).
  5. LightSail: Out of the box cloud, for people who don't know AWS. (Not in Developer Associate Exam).
Storage:
  1. S3: Simple Storage Service, is an object based storage and not to install anything(Important for Developer Exam).
  2. Glacier: Files from S3 are archived from S3, it is much cheaper than S3.
  3. EFS: Elastic File Service, it is a block based storage and used for installation and share the volume with multiple machines.
  4. Storage Gateway: It is a way of connecting up S3 to on premis data center, a VM that you install in premis and that communicates to S3.(Not important for developer, IMP for SYSOPS).  
Databases:
  1. RDS: It is a Relational Database Service, it consists of a number of database technologies (Oracle, MySql etc.,).(Not Stressed much in Developer Exam).
  2. DynamoDB: It is a non relational database service (No Sql DB), it is really scalable and very high performance. (Very Imp for Developer Exam)
  3. RedShift: It is data warehousing solution, a copy of production is moved on to RedShift and reports are run so that we are not effecting the production performance.
  4. ElastiCache: It is a way of caching data in the cloud. Lets say top selling items on a site, it is cached using ElastiCache. 
Migration Services:
  1. SnowBall: It was a way of sending disks to Amazon and the content is transferred to S3 etc., but now it is done at enterprise level and is a briefcase sized appliance and load it and send it back to Amazon. SnowBall Edge has compute capacity to it to have an AWS on prems.
  2. DMS: Database migration services, this allows the on premis data bases to cloud and also to other regions. Important advantage is we don't have to stick to the same data base service.
  3. SMS: Server migration services, to replicate VMs in the cloud.(not in developers exam).
Analytics (Not very Imp for Dev Assoc exams, just need to know what each service does): 
  1. Athena: Allows to run SQL on S3 (not in developers exam). 
  2. EMR: Elastic Map Reduce, this is used for Big Data processing, to process large amounts of data and uses Hadoop(not in developers exam).
  3. Cloud Search/ Elastic Search: Search engines for websites or application. Cloud Search is fully managed by AWS, but Elastic Search is based on an open source.
  4. Kinesis: It s a way of streaming and analyzing real time data at a massive scale.(Imp for Bigdata Speciality Exam).
  5. Data PipeLine: This is used to move data from S3 to DynamoDB and vice versa etc.,
  6. Quick Sight: It is a business analytics tool and create visualizations in rich dashboards.
Security & Identity:
  1. IAM: Provides permissions and assigns roles to the users(Very Very Imp for all the certifications).
  2.  Inspector: Agent on the virtual machine to inspects virtual machines and provides the security reports.
  3. Certificate Manager: Gives free SSL certificates for domain names.
  4. Directory Service: Way of connecting AD to the AWS.(Imp for Assoc Achitect). Only theory
  5. WAF: Web Application Firewall, this gives application level firewall.(Not Very Imp for exam)
  6. Artifacts: These are just documentation on the AWS console.
Management Tools(Asked in all the three Assoc. Exams.):
  1. Cloud Watch: Used to monitor performance like EC2, (mostly asked in SysOps).
  2. Cloud Formation: Is a way of turning infrastructure into cloud.(Very Very Imp for sols architect).
  3. Cloud Trail: Is used for auditing AWS resources. Changes to the AWS is recording.
  4. OpsWorks: Is basically a way of automating deployments using Shift(comes in DevOps).
  5. Config: It a way of monitoring the env and gives warning automatically.
  6. Service Catalog: This is a service for larger enterprises. Allows authorization and non authorization(Not Imp).
  7. Trusted Advisor: It s an automated way of getting suggestions. (Imp for assoc architect).
Application Services:
  1. Step Functions: Its a way of visualizing what is going on inside the applcations.
  2. Simpe WorkFlow: It is a way of coordination automated tasks and manual tasks.
  3. API Gateway: Create, publish, maintain, monitor and secure apps to access data from the backend (business logic, functionality like lambda etc.,). (Not Imp for exam).
  4. AppStream: It is a way of streaming desktop applications to the users.(Not Imp for exam).
  5. Elastic Transcoder: Changes the video format to match all the devices.(Not Imp for exam).
Developer Tools:(Not Imp until now for Assoc Dev)
  1. CodeCommit: It is basically GitHub, where we store the code securely.
  2. CodeBuild: We pay by the min and used for compiling the code.
  3. CodeDeploy: To deploy code to EC2 .
  4. CodePipeLine: Track of code, basically versions of code.
Mobile Services:(Not Imp until now for Assoc Dev)
  1. ModileHub: Add configure and design feature for mobile apps to include things like user authentication, data storage, backend logic, push notifications etc.,
  2. Cognito: Service for the users to sign up and sign in for the apps and using things like social identity providers.
  3. DeviceFarm: Testing the apps on a number of smart devices.
  4. Mobile Analytics: Collect and analyze app usage data.
  5. PinPoint: This enables understand and engage with the application users.
Business Productivity:
  1. WorkDocs: Storing imp word docs.
  2. WorkMail: Exchange for AWS.
IOT:
  1. IoT: Keeping track of all the devices.
Desktop & App Streaming:
  1. WorkSpaces: It is like having a work station in the cloud.
  2. AppStream 2.0:
Artificial Intelligent:
  1. Alexa: Voice Service on the cloud. 
  2. Polly: Most advanced text to speech service.
  3. Machine Learing: Give a data set and AWS will determine the outcome of it.
  4. Rekognition: It will tell you what is in the picture.
Messaging(Imp for all three Assoc Exams):
  1. SNS: Simple Notification Services by sending email, text etc.,
  2. SQS: A way of decoupling applications(Very Imp).
  3. Simple Email Service: Way of sending emails.

Thursday, April 20, 2017

AWS Services Summary:

All the services in AWS are summarized with one liners below for quick reference:

N/W and Content Delivery:
    VPC ; Availability Centers are physical Data centers and VPC is virtual data center. Multiple VPCs in a region.
    Route53 ; Domain names are registered under the route53 service of aws.
    CloudFront ; It is a part of CDN, it consists of all the edge locations which will cache the assets giving faster accessibility to the users
    Direct Connect : It is a dedicated line to the data center from the office for seemless accessibility.

Storage:
    S3 - Simple Storage Services; Virtual Disk in the cloud to store objects (docs/files/text files/movies etc). Not to install a DB, an application, a computer game. This is an object based storage. For installing a DB, application or a computer game we need a block based storage.
    Galcier; Glacier is used to archieve files from S3, It is used when we dont need instant access.
    EFS - Elastic File System; EFS is a block based storage.
    Storage Gateway; It allows users to connect S3 to an on premise data center or to the head quarters. It is a virtual machine installed on premis. 

Compute :
    EC2 Elastic Cloud Compute; Virtual machines in the cloud that run on AWS.
    EC2 Container service; allows applications to run on a managed cluster of EC2 instances (i.e., virtual machines)
    Elastic BeanStalk; Used to deploy the code onto AWS. It will check the code for any errors.
    Lambda; It runs the code without any managed servers. Can run code of virtually any application or backend service
    LightSail; Choose a configuration from the menu and launch a virtual machine preconfigured with SSD based storage, DNS management, and a static IP addres.

Databases :
    RDS; Relational Databases; MySQL, SQL Server, Oracle, etc all come under this section.
    DynamoDB; Non Relational Databases; NoSQL database and can be scalable very vastly.
    Redshift; Amazon warehousing solution; BigData in the warehouse; Query is not directly run on the database instead run on a copy in the redshift.
    Elasticache; Way of caching data in the cloud, usually done on data which is most frequently used.
   
Security and Identity:
    IAM - Identity And Access Management;  Used to sign in to AWS, setup permissions, groups etc.,
    Inspector; Its an agent installed on the virtual machine and inspects the virtual machine and does security reporting.
    Certificate Manager; This gives free SSL certificates for the domain names.
    Directory Service; This is a way of using MS AD with AWS.
    WAF - Web Application Firewall; Firewall gives N/W protection, this gives application level protection.
    Artifacts; Place to get documentation.

Migration Services:
    Snowball; Appliance used to store data and send back to amazon.
    Snowball Edge; Similar to Snowball but with compute capability like a data center.
    DMS - Database Migration Services; DMS will help migrate the data with no downtime.
    SMS - Server Migration Services; Exactly like DMS but for migrating VMWare.

Analytics:
    Athena; Allows SQL queries on S3, Turning Flat files into searchable data bases.
    Ealstic Map Reduce; Used for big data processing and based on Hadoop, Apache Spark etc
    Cloud Search; Similar to elastic, search engine for website or application.
    Elastic Search; This service uses open source.
    Kinesis; Way of analyzing and streaming real time data, analyzing market, social media tweets etc.
    Data Pipeline; It is a service used to move data from place to place. Ex:- from S3 to DynamoDB.
    Quick Sight; This is a business analytics tool used for creating rich dashboards for the data.
   

   
Management Tools:
    Cloud Watch; This is imp for SysOps certification; Used for monitoring performance of AWS environment especially EC2.
    Cloud Formation; It is a way of turning physical infrastructure into code, Using this we can provision production environment using one script.
    Cloud Trial; Auditing AWS resources.
    Opswork; A way of automating deployments using shift.
    Config; Config manager automatically monitors environment and gives warnings if the config might break.
    Trusted Advisor; Designed by the AWS solutions architecture team. They make series of recommendations; Security, Cost optimization, series of recommendations which is automated by scanning the env.
   
Application Services:
    Step Functions; Visualizing whats going on inside the application basically what micro services it is using.
    SWF - Simple Workflow Service; A way of coordinating automated tasks and manual tasks.
    API Gateway; Door for the Apps to access the back end data things like Lambda etc.,
    AppStream; It is a way of streaming desktop applications to the users.
    Elastic Transcoder; Used for changing the video format to suit all different devices based on resolution, size etc.,
   
Developer Tools:
    CodeCommit; It is a GitHub, GitHub is place to store code.
    CodeBuild; It is a way to compile code.
    CodeDeploy; Way of deploying in an automated fashion.
    CodePipeLine; Way of keeping versions of the code.
   
Mobile Services:
    Mobile Hub; This lets to add, configure and design features for mobile apps which includes, user auth, data storage, push notifications, back end logic, content delivery and anlytics.
    Cognito; Used for signing up using gmail credentials. Used in IOS with AWS.
    Device Farm; Helps improve the mobile apps by quickly and securing the apps on hundreds of real smart phones.
    Mobile Analytics; This app helps in analyzing the app usage data in a cost effective manner.
    Pinpoint; Helps in engage and understand with the application users. For Ex:- Helps in creating targeted marketing campaigns.

Business Productivity:
    WorkDocs; Used for storing important word documents in the cloud. In the back end using S3 with extra security added.
    Workmail; This is like exchange for AWS. A way of sending and receiving the emails.
   
iOT- Internet of Things:
    This is a way of having millions of devices keeping keeping track of them using the iOT gateway.
   
Desktop & AppStreaming:
    WorkSpaces; Workspace is a way of having desktop on the cloud.
    AppStream; It is a way of streaming desktop applications to the users.
   
Artificial Intelligence:
    Alexa; Alexa is amazons voice service in the cloud, can be communicated to alexa using echo.
    Polly; It is the most advanced text to speech service on the planet. Can use 47 voices and available in 24 languages.
    Machine Learning; This allows to predict data based on the previous performance.
    Rekognition; This service will recognize an uploaded picture and tells the details in the picture(objects, nature, humans, animals, gender etc.,)

Messaging:
    SNS - Simple Notification Services; this is a way of notifying either by email, text message.
    SQS; This is a way of decoupling applications, it is a queue system.
    SES - Simple Email Serice; A way of sending and receiving emails.



FYI: Regions --> Two or more Availability Zones inside regions --> Each Availability Zone is a data center/centers
Edge Location is a content delivery location. They are CDN end points for CloudFront.
CDN : Way to cache media files in the cloud. From the second user the cached filed is fetched from the edge location.

Amazon Storage Gateway - In Detail

AWS Storage Gateway; is a service that connects an on-premises software appliance with cloud-based storage to provide seemless and secure integration between an organizations on-premise IT environment and AWS's storage infrastructure. The service enables you to securely store data to the AWS cloud for scalable and cost-efficient storage.

Files are stored as objects in your S3 buckets, accessed through a Network File System (NFS) mount point. Ownership, permissions, and timestamps are durably stored in S3 in the user-metadata of the object associated with the file. Once objects are transferred to S3, they can be managed as native S3 objects, and bucket policies such as versioning, lifecycle management and cross-region replication apply directly to the objects stored in the bucket.

Storage Gateway is a virtual appliance installed into a host in the data center and this will propagate replicate data to AWS (particularly to S3/Galcier). This is available for download as a virtual machine image.



Once the Gateway is installed and associated with the AWS account through the activation process, you can use the AWS management console to create the storage gateway option that is right for you.

Four types of Storage Gateways:
  1. File Gateway (NFS) used for storing flat files on S3.
  1. Volume Gateway (iSCSI) used for block based storage.
    - Stored Volumes: Entire copy of data on premis.
    - Cached Volumes: Storing most recently accessed data.
  2. Tape Gateway (VTL)

AWS Identity Access Management