Elastic Network Interfaces

 ELASTIC NETWORK INTERFACES

• Logical component in a VPC that represents a virtual network card

• ENI has following attributes : 
• Primary private IPv4 and one or more secondary IPv4
• One elastic (IP) IPv4 per private IPv4
• One public IP (IPv4)
• One or more security groups
• a MAC address

• ENIs can be created independently and attached on the fly to EC2 instances during failover.

• Bound to a specific AZ

EC2 INSTANCE PLACEMENT STRATEGY

PLACEMENT GROUPS

• Logical grouping of instances within an availability zone.
• Types of placement groups - 
• Cluster - instances bucketed into a low-latency group in a single Availability Zone.
• Same rack
• Same AZ
• Same hardware
• Pro: Provides a great network (10 Gbps speed between instances)
• Con: If the rack fails, all instances will suffer downtime.
• Use-case
• Big Data jobs that need to complete very fast
• Apps requiring extremely low latency and high network throughput
• Risk appetite of failure is acceptable to the use cases.

• Spread - instances are spread across different hardware (max 7 instances per group per AZ)
• Minimises the failure risk
• EC2 instances are placed on different hardware.
• Pro: Reduced risk of simultaneous failure
• Pro: spans across AZ
• Con: Limited to 7 instances per AZ
• Use Case : 
• Apps requiring high availability
• Critical apps where each instance must be isolated from failure of another instance.

• Partition - spreads instances across many different partitions which rely on different sets of racks of hardware within an AZ. Capable of scaling to 100s of instances per group (Hadoop, Cassandra, Kafka)
• instances spread across partitions in multiple AZs
• can have up to 7 partitions per AZ
• each partition can have many EC2 instances
• each partition represents a rack
• Pro : safe from rack failure
• Pro : allows 100s of EC2 instances to be set up
• Pro : instances in a partition do not share hardware racks with other partitions
• partition information available as metadata to the EC2 instances
• Use-Cases : Partition aware apps like Big Data apps : HDFS, HBase, Cassandra, Kafka

Elastic Cloud Compute

 

EC2

Features

• EC2 = Elastic Cloud Compute -> Infrastructure as a Service
• Rent virtual machines
• Store data on virtual drives (EBS)
• Load distribution across machines (ELB)
• Scaling the services using auto-scaling group (ASG)

EC2 sizing and configuration options

• OS
• CPU
• RAM
• Storage space (EBS &EFS)
• Network card speed
• Firewall rules
• Bootstrap script : EC2 user data

Bootstrapping 

• To launch commands when the machine starts
• Carried out using EC2 User data scripts
• Script runs only once at the instance's first start
• Used to automate boot tasks such as 
• install updates
• install software
• download common files from internet
• EC2 User data script runs under root user (sudo rights)

EC2 Instance Types

• General Purpose
• great for diversity of workloads (web services and code repos)
• good balance between compute, memory and networking
• example : t2.micro

• Compute Optimised
• Good for compute intensive tasks 
• batch processing workloads
• media transcoding
• high performance web servers
• high performance computing
• scientific modelling & machine learning
• dedicated gaming servers
• Example C6g, C6gn, C5, C5a, etc

• Memory Optimised
• Fast performance for workloads processing large data sets
• Use cases:
• high performance, relational/non-relational databases
• distributed web scale cache stores
• In-memory databases optimised for BI
• Apps performing real-time processing of big unstructured data
• Example : R-series, X, Z, high memory

• Storage Optimised
• Great for storage intensive tasks requiring high sequential read/write access to large data sets on local storage
• Use cases:
• high frequency online transaction processing (OLTP) systems
• relational and NoSQL databases
• Cache for in-memory databases
• Data warehousing apps
• distributed file systems
• Example : I-series, D and H

AWS naming convention : Using example of m5.2xlarge

m            : instance class

5             : generation (incremented as AWS improves them over time)

2xlarge    : size within the instance class (indicates the size of memory, cpu, etc)

 

Security Groups

• Fundamental of network security of AWS
• controls how traffic is allowed into or out of EC2 instances.
• only contain "allow" rules
• rules can reference by IP or other security groups
• act as "firewall" on EC2 instances
• they regulate :
• access to ports
• authorised IP ranges - IPv4 and IPv6
• control of inbound network 
• control of outbound network
• can be attached to multiple instances
• instances can have multiple security groups
• closely coupled with the region/VPC combination
• exists as a "firewall" outside of the EC2 instance
• Good practice to maintain one separate security group for SSH access
• if application is not accessible (time outs), its probably a security group issue
• However, "connection refused" indicates that traffic went through but application errored out or didn't launch
• By default, all inbound traffic is blocked and all outbound traffic is authorised

Classic Ports to know

• 22 = SSH (secure shell) - log into a Linux instance
• 21 = FTP - upload file into a file share
• 22 = SFTP - upload files using SSH
• 80 = HTTP - to access unsecured websites
• 443 = HTTPS - access secured websites
• 3389 = RDS (remote desktop protocol) - log into a Windows instance

IAM

 

Identity & Access Management (IAM)

• Global Service
• Root account created by default
• Root account should only be used for initial account set up and then should never be used.
• Users should be created and they can be grouped. For example, a group can be created for developers having certain enhanced privileges whereas another for operations team with a restricted access.
• Users don't necessarily have to belong to a group and can be part of multiple groups.

Permissions

• Users & groups can be assigned JSON documents called policies.
• Defines permissions of users
• AWS convention is to apply the least privilege principle; don't give more permissions than a user needs.

Policies

• Policy at the group level - all users in an IAM group would have the same permissions
• Inline policy - policy attached to an individual user - not part of a group.

Policy document structure : 

• Version: policy language version, always "2012-10-17"
• Id: an identifier for the policy (optional)
• Statement : one or more individual rules (required)

Statement structure : 

• Sid : an identifier for the statement (optional)
• Effect : whether the statement allows or denies access (Allow / Deny)
• Principal : account/user/role the policy is applied to
• Action: list of actions this policy allows/denies
• Resource: list of resources to which the actions are applied to
• Condition: conditions for when this policy is in effect (optional)

Multi-factor Authentication

• Virtual MFA devices - Google Authenticator, Authy
• Yubikey - universal 2nd factor security key
• Hardware devices - Key Fob by Gemalto
• AWS GovCloud - Key Fob

IAM roles for Services

• Some AWS services will need to perform actions on our behalf
• Permissions need to be assigned to AWS services with IAM roles
• Common roles : EC2 instance roles / Lambda function roles / Cloudformation

IAM Best Practices

• Root user should not be used except for AWS account setup.
• One physical user = One AWS user
• Assign users to groups and assign permissions to groups
• Create strong password policy
• Use and enforce MFA
• Create and use Roles for giving permissions to AWS services
• Use access keys for programmatic access (CLI/SDK)

Infrastructure

AWS Global Infrastructure

1. AWS Regions

• Located all around the world
• Each region is named such as us-east-1, eu-west-3, etc
• Cluster of data centres
• Most AWS services are region-scoped

How to choose an AWS Region?

Depends on - 

• Compliance with data governance and legal requirements
• Proximity to the customers (reduce latency)
• Available Services within a Region - Not all region have all the AWS services 
• Pricing - varies from region to region

2. AWS Availability Zones

• Each Region has usually 3, minimum 2 and maximum 6 AZs
• Each AZ is composed of one or more discrete DCs with redundant power, networking and connectivity.
• AZs are separate from each other to isolate from disasters.
• They're connected with high bandwidth, ultra-low latency networking

3. AWS Edge locations / points of presence

• AWS has more than 200 points of presence in 84 cities across 42 countries.
• Content is delivered to end users with lower latency

Key Services

AWS Global Services

• Identity and Access Management (IAM)
• Route 53 (DNS Service)
• CloudFront (Content Delivery Network)
• WAF (Web Application Firewall)

Region-scoped Services

• Amazon EC2 (Infra as a service)
• Elastic Beanstalk (Platform as a service)
• Lamba (Function as a service)
• Rekognition (Software as a service)