AWS Cloud Fundamentals for DevOps Engineers

2025-10-30 · Pavan Y N · aws, cloud, ec2, s3, iam, devops

Published: October 30, 2025
Reading Time: 30 minutes
Tags: AWS, Cloud Computing, EC2, S3, IAM, DevOps

# Introduction

Amazon Web Services (AWS) is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. For DevOps engineers, understanding AWS is essential for building scalable, reliable, and cost-effective infrastructure.

This comprehensive guide covers AWS fundamentals, core services, best practices, and real-world implementation patterns.

# AWS Global Infrastructure

## Regions and Availability Zones

Regions: Geographic locations where AWS has data centers

→33+ regions worldwide
→Each region is completely independent
→Choose regions based on latency, compliance, and cost

Availability Zones (AZs): Isolated data centers within a region

→Each region has multiple AZs (typically 3-6)
→Connected through low-latency links
→Design for high availability across AZs

Edge Locations: CDN endpoints for CloudFront

→400+ edge locations globally
→Caches content closer to users
→Reduces latency for global applications

code

Region: us-east-1 (N. Virginia)
├── AZ: us-east-1a
├── AZ: us-east-1b
├── AZ: us-east-1c
├── AZ: us-east-1d
├── AZ: us-east-1e
└── AZ: us-east-1f

# Core AWS Services

## 1. Identity and Access Management (IAM)

Concept: IAM controls who can access AWS resources and what they can do.

Key Components:

→Users: Individual identities
→Groups: Collections of users
→Roles: Temporary credentials for services
→Policies: JSON documents defining permissions

Best Practices:

→Enable MFA for root account
→Use roles instead of access keys
→Follow least privilege principle
→Rotate credentials regularly

Example IAM Policy:

code

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowEC2ReadOnly",
      "Effect": "Allow",
      "Action": [
        "ec2:Describe*",
        "ec2:Get*",
        "ec2:List*"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowS3BucketAccess",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::my-app-bucket/*"
    },
    {
      "Sid": "DenyProductionAccess",
      "Effect": "Deny",
      "Action": "*",
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "aws:RequestedRegion": "us-east-1",
          "ec2:ResourceTag/Environment": "production"
        }
      }
    }
  ]
}

Creating IAM Role for EC2:

code

# Create trust policy
cat > trust-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

# Create role
aws iam create-role \
  --role-name EC2-S3-Access-Role \
  --assume-role-policy-document file://trust-policy.json

# Attach policy
aws iam attach-role-policy \
  --role-name EC2-S3-Access-Role \
  --policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

# Create instance profile
aws iam create-instance-profile \
  --instance-profile-name EC2-S3-Access-Profile

# Add role to instance profile
aws iam add-role-to-instance-profile \
  --instance-profile-name EC2-S3-Access-Profile \
  --role-name EC2-S3-Access-Role

## 2. Elastic Compute Cloud (EC2)

Concept: Virtual servers in the cloud with flexible compute capacity.

Instance Types:

→General Purpose: t3, t4g, m5, m6i (balanced compute, memory, networking)
→Compute Optimized: c5, c6i (high-performance processors)
→Memory Optimized: r5, r6i, x2 (large datasets in memory)
→Storage Optimized: i3, d2 (high sequential read/write)
→Accelerated Computing: p4, g5 (GPU instances)

Pricing Models:

→On-Demand: Pay per hour/second
→Reserved: 1-3 year commitment (up to 75% savings)
→Spot: Bid on spare capacity (up to 90% savings)
→Savings Plans: Flexible pricing (up to 72% savings)

Launching EC2 Instance:

code

# Create key pair
aws ec2 create-key-pair \
  --key-name my-key-pair \
  --query 'KeyMaterial' \
  --output text > my-key-pair.pem

chmod 400 my-key-pair.pem

# Create security group
aws ec2 create-security-group \
  --group-name web-server-sg \
  --description "Security group for web servers"

# Add inbound rules
aws ec2 authorize-security-group-ingress \
  --group-name web-server-sg \
  --protocol tcp \
  --port 22 \
  --cidr 0.0.0.0/0

aws ec2 authorize-security-group-ingress \
  --group-name web-server-sg \
  --protocol tcp \
  --port 80 \
  --cidr 0.0.0.0/0

# Launch instance
aws ec2 run-instances \
  --image-id ami-0c55b159cbfafe1f0 \
  --instance-type t3.micro \
  --key-name my-key-pair \
  --security-groups web-server-sg \
  --iam-instance-profile Name=EC2-S3-Access-Profile \
  --user-data file://user-data.sh \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=WebServer},{Key=Environment,Value=Production}]'

User Data Script:

code

#!/bin/bash
# user-data.sh

# Update system
yum update -y

# Install Docker
yum install -y docker
systemctl start docker
systemctl enable docker

# Add ec2-user to docker group
usermod -a -G docker ec2-user

# Install Docker Compose
curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

# Install CloudWatch agent
wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
rpm -U ./amazon-cloudwatch-agent.rpm

# Pull and run application
docker pull myregistry.azurecr.io/myapp:latest
docker run -d -p 80:3000 --name myapp myregistry.azurecr.io/myapp:latest

Auto Scaling Group:

code

# Create launch template
aws ec2 create-launch-template \
  --launch-template-name web-server-template \
  --version-description "Version 1" \
  --launch-template-data '{
    "ImageId": "ami-0c55b159cbfafe1f0",
    "InstanceType": "t3.micro",
    "KeyName": "my-key-pair",
    "SecurityGroupIds": ["sg-0123456789abcdef0"],
    "IamInstanceProfile": {
      "Name": "EC2-S3-Access-Profile"
    },
    "UserData": "'"$(base64 -w 0 user-data.sh)"'",
    "TagSpecifications": [{
      "ResourceType": "instance",
      "Tags": [
        {"Key": "Name", "Value": "WebServer"},
        {"Key": "Environment", "Value": "Production"}
      ]
    }]
  }'

# Create Auto Scaling group
aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name web-server-asg \
  --launch-template LaunchTemplateName=web-server-template,Version='$Latest' \
  --min-size 2 \
  --max-size 10 \
  --desired-capacity 3 \
  --availability-zones us-east-1a us-east-1b us-east-1c \
  --target-group-arns arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/web-servers/1234567890abcdef \
  --health-check-type ELB \
  --health-check-grace-period 300

# Create scaling policy
aws autoscaling put-scaling-policy \
  --auto-scaling-group-name web-server-asg \
  --policy-name scale-up-policy \
  --policy-type TargetTrackingScaling \
  --target-tracking-configuration '{
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ASGAverageCPUUtilization"
    },
    "TargetValue": 70.0
  }'

## 3. Simple Storage Service (S3)

Concept: Object storage service with 99.999999999% (11 9's) durability.

Storage Classes:

→S3 Standard: Frequently accessed data
→S3 Intelligent-Tiering: Automatic cost optimization
→S3 Standard-IA: Infrequently accessed data
→S3 One Zone-IA: Single AZ storage
→S3 Glacier: Long-term archive (minutes to hours retrieval)
→S3 Glacier Deep Archive: Lowest cost (12-hour retrieval)

Creating and Configuring S3 Bucket:

code

# Create bucket
aws s3 mb s3://my-app-bucket-12345 --region us-east-1

# Enable versioning
aws s3api put-bucket-versioning \
  --bucket my-app-bucket-12345 \
  --versioning-configuration Status=Enabled

# Enable encryption
aws s3api put-bucket-encryption \
  --bucket my-app-bucket-12345 \
  --server-side-encryption-configuration '{
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "AES256"
      },
      "BucketKeyEnabled": true
    }]
  }'

# Configure lifecycle policy
cat > lifecycle-policy.json <<EOF
{
  "Rules": [
    {
      "Id": "Move to IA after 30 days",
      "Status": "Enabled",
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        }
      ],
      "NoncurrentVersionTransitions": [
        {
          "NoncurrentDays": 30,
          "StorageClass": "GLACIER"
        }
      ],
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 365
      }
    }
  ]
}
EOF

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-app-bucket-12345 \
  --lifecycle-configuration file://lifecycle-policy.json

# Enable access logging
aws s3api put-bucket-logging \
  --bucket my-app-bucket-12345 \
  --bucket-logging-status '{
    "LoggingEnabled": {
      "TargetBucket": "my-logs-bucket",
      "TargetPrefix": "s3-access-logs/"
    }
  }'

# Configure CORS
cat > cors-config.json <<EOF
{
  "CORSRules": [
    {
      "AllowedOrigins": ["https://example.com"],
      "AllowedMethods": ["GET", "PUT", "POST", "DELETE"],
      "AllowedHeaders": ["*"],
      "ExposeHeaders": ["ETag"],
      "MaxAgeSeconds": 3000
    }
  ]
}
EOF

aws s3api put-bucket-cors \
  --bucket my-app-bucket-12345 \
  --cors-configuration file://cors-config.json

S3 Bucket Policy:

code

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PublicReadGetObject",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::my-app-bucket-12345/public/*"
    },
    {
      "Sid": "DenyInsecureTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-app-bucket-12345",
        "arn:aws:s3:::my-app-bucket-12345/*"
      ],
      "Condition": {
        "Bool": {
          "aws:SecureTransport": "false"
        }
      }
    },
    {
      "Sid": "AllowCloudFrontAccess",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E1234567890ABC"
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::my-app-bucket-12345/*"
    }
  ]
}

## 4. Virtual Private Cloud (VPC)

Concept: Isolated virtual network in AWS cloud.

Components:

→Subnets: IP address ranges within VPC
→Route Tables: Control traffic routing
→Internet Gateway: Connect to internet
→NAT Gateway: Outbound internet for private subnets
→Security Groups: Instance-level firewall
→Network ACLs: Subnet-level firewall

Creating VPC with Terraform:

code

# VPC
resource "aws_vpc" "main" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name        = "main-vpc"
    Environment = "production"
  }
}

# Internet Gateway
resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name = "main-igw"
  }
}

# Public Subnets
resource "aws_subnet" "public" {
  count                   = 3
  vpc_id                  = aws_vpc.main.id
  cidr_block              = "10.0.${count.index + 1}.0/24"
  availability_zone       = data.aws_availability_zones.available.names[count.index]
  map_public_ip_on_launch = true

  tags = {
    Name = "public-subnet-${count.index + 1}"
    Type = "public"
  }
}

# Private Subnets
resource "aws_subnet" "private" {
  count             = 3
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.0.${count.index + 10}.0/24"
  availability_zone = data.aws_availability_zones.available.names[count.index]

  tags = {
    Name = "private-subnet-${count.index + 1}"
    Type = "private"
  }
}

# NAT Gateway
resource "aws_eip" "nat" {
  count  = 3
  domain = "vpc"

  tags = {
    Name = "nat-eip-${count.index + 1}"
  }
}

resource "aws_nat_gateway" "main" {
  count         = 3
  allocation_id = aws_eip.nat[count.index].id
  subnet_id     = aws_subnet.public[count.index].id

  tags = {
    Name = "nat-gateway-${count.index + 1}"
  }
}

# Route Tables
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.main.id
  }

  tags = {
    Name = "public-route-table"
  }
}

resource "aws_route_table" "private" {
  count  = 3
  vpc_id = aws_vpc.main.id

  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.main[count.index].id
  }

  tags = {
    Name = "private-route-table-${count.index + 1}"
  }
}

# Route Table Associations
resource "aws_route_table_association" "public" {
  count          = 3
  subnet_id      = aws_subnet.public[count.index].id
  route_table_id = aws_route_table.public.id
}

resource "aws_route_table_association" "private" {
  count          = 3
  subnet_id      = aws_subnet.private[count.index].id
  route_table_id = aws_route_table.private[count.index].id
}

# Security Groups
resource "aws_security_group" "web" {
  name        = "web-sg"
  description = "Security group for web servers"
  vpc_id      = aws_vpc.main.id

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "web-security-group"
  }
}

resource "aws_security_group" "database" {
  name        = "database-sg"
  description = "Security group for databases"
  vpc_id      = aws_vpc.main.id

  ingress {
    from_port       = 5432
    to_port         = 5432
    protocol        = "tcp"
    security_groups = [aws_security_group.web.id]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "database-security-group"
  }
}

## 5. Elastic Load Balancing (ELB)

Types:

→Application Load Balancer (ALB): HTTP/HTTPS traffic (Layer 7)
→Network Load Balancer (NLB): TCP/UDP traffic (Layer 4)
→Gateway Load Balancer (GWLB): Third-party virtual appliances
→Classic Load Balancer: Legacy (not recommended)

Creating Application Load Balancer:

code

# Create ALB
aws elbv2 create-load-balancer \
  --name web-alb \
  --subnets subnet-12345678 subnet-87654321 \
  --security-groups sg-0123456789abcdef0 \
  --scheme internet-facing \
  --type application \
  --ip-address-type ipv4 \
  --tags Key=Name,Value=web-alb Key=Environment,Value=production

# Create target group
aws elbv2 create-target-group \
  --name web-targets \
  --protocol HTTP \
  --port 80 \
  --vpc-id vpc-12345678 \
  --health-check-protocol HTTP \
  --health-check-path /health \
  --health-check-interval-seconds 30 \
  --health-check-timeout-seconds 5 \
  --healthy-threshold-count 2 \
  --unhealthy-threshold-count 3 \
  --matcher HttpCode=200

# Create listener
aws elbv2 create-listener \
  --load-balancer-arn arn:aws:elasticloadbalancing:us-east-1:123456789012:loadbalancer/app/web-alb/1234567890abcdef \
  --protocol HTTPS \
  --port 443 \
  --certificates CertificateArn=arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012 \
  --default-actions Type=forward,TargetGroupArn=arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/web-targets/1234567890abcdef

# Add listener rule for path-based routing
aws elbv2 create-rule \
  --listener-arn arn:aws:elasticloadbalancing:us-east-1:123456789012:listener/app/web-alb/1234567890abcdef/1234567890abcdef \
  --priority 10 \
  --conditions Field=path-pattern,Values='/api/*' \
  --actions Type=forward,TargetGroupArn=arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/api-targets/1234567890abcdef

# STAR Approach: AWS Migration

Situation: Company running on-premises infrastructure with high costs, limited scalability, and frequent outages.

Task: Migrate 50+ applications to AWS while maintaining 99.9% uptime and reducing costs by 40%.

Action:

•
Assessment & Planning:
- →Conducted application discovery and dependency mapping
- →Categorized applications (rehost, replatform, refactor)
- →Created migration roadmap with 6-month timeline
•
Landing Zone Setup:
- →Designed multi-account strategy using AWS Organizations
- →Implemented VPC architecture with public/private subnets
- →Set up Transit Gateway for inter-VPC communication
- →Configured AWS Direct Connect for hybrid connectivity
•
Migration Execution:
- →Wave 1: Lift-and-shift using AWS Application Migration Service
- →Wave 2: Containerized applications on ECS/EKS
- →Wave 3: Serverless refactoring with Lambda
•
Security Implementation:
- →Enabled AWS Config for compliance monitoring
- →Implemented AWS Security Hub for centralized security
- →Set up AWS GuardDuty for threat detection
- →Configured AWS WAF for application protection
•
Monitoring & Optimization:
- →Deployed CloudWatch for metrics and logging
- →Set up AWS Cost Explorer and Budgets
- →Implemented auto-scaling for all workloads
- →Used Reserved Instances and Savings Plans

Result:

→Successfully migrated all applications within 5 months
→Achieved 99.95% uptime (exceeded target)
→Reduced infrastructure costs by 45%
→Improved deployment speed by 10x
→Reduced time to provision resources from weeks to minutes
→Enhanced disaster recovery capabilities (RPO: 1 hour, RTO: 4 hours)

# Common Issues & Troubleshooting

## Issue 1: EC2 Instance Not Accessible

Symptoms:

→Cannot SSH/RDP to instance
→Connection timeout

Diagnosis:

code

# Check instance status
aws ec2 describe-instance-status --instance-ids i-1234567890abcdef0

# Check security group rules
aws ec2 describe-security-groups --group-ids sg-0123456789abcdef0

# Check route table
aws ec2 describe-route-tables --filters "Name=association.subnet-id,Values=subnet-12345678"

Solutions:

→Verify security group allows inbound traffic on correct port
→Ensure instance has public IP or Elastic IP
→Check Network ACL rules
→Verify route table has route to Internet Gateway
→Check key pair permissions (chmod 400)

## Issue 2: S3 Access Denied

Symptoms:

→403 Forbidden errors
→Access Denied messages

Diagnosis:

code

# Check bucket policy
aws s3api get-bucket-policy --bucket my-bucket

# Check bucket ACL
aws s3api get-bucket-acl --bucket my-bucket

# Check object ACL
aws s3api get-object-acl --bucket my-bucket --key myfile.txt

# Test IAM permissions
aws iam simulate-principal-policy \
  --policy-source-arn arn:aws:iam::123456789012:user/myuser \
  --action-names s3:GetObject \
  --resource-arns arn:aws:s3:::my-bucket/myfile.txt

Solutions:

→Review and update bucket policy
→Check IAM user/role permissions
→Verify object ownership
→Ensure bucket is not blocking public access when needed
→Check for conflicting deny statements

## Issue 3: High AWS Costs

Symptoms:

→Unexpected bill increases
→Budget alerts triggered

Diagnosis:

code

# Get cost and usage
aws ce get-cost-and-usage \
  --time-period Start=2025-10-01,End=2025-10-31 \
  --granularity MONTHLY \
  --metrics BlendedCost \
  --group-by Type=SERVICE

# Identify unused resources
aws ec2 describe-volumes --filters "Name=status,Values=available"
aws ec2 describe-addresses --filters "Name=domain,Values=vpc" --query 'Addresses[?AssociationId==null]'
aws rds describe-db-instances --query 'DBInstances[?DBInstanceStatus==`available`]'

Solutions:

→Delete unused EBS volumes and snapshots
→Release unattached Elastic IPs
→Stop/terminate unused EC2 instances
→Use Reserved Instances or Savings Plans
→Implement auto-scaling
→Enable S3 Intelligent-Tiering
→Use AWS Cost Anomaly Detection

# Best Practices

## 1. Security

→Enable MFA for all users
→Use IAM roles instead of access keys
→Encrypt data at rest and in transit
→Implement least privilege access
→Enable CloudTrail for audit logging
→Use AWS Secrets Manager for credentials

## 2. High Availability

→Deploy across multiple AZs
→Use Auto Scaling groups
→Implement health checks
→Use managed services when possible
→Design for failure

## 3. Cost Optimization

→Right-size instances
→Use Reserved Instances/Savings Plans
→Implement auto-scaling
→Delete unused resources
→Use S3 lifecycle policies
→Monitor with Cost Explorer

## 4. Performance

→Use CloudFront for content delivery
→Implement caching strategies
→Choose appropriate instance types
→Use Elastic Load Balancing
→Optimize database queries

## 5. Operational Excellence

→Use Infrastructure as Code
→Implement CI/CD pipelines
→Automate deployments
→Monitor with CloudWatch
→Conduct regular reviews

# Conclusion

AWS provides a comprehensive set of services that enable organizations to build scalable, reliable, and cost-effective infrastructure. By understanding core services, following best practices, and implementing proper architecture patterns, DevOps engineers can leverage AWS to accelerate innovation and deliver value faster.

Whether you're just starting with AWS or looking to optimize existing workloads, continuous learning and hands-on practice are key to mastering the platform.

# Additional Resources

Ready to build on AWS? Start with a simple web application and gradually add complexity as you learn more services!