End-to-End DevOps Project: Complete Implementation
· Pavan Y N · devops, project, end-to-end, implementation
Published: October 30, 2025
Reading Time: 28 minutes
Tags: DevOps, Project, End-to-End, Implementation, Microservices
# Introduction
This comprehensive guide walks through building a complete end-to-end DevOps project, from application development to production deployment. We'll implement a microservices application with full CI/CD, infrastructure as code, monitoring, logging, and security best practices.
# Project Overview
## Application Architecture
code
┌─────────────────────────────────────────────────────────┐
│ Load Balancer (ALB) │
└────────────────────┬────────────────────────────────────┘
│
┌────────────┴────────────┐
│ │
┌───────▼────────┐ ┌────────▼────────┐
│ Frontend │ │ API Gateway │
│ (React) │ │ (Node.js) │
└────────────────┘ └────────┬────────┘
│
┌────────────┼────────────┐
│ │ │
┌───────▼──┐ ┌─────▼───┐ ┌────▼─────┐
│ User │ │ Product │ │ Order │
│ Service │ │ Service │ │ Service │
└────┬─────┘ └────┬────┘ └────┬─────┘
│ │ │
┌────▼─────┐ ┌────▼────┐ ┌───▼──────┐
│ PostgreSQL│ │ MongoDB │ │ PostgreSQL│
└──────────┘ └─────────┘ └──────────┘
## Technology Stack
- →Frontend: React, TypeScript, Tailwind CSS
- →Backend: Node.js, Express, TypeScript
- →Databases: PostgreSQL, MongoDB, Redis
- →Container: Docker, Docker Compose
- →Orchestration: Kubernetes (EKS)
- →CI/CD: GitHub Actions, ArgoCD
- →Infrastructure: Terraform, AWS
- →Monitoring: Prometheus, Grafana
- →Logging: ELK Stack (Elasticsearch, Logstash, Kibana)
- →Security: Trivy, SonarQube, AWS Secrets Manager
# Phase 1: Application Development
## Microservice Structure
code
user-service/
├── src/
│ ├── controllers/
│ ├── models/
│ ├── routes/
│ ├── middleware/
│ ├── utils/
│ └── app.ts
├── tests/
├── Dockerfile
├── package.json
└── tsconfig.json
## User Service Example
src/app.ts:
code
import express from 'express';
import { createClient } from 'redis';
import { Pool } from 'pg';
import promClient from 'prom-client';
const app = express();
const register = new promClient.Registry();
// Metrics
const httpRequestDuration = new promClient.Histogram({
name: 'http_request_duration_seconds',
help: 'Duration of HTTP requests in seconds',
labelNames: ['method', 'route', 'status_code'],
buckets: [0.1, 0.5, 1, 2, 5]
});
const httpRequestTotal = new promClient.Counter({
name: 'http_requests_total',
help: 'Total number of HTTP requests',
labelNames: ['method', 'route', 'status_code']
});
register.registerMetric(httpRequestDuration);
register.registerMetric(httpRequestTotal);
promClient.collectDefaultMetrics({ register });
// Database connection
const pool = new Pool({
host: process.env.DB_HOST,
port: parseInt(process.env.DB_PORT || '5432'),
database: process.env.DB_NAME,
user: process.env.DB_USER,
password: process.env.DB_PASSWORD,
max: 20,
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 2000,
});
// Redis connection
const redis = createClient({
url: process.env.REDIS_URL
});
redis.on('error', (err) => console.error('Redis error:', err));
redis.connect();
// Middleware
app.use(express.json());
app.use((req, res, next) => {
const start = Date.now();
res.on('finish', () => {
const duration = (Date.now() - start) / 1000;
const labels = {
method: req.method,
route: req.route?.path || req.path,
status_code: res.statusCode
};
httpRequestDuration.observe(labels, duration);
httpRequestTotal.inc(labels);
});
next();
});
// Routes
app.get('/health', (req, res) => {
res.json({ status: 'healthy', timestamp: new Date().toISOString() });
});
app.get('/metrics', async (req, res) => {
res.set('Content-Type', register.contentType);
res.end(await register.metrics());
});
app.get('/users', async (req, res) => {
try {
// Check cache
const cached = await redis.get('users:all');
if (cached) {
return res.json(JSON.parse(cached));
}
// Query database
const result = await pool.query('SELECT * FROM users');
// Cache result
await redis.setEx('users:all', 300, JSON.stringify(result.rows));
res.json(result.rows);
} catch (error) {
console.error('Error fetching users:', error);
res.status(500).json({ error: 'Internal server error' });
}
});
app.post('/users', async (req, res) => {
try {
const { name, email } = req.body;
const result = await pool.query(
'INSERT INTO users (name, email) VALUES ($1, $2) RETURNING *',
[name, email]
);
// Invalidate cache
await redis.del('users:all');
res.status(201).json(result.rows[0]);
} catch (error) {
console.error('Error creating user:', error);
res.status(500).json({ error: 'Internal server error' });
}
});
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log(`User service listening on port ${PORT}`);
});
## Dockerfile (Multi-Stage)
code
# Build stage
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY tsconfig.json ./
COPY src ./src
RUN npm run build
# Production stage
FROM node:20-alpine
WORKDIR /app
RUN addgroup -g 1001 -S nodejs && \
adduser -S nodejs -u 1001
COPY package*.json ./
RUN npm ci --only=production && \
npm cache clean --force
COPY --from=builder --chown=nodejs:nodejs /app/dist ./dist
USER nodejs
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD node -e "require('http').get('http://localhost:3000/health', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})"
CMD ["node", "dist/app.js"]
# Phase 2: Infrastructure as Code
## Terraform Configuration
main.tf:
code
# VPC
module "vpc" {
source = "./modules/vpc"
name = "production-vpc"
vpc_cidr = "10.0.0.0/16"
az_count = 3
enable_nat_gateway = true
single_nat_gateway = false
create_database_subnets = true
enable_flow_logs = true
tags = local.common_tags
}
# EKS Cluster
module "eks" {
source = "terraform-aws-modules/eks/aws"
cluster_name = "production-eks"
cluster_version = "1.28"
vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnet_ids
cluster_endpoint_public_access = true
eks_managed_node_groups = {
general = {
desired_size = 3
min_size = 2
max_size = 10
instance_types = ["t3.large"]
capacity_type = "ON_DEMAND"
labels = {
role = "general"
}
tags = local.common_tags
}
spot = {
desired_size = 2
min_size = 0
max_size = 5
instance_types = ["t3.large", "t3a.large"]
capacity_type = "SPOT"
labels = {
role = "spot"
}
taints = [{
key = "spot"
value = "true"
effect = "NoSchedule"
}]
tags = local.common_tags
}
}
tags = local.common_tags
}
# RDS PostgreSQL
module "rds" {
source = "./modules/rds"
identifier = "production-db"
engine = "postgres"
engine_version = "15.3"
instance_class = "db.t3.medium"
allocated_storage = 100
max_allocated_storage = 500
db_name = "appdb"
username = "admin"
password = random_password.db_password.result
vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.database_subnet_ids
security_group_ids = [aws_security_group.rds.id]
backup_retention_period = 7
backup_window = "03:00-04:00"
maintenance_window = "Mon:04:00-Mon:05:00"
enabled_cloudwatch_logs_exports = ["postgresql", "upgrade"]
tags = local.common_tags
}
# ElastiCache Redis
resource "aws_elasticache_cluster" "redis" {
cluster_id = "production-redis"
engine = "redis"
node_type = "cache.t3.medium"
num_cache_nodes = 1
parameter_group_name = "default.redis7"
engine_version = "7.0"
port = 6379
subnet_group_name = aws_elasticache_subnet_group.redis.name
security_group_ids = [aws_security_group.redis.id]
tags = local.common_tags
}
# Application Load Balancer
resource "aws_lb" "main" {
name = "production-alb"
internal = false
load_balancer_type = "application"
security_groups = [aws_security_group.alb.id]
subnets = module.vpc.public_subnet_ids
enable_deletion_protection = true
enable_http2 = true
enable_cross_zone_load_balancing = true
tags = local.common_tags
}
# Secrets Manager
resource "aws_secretsmanager_secret" "app_secrets" {
name = "production/app-secrets"
tags = local.common_tags
}
resource "aws_secretsmanager_secret_version" "app_secrets" {
secret_id = aws_secretsmanager_secret.app_secrets.id
secret_string = jsonencode({
database_url = "postgresql://${module.rds.endpoint}/${module.rds.db_name}"
redis_url = "redis://${aws_elasticache_cluster.redis.cache_nodes[0].address}:6379"
jwt_secret = random_password.jwt_secret.result
})
}
# Phase 3: Kubernetes Deployment
## Kubernetes Manifests
k8s/user-service/deployment.yaml:
code
apiVersion: apps/v1
kind: Deployment
metadata:
name: user-service
namespace: production
labels:
app: user-service
version: v1
spec:
replicas: 3
selector:
matchLabels:
app: user-service
template:
metadata:
labels:
app: user-service
version: v1
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "3000"
prometheus.io/path: "/metrics"
spec:
serviceAccountName: user-service
containers:
- name: user-service
image: 123456789012.dkr.ecr.us-east-1.amazonaws.com/user-service:latest
ports:
- containerPort: 3000
name: http
env:
- name: NODE_ENV
value: "production"
- name: PORT
value: "3000"
- name: DB_HOST
valueFrom:
secretKeyRef:
name: database-credentials
key: host
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: database-credentials
key: password
- name: REDIS_URL
valueFrom:
secretKeyRef:
name: redis-credentials
key: url
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 512Mi
livenessProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- user-service
topologyKey: kubernetes.io/hostname
---
apiVersion: v1
kind: Service
metadata:
name: user-service
namespace: production
labels:
app: user-service
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 3000
protocol: TCP
name: http
selector:
app: user-service
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: user-service
namespace: production
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: user-service
minReplicas: 3
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80
behavior:
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 50
periodSeconds: 60
scaleUp:
stabilizationWindowSeconds: 0
policies:
- type: Percent
value: 100
periodSeconds: 15
- type: Pods
value: 4
periodSeconds: 15
selectPolicy: Max
# Phase 4: CI/CD Pipeline
## GitHub Actions Workflow
.github/workflows/deploy.yml:
code
name: CI/CD Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
env:
AWS_REGION: us-east-1
ECR_REGISTRY: 123456789012.dkr.ecr.us-east-1.amazonaws.com
EKS_CLUSTER: production-eks
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run linter
run: npm run lint
- name: Run tests
run: npm test -- --coverage
- name: Upload coverage
uses: codecov/codecov-action@v3
with:
files: ./coverage/lcov.info
security-scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
scan-type: 'fs'
scan-ref: '.'
severity: 'CRITICAL,HIGH'
- name: SonarQube Scan
uses: sonarsource/sonarqube-scan-action@master
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
build-and-push:
needs: [test, security-scan]
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
outputs:
image-tag: ${{ steps.meta.outputs.tags }}
steps:
- uses: actions/checkout@v4
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ env.AWS_REGION }}
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.ECR_REGISTRY }}/user-service
tags: |
type=ref,event=branch
type=sha,prefix={{branch}}-
type=semver,pattern={{version}}
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan image with Trivy
uses: aquasecurity/trivy-action@master
with:
image-ref: ${{ steps.meta.outputs.tags }}
format: 'sarif'
output: 'trivy-results.sarif'
- name: Upload Trivy results
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: 'trivy-results.sarif'
deploy:
needs: build-and-push
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Update Kubernetes manifests
run: |
cd k8s/user-service
kustomize edit set image \
user-service=${{ needs.build-and-push.outputs.image-tag }}
- name: Commit and push
run: |
git config user.name "GitHub Actions"
git config user.email "actions@github.com"
git add k8s/
git commit -m "Update image to ${{ needs.build-and-push.outputs.image-tag }}"
git push
- name: Notify Slack
uses: 8398a7/action-slack@v3
with:
status: ${{ job.status }}
text: 'Deployment completed'
webhook_url: ${{ secrets.SLACK_WEBHOOK }}
# Phase 5: Monitoring and Logging
## Prometheus Configuration
prometheus/values.yaml:
code
prometheus:
prometheusSpec:
retention: 30d
storageSpec:
volumeClaimTemplate:
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 50Gi
additionalScrapeConfigs:
- job_name: 'kubernetes-pods'
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: true
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
target_label: __address__
alertmanager:
config:
global:
resolve_timeout: 5m
route:
group_by: ['alertname', 'cluster', 'service']
group_wait: 10s
group_interval: 10s
repeat_interval: 12h
receiver: 'slack'
receivers:
- name: 'slack'
slack_configs:
- api_url: '{{ .Values.slackWebhook }}'
channel: '#alerts'
title: 'Alert: {{ .GroupLabels.alertname }}'
# STAR Approach: Complete Implementation
Situation: Startup needed to build production-ready infrastructure from scratch with limited DevOps resources.
Task: Implement complete end-to-end DevOps solution including CI/CD, infrastructure, monitoring, and security.
Action:
- •
Application Development:
- →Built microservices architecture
- →Implemented health checks and metrics
- →Added comprehensive logging
- →Created Docker images
- •
Infrastructure Setup:
- →Provisioned AWS infrastructure with Terraform
- →Deployed EKS cluster
- →Set up RDS and ElastiCache
- →Configured networking and security
- •
CI/CD Implementation:
- →Created GitHub Actions workflows
- →Implemented automated testing
- →Added security scanning
- →Set up ArgoCD for GitOps
- •
Monitoring & Logging:
- →Deployed Prometheus and Grafana
- →Set up ELK stack
- →Created dashboards
- →Configured alerting
- •
Security:
- →Implemented secrets management
- →Added vulnerability scanning
- →Configured network policies
- →Set up RBAC
Result:
- →Zero-downtime deployments achieved
- →Deployment time reduced from hours to 10 minutes
- →Infrastructure provisioning automated (30 minutes vs 2 weeks)
- →99.9% uptime achieved
- →Mean time to detect issues: 2 minutes
- →Mean time to recovery: 15 minutes
- →Cost optimized by 40% through auto-scaling
# Conclusion
Building an end-to-end DevOps project requires careful planning, implementation of best practices, and integration of multiple tools and technologies. By following this guide, you can create a production-ready infrastructure that is scalable, secure, and maintainable.
# Additional Resources
- →Kubernetes Documentation
- →Terraform AWS Provider
- →GitHub Actions Documentation
- →Prometheus Documentation
Congratulations on completing the 20-day DevOps journey! You now have the knowledge to build and manage production-grade infrastructure!