← Innovation & Leadership
IITK · CTO Programme Track AIML · 2025

Machine Learning
Capstone Projects
Portfolio

Three industry-grade AI/ML projects spanning computer vision, time-series forecasting, deep learning, and recommendation systems — built as part of the IIT Kanpur Advanced AI/ML programme.

3
Capstone Projects
6
ML Models
5
Datasets
12
Months Forecast
Python TensorFlow YOLOv8 XGBoost Collaborative Filtering Transfer Learning

Explore Each Capstone

Click a tab to deep-dive into the problem statement, datasets, methodology, code, and outcomes of each project.

Course-end Project 1 of 3

Autonomous Driving Intelligence

A two-part deep learning system for real-time vehicle detection using YOLOv8 and comprehensive autopilot safety analysis of Tesla accident data.

YOLOv8 Transfer Learning Object Detection EDA Tesla Autopilot Analysis PyTorch
2
Project Parts
mAP
Detection Metric
4
Vehicle Classes
300+
Accident Records
8
EDA Charts

Business Scenario

Problem Statement

Autonomous vehicles (AV) and intelligent transport systems (ITS) are the future of road transport. Automatic detection of vehicles on the road in real-time is essential for vehicle tracking, vehicle counting, and road incident response.

Real-time Detection Road Safety ITS

Industry Context

Tesla, Inc. launched its full self-driving (FSD) beta programme in October 2020 in the United States with over 100,000 participants. This project builds the computer vision backbone that such systems rely on, and analyses real-world safety outcomes.

Tesla FSD 100K+ Beta Users 2020 Launch
01
Part One
Vehicle Object Detection using CNN (YOLOv8)
Dataset — Images.zip

Image Format

Traffic surveillance JPEG images captured from overhead cameras. Numbered sequentially from 00000000. Multi-condition: day, night, rain, motion blur.

JPEGSequential

Annotation Format

YOLO-format bounding box labels: class_id, x_center, y_center, width, height (all normalised 0–1). One .txt per image.

YOLO FormatNormalised Coords

Vehicle Classes

Four vehicle categories to detect and classify: Car, Truck, Bus, Motorcycle. Train 70% · Val 20% · Test 10%.

4 Classes70/20/10 Split
YOLOv8 Architecture
Input
640×640px
›
CSPDarkNet Backbone
Pre-trained COCO
›
FPN + PAN
Neck
›
Detection Head
Fine-tuned
›
NMS Output
Boxes + Class
Frozen (COCO ImageNet weights) Trainable (fine-tuned)
Training Workflow
01

Folder Setup & Dataset Extraction

Create project directory structure: AV_Detection/train/, /val/, /test/. Extract Images.zip (large file — plan for compute time). Verify image-label pair integrity for all splits.

02

data.yaml Configuration

Generate YAML config specifying dataset paths, number of classes (nc: 4), and class names. This is the single source of truth for the YOLOv8 training run.

03

Load Pre-trained Weights (Transfer Learning)

Load yolov8s.pt — the YOLOv8 Small model pre-trained on COCO (80 classes, ~11M parameters). The backbone's learned feature extraction transfers directly to custom vehicle classes.

04

Train: 50 Epochs, Early Stopping

Train with Adam optimiser (lr=0.01), batch=16, imgsz=640. Early stopping with patience=10 prevents overfitting. Best checkpoint saved by val mAP@0.5.

05

Evaluate & Run Inference

Report mAP@0.5, mAP@0.5:0.95, Precision, Recall per class. Run inference on 10+ test images — visualise annotated bounding boxes with class labels and confidence scores.

Core Code — YOLOv8 Training
Capstone1_Autonomous_Driving.ipynb
from ultralytics import YOLO import os, zipfile PATH = r"C:\Viswa\IITK\AIML\Datasets" # 1. Extract dataset with zipfile.ZipFile(os.path.join(PATH, "Images.zip")) as z: z.extractall(os.path.join(PATH, "AV_Detection")) # 2. Load YOLOv8s with COCO pre-trained weights (transfer learning) model = YOLO("yolov8s.pt") # 3. Train on custom vehicle dataset results = model.train( data=r"...\AV_Detection\data.yaml", epochs=50, imgsz=640, batch=16, patience=10, # early stopping pretrained=True, optimizer='Adam', device=0 # GPU; use 'cpu' if no GPU ) # 4. Evaluate: mAP@0.5, mAP@0.5:0.95, precision, recall metrics = model.val() print(f"mAP@0.5: {metrics.box.map50:.4f}") # 5. Inference with bounding box visualisation model.predict(source=r"...\test", save=True, conf=0.25)
02
Part Two
Tesla Autopilot Safety Data Analysis
Dataset — Tesla-Deaths.csv
VariableTypeDescription
Case#IntegerUnique accident case identifier
Year / DateDateTemporal accident metadata — parsed to Year, Month, DayOfWeek
Country / StateStringGeographic location — used for map analysis
DeathsIntegerTotal fatalities per accident event
Tesla driverBinaryWhether the Tesla driver was killed in the accident
Tesla OccupantIntegerNumber of Tesla occupant fatalities
Cyclists / PedsIntegerCyclist or pedestrian deaths involved
ModelStringTesla vehicle model: S, 3, X, Y, Cybertruck
Autopilot claimedBinaryWhether autopilot was claimed as a factor
Verified Tesla Autopilot DeathsIntegerConfirmed autopilot-related fatalities ★
EDA Analyses Performed

Temporal

Deaths per year trend, monthly patterns, day-of-week distribution to detect when accidents peak.

Geographic

Country breakdown (USA dominant), US state-level analysis — California leads with most reported events.

Victim Types

Driver deaths, occupant fatalities, cyclist/pedestrian involvement — proportional breakdown across all events.

Autopilot Verified

Verified autopilot deaths vs. total deaths trend over years. Is safety improving? Year-over-year percentage analysis.

Key Insight: Verified autopilot-related deaths represent a subset of total Tesla fatalities. Accident frequency has increased proportionally with growing Tesla fleet size globally — not necessarily indicating declining safety per vehicle. The data requires per-mile normalisation for a fair safety comparison.


Technology Stack
Python 3.10+
Core language for all processing, modelling, and visualisation
Ultralytics YOLOv8
State-of-the-art object detection — architecture, training, inference API
PyTorch 2.x
Deep learning backend powering YOLOv8 training loop
OpenCV
Image loading, preprocessing, bounding box annotation drawing
Pandas / NumPy
Tesla dataset loading, cleaning, EDA aggregations
Matplotlib / Seaborn
8+ statistical charts: bar, pie, line, scatter, heatmap

Submission Checklist
Course-end Project 2 of 3

Sales Forecasting

A complete data science pipeline for Fresh Analytics — merging restaurant sales data, exploring demand patterns through EDA, and forecasting 12 months of future demand using three competing ML models.

Linear Regression Random Forest XGBoost Time Series EDA Feature Engineering 365-Day Forecast
6
Restaurants
3
ML Models
14
Features
365
Day Forecast
RMSE
Evaluation Metric

Business Scenario

Problem Statement

In ever-changing competitive market conditions, there is a need to make correct business decisions and plan for future events. Demand is the most important aspect of a business's ability to achieve its objectives — decisions on production, sales, and staffing all depend on it.

Demand ForecastingDecision Making

Objective

Fresh Analytics aims to predict the daily sales count of items across 6 different restaurants over a 12-month forward horizon, enabling smarter procurement, staffing, and inventory management decisions.

Fresh Analytics6 Restaurants12-Month Horizon
Three Datasets — Merged into One
restaurants.csv

Restaurant Master

6 restaurant identities: Bob's Diner, Beachfront Bar, Sweet Shack, Fou Cher, Corner Cafe, Surfs Up. Columns: id, name.

Dimension Table6 rows
items.csv

Item Catalogue

Menu items with caloric content and unit cost. Columns: id, store_id, name, kcal, cost.

Dimension TableCalorie + Price
sales.csv

Daily Transactions

One row per item per date per restaurant. Columns: date, item_id, price, item_count ★ (target variable).

Fact TableTarget: item_count

Merged Dataset Target: After joining all 3 tables, the unified dataset includes: date · item_id · price · item_count · item_name · kcal · store_id · store_name — all 8 required columns verified.

Exploratory Data Analysis

Overall Sales Trend

Daily sales time series with 30-day rolling average. Reveals macro growth trend and seasonal cycles across all restaurants.

Day-of-Week Patterns

Sales distribution across Mon–Sun. Identifies peak and trough days — weekends typically show higher demand for food service.

Monthly Seasonality

Average daily sales per month across all years. Line chart per year reveals consistent seasonal peaks and troughs.

Quarterly Distribution

Box plots and bar charts of Q1–Q4 sales averaged across years. Identifies strongest and weakest business quarters.

Restaurant Comparison

Total volume, revenue, YoY trends per restaurant. Scatter: does highest-volume store = highest-revenue? (Volume ≠ Revenue.)

Item Popularity

Top 10 items overall + most popular item per restaurant. Most expensive item per restaurant with its calorie count.

Feature Engineering

14 features extracted from the date column to capture all temporal patterns relevant to sales forecasting:

day_of_week day_of_month month quarter year week_of_year is_weekend day_of_year store_id item_id price kcal store_avg_sales item_avg_sales

Time-based split: Last 6 months of data = test set. All earlier data = training set. No random shuffling to prevent data leakage — strict chronological integrity.

ML Model Comparison
Linear Regression

Baseline Model

  • Fully interpretable coefficients
  • Fastest training time
  • Good for linear trend-dominant series
  • Struggles with non-linear seasonality
  • Sets the performance floor (RMSE baseline)
Random Forest

Ensemble Model

  • 200 decision trees via bagging
  • Handles non-linearity automatically
  • Built-in feature importance ranking
  • No feature scaling required
  • Medium RMSE — strong mid-tier
XGBoost 🏆 Best

Gradient Boosting

  • 300 estimators, lr=0.05, depth=6
  • L1/L2 regularisation built in
  • State-of-the-art on tabular data
  • SHAP feature importance support
  • Lowest RMSE — used for 12M forecast
End-to-End Pipeline
01
📂
Load CSVs
3 tables
›
02
🔀
Merge
8 columns
›
03
🔍
EDA
8 charts
›
04
⚙️
Features
14 feats
›
05
✂️
Split
Last 6M test
›
06
🤖
Train 3 Models
RMSE compare
›
07
📅
Forecast
365 days
Core Code — Model Training & Forecast
Capstone2_Sales_Forecasting.ipynb
from sklearn.ensemble import RandomForestRegressor from xgboost import XGBRegressor from sklearn.metrics import mean_squared_error import numpy as np, pandas as pd # Time-based split — NO data leakage split = df['date'].max() - pd.DateOffset(months=6) train, test = df[df.date <= split], df[df.date > split] FEATURES = ['day_of_week','month','quarter','year', 'week_of_year','is_weekend','store_id', 'item_id','price','kcal','store_avg_sales',...] models = { 'Random Forest' : RandomForestRegressor(n_estimators=200, random_state=42), 'XGBoost' : XGBRegressor(n_estimators=300, learning_rate=0.05, max_depth=6, random_state=42) } for name, m in models.items(): m.fit(train[FEATURES], train['item_count']) preds = m.predict(test[FEATURES]) rmse = np.sqrt(mean_squared_error(test['item_count'], preds)) print(f"{name}: RMSE = {rmse:.4f}") # Best model → 365-day forward forecast future = pd.date_range(df.date.max() + pd.Timedelta('1D'), periods=365) # ... generate feature rows for each future date × store × item ... df_forecast['predicted_count'] = np.maximum(0, best_model.predict(X_future))

Submission Checklist
Course-end Project 3 of 3

Preserving Heritage: Enhancing Tourism with AI

Two AI systems in one: a TensorFlow CNN that classifies 11 types of historical architectural structures using transfer learning, and a collaborative filtering recommendation engine that personalises tourist destination suggestions across Indonesian cities.

EfficientNetB0 Transfer Learning 11 Structure Classes SVD Collaborative Filtering Image Augmentation Indonesia Tourism
11
Structure Classes
90%+
Target Val Accuracy
5
Indonesian Cities
3
Tourism Datasets
Top-N
Recommendations

Business Scenario

Problem Statement

There are centuries-old historical structures that preserve a country's and community's history. A government agency uses advanced ML to monitor their condition and report if any need maintenance. XYZ Pvt. Ltd. builds the AI classifier.

Heritage PreservationGovernment Agency

Tourism Objective

Understanding tourists and their expectations is critical for effective marketing. A recommendation engine is an excellent way to supplement existing outreach — guiding visitors to personalised destinations based on community ratings.

Recommender EngineTourism Marketing
01
Part One
Historical Structure Image Classification (TensorFlow)
Dataset — Structures_dataset.zip + dataset_test

altar

Sacred ceremonial platform inside religious buildings — focal point of worship spaces.

apse

Semi-circular recess at the end of a church nave — characteristic of Romanesque architecture.

bell_tower

Freestanding tower housing church bells — civic and religious landmark across Europe.

column

Load-bearing or decorative vertical pillar — hallmark of Classical Greek and Roman architecture.

dome (inner)

Interior view of a domed ceiling — defining feature of Byzantine and Renaissance churches.

dome (outer)

Exterior dome view — creates iconic skyline silhouettes recognisable worldwide.

flying buttress

External arch supporting cathedral walls — a Gothic structural innovation enabling tall naves.

gargoyle

Carved grotesque figure doubling as a water spout — decorative element of Gothic cathedrals.

portal

Ornate main entrance doorway — richly decorated entry point of medieval churches.

stained glass

Coloured glass window art — narrative pictorial art in Gothic and Romanesque churches.

vault

Arched ceiling or roof structure — structural elegance of Romanesque and Gothic naves.

EfficientNetB0 Transfer Learning Architecture
Input
224×224×3
›
EfficientNetB0 Backbone
ImageNet weights — FROZEN
›
GlobalAvgPool
Trainable
›
Dense 512 + Drop 0.4
ReLU
›
Dense 256 + Drop 0.3
ReLU
›
Softmax
11 classes
Frozen (ImageNet pre-trained — not updated) Trainable (fine-tuned on structures dataset)
Training Workflow
01

Plot Sample Images (OpenCV)

Display 8–10 images per class using cv2.imread() and Matplotlib grid. Inspect class balance, image variation, and visual complexity before training.

02

Load EfficientNetB0 Backbone

EfficientNetB0(include_top=False, weights='imagenet') — loads all convolutional feature extraction layers without ImageNet classifier head.

03

Freeze Backbone, Build Custom Head

Set base_model.trainable = False. Stack: GlobalAveragePooling2D → Dense(512, relu) → Dropout(0.4) → Dense(256, relu) → Dropout(0.3) → Dense(11, softmax).

04

Custom StopAtAccuracy Callback

Callback halts training as soon as val_accuracy ≥ 90% — prevents unnecessary epochs and reduces overfitting after the target is met.

05

Train Without Augmentation (Baseline)

Initial training run with no augmentation. Establishes baseline accuracy and shows the raw learning curve — often reveals early overfitting.

06

Train With Augmentation

ImageDataGenerator with: rotation_range=20°, width/height_shift=0.2, horizontal_flip=True, zoom_range=0.2, shear_range=0.1, brightness_range=[0.8,1.2]. Improves generalisation.

07

Plot Accuracy & Loss Curves

Training vs validation accuracy per epoch. Loss curves. Identify overfitting onset — the epoch where train-val accuracy gap exceeds ~10%. Compare augmented vs non-augmented.

Core Code — TensorFlow Transfer Learning
Capstone3_Heritage_Tourism_AI.ipynb — Part 1
import tensorflow as tf from tensorflow.keras.applications import EfficientNetB0 from tensorflow.keras import layers, Model # Load pre-trained backbone — FROZEN base = EfficientNetB0(include_top=False, weights='imagenet', input_shape=(224,224,3)) base.trainable = False # Build custom 11-class head x = layers.GlobalAveragePooling2D()(base.output) x = layers.Dense(512, activation='relu')(x) x = layers.Dropout(0.4)(x) x = layers.Dense(256, activation='relu')(x) x = layers.Dropout(0.3)(x) out = layers.Dense(11, activation='softmax')(x) model = Model(base.input, out) model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) # Custom callback: stop at val_accuracy >= 90% class StopAtAccuracy(tf.keras.callbacks.Callback): def on_epoch_end(self, epoch, logs=None): if logs.get('val_accuracy', 0) >= 0.90: self.model.stop_training = True # Train WITH augmentation from tensorflow.keras.preprocessing.image import ImageDataGenerator aug = ImageDataGenerator(rescale=1/255, rotation_range=20, horizontal_flip=True, zoom_range=0.2, width_shift_range=0.2) history = model.fit(aug.flow_from_directory(TRAIN_DIR, target_size=(224,224), batch_size=32, class_mode='categorical'), validation_data=test_gen, epochs=50, callbacks=[StopAtAccuracy()])
02
Part Two
Tourist Recommendation Engine — Indonesia
Three Tourism Datasets
user.csv

User Demographics

ColumnDescription
User_IdUnique visitor ID
LocationHome city of user
AgeVisitor age
tourisim_with_id.csv

Tourist Places

ColumnDescription
Place_IdUnique place ID
Place_NameName of attraction
CategoryTourism category
CityCity location
PriceEntry price (IDR)
RatingAverage rating
tourism_rating.csv

User-Place Ratings

ColumnDescription
User_IdFK → user.csv
Place_IdFK → places
Place_Ratings1–5 star rating ★
Rating MatrixCF Input
EDA Questions Answered

User Age Distribution

Age histogram of tourists who rate places. Most active age groups and home cities identified.

Category per City

Stacked bar: which categories dominate each of the 5 cities? Budaya, Taman Hiburan, Cagar Alam, Bahari, Tempat Ibadah.

Best City for Nature

Which Indonesian city has the most Cagar Alam + Bahari spots? Ideal for nature enthusiast tourists.

Most Loved Spots

Top-rated places by average user score (min 5 ratings). Which city has the most beloved attractions?

Category Preference

Which tourism category do users rate highest on average? Cultural vs nature vs amusement parks.

SVD Recommender

Matrix factorisation (50 latent factors). 5-fold cross-validation. RMSE + MAE reported. Top-N unvisited places per user.

Core Code — SVD Collaborative Filtering
Capstone3_Heritage_Tourism_AI.ipynb — Part 2
from surprise import SVD, Dataset, Reader, accuracy from surprise.model_selection import cross_validate, train_test_split # Build Surprise dataset from ratings CSV reader = Reader(rating_scale=(1, 5)) data = Dataset.load_from_df( ratings[['User_Id', 'Place_Id', 'Place_Ratings']], reader) trainset, testset = train_test_split(data, test_size=0.2, random_state=42) # SVD — matrix factorisation with 50 latent factors svd = SVD(n_factors=50, n_epochs=30, lr_all=0.005, reg_all=0.02) svd.fit(trainset) preds = svd.test(testset) print(f"RMSE: {accuracy.rmse(preds):.4f}") # Recommend top-5 unvisited places for any user def recommend_places(user_id, n=5): visited = ratings[ratings.User_Id == user_id].Place_Id.tolist() candidates = [p for p in places.Place_Id.unique() if p not in visited] preds_list = [(p, svd.predict(user_id, p).est) for p in candidates] top_n = sorted(preds_list, key=lambda x: x[1], reverse=True)[:n] return places[places.Place_Id.isin([p[0] for p in top_n])] recommend_places(user_id=1, n=5)

Submission Checklist

Next Steps — Cloud Production Deployment

Deployment Architecture

The three AI models will be deployed as serverless endpoints on public cloud infrastructure, accessible via a unified REST API integrated into agent.viswanext.com/aiml.html.

Target URL

The live portfolio and model playground will be published at https://agent.viswanext.com/aiml.html — publicly accessible for demonstrations and learning.

🪣

Step 1 — Object storage: datasets & models

Upload all datasets (restaurants.csv, items.csv, sales.csv, Tesla-Deaths.csv, Structures images), trained model weights (.pt, .h5), and result CSVs to object-storage buckets. Organise into prefixes by capstone.

Object storageIAMObject versioning
⚡

Step 2 — Serverless inference functions

Package each model's inference logic as a serverless function: (1) vehicle detection endpoint, (2) sales forecast endpoint, (3) structure classifier + tourism recommender endpoint. Use container images for serverless functions for large model files.

Serverless functionsContainer registryFunction layers
🔌

Step 3 — API Gateway: REST Endpoints

Expose the functions as REST API endpoints. Implement CORS for browser requests from agent.viswanext.com. Add API key authentication and rate limiting.

API gatewayCORSAPI Keys
🌐

Step 4 — Global CDN

Serve the static portfolio HTML through a CDN for low-latency global access. Cache model results and static assets at edge locations worldwide.

CDNDNSTLS certificate
🤖

Step 5 — agent.viswanext.com/aiml.html

Publish this portfolio page + an interactive model playground. Users can upload images for vehicle detection and structure classification, input restaurant IDs for sales forecasts, and get tourist recommendations — all powered by live serverless endpoints.

Static site hostingCDNCustom Domain
Personal IIT Kanpur CTO-program learning notes shared by Viswa · not an official IIT Kanpur site · not linked to any employer or company · company and product names intentionally left out.