Promote Sign in Sign up

Repository

Virtual-Self-Driving-Car-AI-using-Kivy-and-Reinforcement-Learning

πŸš— Virtual Self-Driving Car AI A self-driving car simulation using Deep Q-Learning and Kivy β€” the agent learns to navigate a 2D map while avoiding user-drawn obstacles in real time. Β· License MIT Β· Install via pip Β· Maintenance unknown

stabgan
Repository
License
MIT
Component
Detail
Network
input(5) β†’ fc(30, ReLU) β†’ output(3)
Replay buffer
100,000 transitions, batch size 100

Specs

  • ComponentDetail
  • Networkinput(5) β†’ fc(30, ReLU) β†’ output(3)
  • Replay buffer100,000 transitions, batch size 100
  • OptimizerAdam, lr = 0.001
  • LossSmooth L1 (Huber)
  • Ξ³ (discount)0.9
  • ExplorationSoftmax with temperature T=100
  • InputDescription
  • signal1Sand density around front sensor (20Γ—20 px window)
  • signal2Sand density around left sensor (+30Β°)
  • signal3Sand density around right sensor (βˆ’30Β°)
  • orientationAngle between velocity and goal direction ∈ [-1, 1]
  • -orientationNegated orientation (symmetry signal)
  • ActionRotation
  • 00Β° (straight)
  • 1+20Β° (left)
  • 2βˆ’20Β° (right)
  • ConditionReward
  • On sandβˆ’1.0
  • On road, moving awayβˆ’0.2
  • On road, approaching goal+0.1
  • Hitting map edgeβˆ’1.0
  • TechnologyPurpose
  • 🧠PyTorch
  • πŸ–ΌοΈKivy
  • πŸ”’NumPy
  • πŸ“ŠMatplotlib
  • 🐍Python 3.8+
  • Original (broken)Fixed
  • from torch.autograd import VariableRemoved β€” raw tensors used directly
  • Variable(state, volatile=True)torch.no_grad() context manager
  • F.softmax(x * 100) (no dim)F.softmax(x * 100, dim=1)
  • probs.multinomial()probs.multinomial(num_samples=1)
  • action.data[0, 0] (returns tensor)action.item() (returns Python int)
  • retain_variables=Trueretain_graph=True
  • Variable(torch.cat(x, 0))torch.cat(x, 0)
  • torch.Tensor(input_size) (uninitialized)torch.zeros(input_size)
  • torch.Tensor([val])torch.tensor([val], dtype=...)
  • Hardcoded 'last_brain.pth' pathos.path.join(os.path.dirname(__file__), ...)
  • Sensor reads with no bounds checking_read_sensor() helper with clamped indices

README

πŸš— Virtual Self-Driving Car AI

A self-driving car simulation using Deep Q-Learning and Kivy β€” the agent learns to navigate a 2D map while avoiding user-drawn obstacles in real time.

demo
demo2

What It Does

A small car navigates a 2D canvas, learning in real time to drive between two goal points (top-left ↔ bottom-right). Draw "sand" obstacles on the map with your mouse β€” the car learns to avoid them using reinforcement learning. The brain can be saved and loaded between sessions.

Architecture

Deep Q-Network (DQN)

Component Detail
Network input(5) β†’ fc(30, ReLU) β†’ output(3)
Replay buffer 100,000 transitions, batch size 100
Optimizer Adam, lr = 0.001
Loss Smooth L1 (Huber)
Ξ³ (discount) 0.9
Exploration Softmax with temperature T=100

State Space (5 inputs)

Input Description
signal1 Sand density around front sensor (20Γ—20 px window)
signal2 Sand density around left sensor (+30Β°)
signal3 Sand density around right sensor (βˆ’30Β°)
orientation Angle between velocity and goal direction ∈ [-1, 1]
-orientation Negated orientation (symmetry signal)

Action Space

Action Rotation
0 0Β° (straight)
1 +20Β° (left)
2 βˆ’20Β° (right)

Rewards

Condition Reward
On sand βˆ’1.0
On road, moving away βˆ’0.2
On road, approaching goal +0.1
Hitting map edge βˆ’1.0

Goal flips to the opposite corner when the car gets within 100 px.

Project Structure

Self_Driving_Car/
β”œβ”€β”€ ai.py              # DQN agent: network, replay memory, training loop
β”œβ”€β”€ map_commented.py   # Kivy app: car physics, sensors, game loop, rewards
└── car.kv             # Kivy layout: car and sensor widget definitions

πŸ›  Tech Stack

Technology Purpose
🧠 PyTorch Deep Q-Network
πŸ–ΌοΈ Kivy 2D simulation GUI
πŸ”’ NumPy Sensor signal processing
πŸ“Š Matplotlib Score plotting
🐍 Python 3.8+ Runtime

Getting Started

Install dependencies

pip install kivy torch numpy matplotlib

Run

cd Self_Driving_Car
python map_commented.py

Controls

  • Draw obstacles β€” click and drag to paint sand
  • Clear β€” remove all sand
  • Save β€” save brain to last_brain.pth + plot scores
  • Load β€” restore a previously saved brain

Modernization Changelog

Fixes applied to bring the codebase up to date with current PyTorch (2.x):

Original (broken) Fixed
from torch.autograd import Variable Removed β€” raw tensors used directly
Variable(state, volatile=True) torch.no_grad() context manager
F.softmax(x * 100) (no dim) F.softmax(x * 100, dim=1)
probs.multinomial() probs.multinomial(num_samples=1)
action.data[0, 0] (returns tensor) action.item() (returns Python int)
retain_variables=True retain_graph=True
Variable(torch.cat(x, 0)) torch.cat(x, 0)
torch.Tensor(input_size) (uninitialized) torch.zeros(input_size)
torch.Tensor([val]) torch.tensor([val], dtype=...)
Hardcoded 'last_brain.pth' path os.path.join(os.path.dirname(__file__), ...)
Sensor reads with no bounds checking _read_sensor() helper with clamped indices
Sand painting with negative indices Clamped to valid array range

⚠️ Known Issues

  • The simulation relies on global state, making it fragile for extension but functional as a learning project.
  • No GPU support configured β€” runs on CPU only (sufficient for this network size).
  • Button positions are hardcoded relative to initial widget size and may overlap on small windows.

License

MIT β€” Kaustabh Ganguly, 2018