DrishtiStick: AI-Powered Smart Cane — Fusion Innovation of IoT and Machine Learning

DrishtiStick is an AI-enabled smart cane system designed for visually impaired individuals and the elderly. It integrates ESP32 microcontroller, YOLOv8 object detection, GPS positioning, and cloud monitoring to provide comprehensive environmental perception and safety protection. Key features include obstacle detection, fall detection, real-time location tracking, remote cloud monitoring, and AI-powered visual navigation with voice guidance.

Global visually impaired population exceeds 250 million, facing significant daily challenges. Traditional aids like white canes and guide dogs offer basic support but have limitations in complex urban environments. The maturity of IoT, AI, and edge computing technologies presents an opportunity to redefine assistive tech, leading to innovations like DrishtiStick.

System Components

• Main Control Unit: ESP32 microcontroller coordinating sensor data (GPS, MPU6050, ultrasonic) and initial processing.
• Visual Unit: ESP32-CAM with OV2640 camera for video streaming and image capture.
• Cloud Platform: Next.js-based web dashboard for real-time data visualization and remote monitoring.
• Mobile App: Android app for push notifications and offline data access.

Key Innovations

• YOLOv8 Visual Navigation: Wireless video streaming to PC for real-time object detection, generating voice navigation commands.
• Sensor Fusion: Complementary filtering of MPU6050 (accelerometer/gyroscope) and ultrasonic data for accurate posture estimation and obstacle detection.
• Edge-Cloud Collaboration: Local edge processing for real-time safety functions (e.g., obstacle alerts) and cloud for complex AI inference (YOLO) and data storage.

Hardware

• Circuit Connections: ESP32 pins for ultrasonic (GPIO18/19), MPU6050 (I2C GPIO21/22), GPS (UART1 GPIO16/17), ESP32-CAM (UART2 GPIO13/14).
• Power Management: Lithium battery-powered, with 4-6 hours continuous use and 24+ hours standby.

Software

• Firmware: Arduino framework with libraries like TinyGPS++, ArduinoJson, ESP32 Camera.
• Web App: Next.js deployment steps: npm install, npm run build, npm start (Firebase real-time database integration).
• Mobile App: Android Studio build for SDK 36+ with Material Design 3 and Retrofit network library.

Use Cases

• Visually Impaired Independent Travel: Ultrasonic obstacle detection and YOLO-based visual navigation with voice guidance.
• Elderly Fall Prevention: MPU6050-based fall detection triggering automatic alerts to caregivers.
• Remote Monitoring: Caregivers use web/mobile apps to track location, receive alerts, and view activity data.

Social Value

• Provides unprecedented independence and safety for users.
• Open-source (MIT license) and low-cost (hundreds of yuan) makes it accessible to resource-limited communities.

1. WiFi Coverage: Use mobile hotspots as backup; local cache for network-independent functions.
2. YOLO Real-Time Performance: Optimize video encoding, use lightweight YOLOv8n model, and edge-triggered navigation.
3. Environment Adaptability: Image preprocessing (contrast adjustment, noise reduction), multi-sensor fusion, and continuous model optimization via user feedback.

Short-Term

• Enhanced voice feedback and diverse vibration modes for obstacle types.
• Battery monitoring and low-power alerts.

Mid-Term

• ML-based fall prediction (not just detection).
• Multi-user support for caregivers.
• Offline data caching.

Long-Term

• SMS/phone emergency notifications.
• Smart home system integration.
• Personalized health suggestions based on usage data.

DrishtiStick demonstrates the power of tech for social good, merging IoT and AI to improve quality of life for visually impaired and elderly users. Its open-source nature and low cost make it accessible globally, especially in developing regions. As AI and hardware costs evolve, such innovations will continue to drive inclusive assistive tech.