An AI-based robotic system that detects tomato ripeness and performs automated harvesting using a Fin Ray Effect gripper.
This project focuses on building an intelligent agricultural robot capable of identifying the optimal stage of tomato ripeness and performing gentle harvesting.
Instead of simply detecting "ripe" tomatoes, the system is designed with real-world agricultural considerations, ensuring tomatoes are harvested at the correct stage for transportation and market delivery.
- Tomato ripeness detection using Machine Learning (Teachable Machine)
- Real-time camera-based classification
- ESP32-based control system
- Fin Ray Effect gripper for soft and adaptive gripping
- Automated decision-making (PLUCK / SKIP / SEARCH)
- Serial communication between Python and ESP32
- Python
- OpenCV
- NumPy
- Teachable Machine (for rapid prototyping)
- ESP32 (hardware control)
- Serial Communication (PySerial)
- Camera captures real-time video feed
- ML model classifies tomato as:
- Ripe
- Unripe
- Others
- Based on prediction:
- Ripe → PLUCK
- Unripe → SKIP
- Others → SEARCH
- Command is sent to ESP32
- Fin Ray gripper performs gentle harvesting
The system uses a Fin Ray Effect gripper, which:
- Adapts to the shape of the tomato
- Distributes force evenly
- Reduces pressure points
- Minimizes bruising and damage
- Motor current will be used as a proxy for gripping force
- Threshold-based control to prevent over-compression
- Twist-assisted detachment instead of pulling
- Optional stem-cutting mechanism for safer harvesting
- Multi-stage ripeness classification (Breaker, Turning, Pink, etc.)
- Custom-trained ML model with real farm dataset
- Object detection (YOLO) for multiple tomatoes
- Edge AI deployment on embedded systems
- Force sensor integration for precise control
- Autonomous navigation in farm environments
- Reduces manual labor in agriculture
- Minimizes post-harvest loss
- Improves harvesting efficiency
- Enables precision agriculture