diff --git a/project/README.md b/project/README.md index ddb63a38c71296de82d23f34c317019ab62654a9..455a5a07c5364f45399d4744a833a959072f4862 100644 --- a/project/README.md +++ b/project/README.md @@ -4,30 +4,64 @@ ## Description & Motivation -This folder contains the work for our final project of the EML Proseminar. The project is about classifying different types of fruit based on their images (:arrow_right: **multi-class image classification**) +This folder contains the work for our final project of the EML Proseminar. The project is about classifying different types of fruit based on their images (âž¡ï¸ **multi-class image classification**) -:arrow_right: Given any picture, we want to make the correct prediction: *Is it a banana, an apple, a strawberry, ...?* :banana: :apple: :strawberry: +âž¡ï¸ Given any picture, we want to make the correct prediction: *Is it a banana, an apple, a strawberry, ...?* 🌠🎠📠-The motivation for this project is to explore the capabilities of machine learning models for image classification tasks. The inspiration for this project comes from the idea of automatic recognition of fruits, which has a practical application in for instance supermarkets. Customers currently have to manually select the type of fruit they are purchasing at self-service scales. This process is time consuming and also inconvenient. This could be automated, made more efficient and provide a better customer experience, particularly in busy shopping environments. (of course, this can be scaled and customized to all the other groceries that have to be manually selected such as bread or vegetables and also to other areas where things need to be recognize such as material in recycling plants or products in warehouses) +The motivation for this project is to explore the capabilities of machine learning models for image classification tasks. The inspiration for this project comes from the idea of automatic fruit recognition, which has **practical applications** for instance in supermarkets. Currently, customers have to manually select the type of fruit they are purchasing at self-service scales. This process is time-consuming and also inconvenient. Automating this process not only enhances efficiency but also provides a better customer experience, particularly in busy shopping environments. + +Moreover, the scalability of this approach extends beyond just fruits; it could be adapted for other grocery items like bread or vegetables, as well as various industries requiring object recognition, such as material sorting in recycling plants or inventory management in warehouses. ## Related Work -In our project, we lokked at some related works: the bachelor's thesis that created the dataset we use, essential for understanding its structure and development, and a study on fruit recognition using deep learning, providing advanced insights into neural network applications in fruit classification. +In our project, we looked related work on (fruit) image classification to gain insights into the best practices and build upon the existing knowledge in the field. We found two papers that were particularly relevant to our project: + +- the bachelor's thesis that created the dataset we use, which is essential for understanding its structure and development, and +- a study on fruit recognition using deep learning, providing advanced insights into neural network applications in fruit classification. -**1. Creating a Dataset and Models Based on Convolutional Neural Networks to Improve Fruit Classification** +**1. Creating a Dataset and Models Based on Convolutional Neural Networks to Improve Fruit Classification** (MinuÈ› & Iftene, 2021) -The dataset we are utilizing for our fruit image classification project was originally created as part of a bachelor's thesis, detailed in the paper "Creating a Dataset and Models Based on Convolutional Neural Networks to Improve Fruit Classification"​​. This work involved compiling a diverse and extensive collection of fruit images from various sources, which were then meticulously processed to make them suitable for machine learning applications. The paper not only discusses the dataset creation but also delves into the development of Convolutional Neural Network (CNN) models specifically tailored for the task of fruit classification. The efforts in dataset preparation, including image filtering and resizing, and the insights into CNN model training provided in this thesis, form the backbone of the dataset we are using in our project. +The dataset we are utilizing for our fruit image classification project was originally created as part of a bachelor's thesis, detailed in the paper "Creating a Dataset and Models Based on Convolutional Neural Networks to Improve Fruit Classification"​​. This work involved compiling a diverse and extensive collection of fruit images from various sources, which were then meticulously processed to make them suitable for machine learning applications. The paper not only discusses the dataset creation but also delves into the development of Convolutional Neural Network (CNN) models specifically tailored for the task of fruit classification. The efforts in dataset preparation, including image filtering and resizing, and the insights into CNN model training provided in this thesis, form the backbone of the dataset we are using in our project. Another valuable finding of this work is that using RGB, HSV and grayscale features in combination can improve the models' performance. -[Read more about the work here](https://ieeexplore.ieee.org/document/9700237). +â†ªï¸ [Read more about the work here](https://ieeexplore.ieee.org/document/9700237). -**2. Fruit Recognition from Images Using Deep Learning** +**2. Fruit Recognition from Images Using Deep Learning** (Muresan & Oltean, 2018) -This paper, available on Arxiv, explores fruit recognition using deep learning techniques. The authors developed a deep learning-based system for classifying fruits in images, addressing the challenges posed by variations in fruit appearance and environmental conditions. Their work on neural network model construction and optimization offers valuable insights for projects like ours, showcasing the effectiveness of deep learning in distinguishing between different fruit types. +This paper explores fruit recognition using deep learning techniques. The authors developed a deep learning-based system for classifying fruits in images, addressing the challenges posed by variations in fruit appearance and environmental conditions. Their work on neural network model construction and optimization offers valuable insights for projects like ours, showcasing the effectiveness of deep learning in distinguishing between different fruit types. -[Read the full paper here](https://arxiv.org/pdf/1712.00580.pdf). +â†ªï¸ [Read the full paper here](https://arxiv.org/pdf/1712.00580.pdf). ## Data +### Overview + +The dataset we are using for our project is based on an **fruit image dataset**, available on [Kaggle](https://www.kaggle.com/datasets/aelchimminut/fruits262). Here are some key details about this dataset: +- 225,640 images +- 262 types of fruits +- ~ 860 images per class +- format: *jpg* +- images have different sizes +- contains subfolders for each class with the class name as folder name + +### Insights + +The dataset is much cleaner than other fruit datasets we found on Kaggle, which often contain images of cooked food or fruit juices. + + +The images in the dataset depict fruits in +- varying stages of their life cycle +- different quantities of fruits, and +- different colors of fruits (e.g. green, red and yellow apples) +- some fruits are peeled, cut in half or in slices + +<figure> +<img style="float: right; margin-left: 10px; margin-bottom: 10px" src="figures/examples_from_dataset/bananas-different-stages.png" alt= "Banana images" width="50%" height="auto"> +</figure> + + +This diversity in the dataset is beneficial for our project, as it allows our models to learn from a wide range of fruit images, making them more robust and adaptable to different real-world scenarios. + + ### Train / Dev / Test Split In our project, we have focused on **30** specific **classes** out of the 262 available fruit classes. This decision was based on the relevance and diversity we aimed to achieve in our model's learning scope. We selected classes that are typically found in supermarkets in Germany (e.g. apples, bananas, oranges, mandarins, strawberries, etc.) but also included some exotic fruits (e.g. passion fruits, buddha's hand). We deliberately included fruits that are similar in appearance (e.g. mandarins and oranges) to challenge the model's ability to distinguish between them. A comprehensive list of the selected classes can be found [here](data/class_counts.md).