From 11000984b64e9a8b7fe172b03135dc680bfab728 Mon Sep 17 00:00:00 2001 From: igraf <igraf@cl.uni-heidelberg.de> Date: Fri, 23 Feb 2024 18:41:23 +0100 Subject: [PATCH] Update data and metrics part --- project/README.md | 59 +++++++++++++++++++++++++---------------------- 1 file changed, 31 insertions(+), 28 deletions(-) diff --git a/project/README.md b/project/README.md index 455a5a0..5d841c6 100644 --- a/project/README.md +++ b/project/README.md @@ -35,7 +35,7 @@ This paper explores fruit recognition using deep learning techniques. The author ### Overview -The dataset we are using for our project is based on an **fruit image dataset**, available on [Kaggle](https://www.kaggle.com/datasets/aelchimminut/fruits262). Here are some key details about this dataset: +The dataset we are using for our project is based on the Fruit-262 **fruit image dataset**, available on [Kaggle](https://www.kaggle.com/datasets/aelchimminut/fruits262). Here are some key details about this original dataset: - 225,640 images - 262 types of fruits - ~ 860 images per class @@ -43,33 +43,16 @@ The dataset we are using for our project is based on an **fruit image dataset**, - images have different sizes - contains subfolders for each class with the class name as folder name -### Insights - -The dataset is much cleaner than other fruit datasets we found on Kaggle, which often contain images of cooked food or fruit juices. - - -The images in the dataset depict fruits in -- varying stages of their life cycle -- different quantities of fruits, and -- different colors of fruits (e.g. green, red and yellow apples) -- some fruits are peeled, cut in half or in slices - -<figure> -<img style="float: right; margin-left: 10px; margin-bottom: 10px" src="figures/examples_from_dataset/bananas-different-stages.png" alt= "Banana images" width="50%" height="auto"> -</figure> - - -This diversity in the dataset is beneficial for our project, as it allows our models to learn from a wide range of fruit images, making them more robust and adaptable to different real-world scenarios. ### Train / Dev / Test Split -In our project, we have focused on **30** specific **classes** out of the 262 available fruit classes. This decision was based on the relevance and diversity we aimed to achieve in our model's learning scope. We selected classes that are typically found in supermarkets in Germany (e.g. apples, bananas, oranges, mandarins, strawberries, etc.) but also included some exotic fruits (e.g. passion fruits, buddha's hand). We deliberately included fruits that are similar in appearance (e.g. mandarins and oranges) to challenge the model's ability to distinguish between them. A comprehensive list of the selected classes can be found [here](data/class_counts.md). +In our project, we have focused on **30** specific **classes** out of the 262 available fruit classes, constituting a total of **29,430** images. This decision was based on the relevance and diversity we aimed to achieve in our model's learning scope. We selected classes that are typically found in supermarkets in Germany (e.g. apples, bananas, oranges, mandarins, strawberries, etc.) but also included some exotic fruits (e.g. passion fruits, buddha's hand). We deliberately included fruits that are similar in appearance (e.g. mandarins and oranges) to challenge the model's ability to distinguish between them. A comprehensive list of the selected classes can be found [here](data/class_counts.md). -The original data set lacks a predefined split into training, development (validation), and testing sets. To tailor our dataset for effective model training and evaluation, we implemented a custom script that methodically divides the dataset into specific proportions. +The original dataset lacks a predefined split into training, development (validation), and testing sets. To tailor our dataset for effective model training and evaluation, we implemented a custom script that methodically divides the dataset into specific proportions. <figure> -<img style="float: right; margin-left: 10px; margin-bottom: 10px" src="figures/dataset_split.png" alt= "Dataset Split" width="500" height="auto"> +<img align="left" src="figures/dataset_split.png" alt= "Dataset Split" width="55%" height="auto"> </figure> @@ -91,13 +74,25 @@ We opted not to use cross-validation for the train-dev split, considering the la The data partitioning script randomly segregates the images for each fruit class into the designated training, development, and testing sets. This **random allocation per class** is pivotal for maintaining the data integrity and representativeness in each subset, facilitating an unbiased evaluation of the model's performance. -**To prepare the dataset for using it with this project please refer to the Data Preparation section in the [data folder](data/README.md).** +â†ªï¸ **To prepare the dataset for using it with this project please refer to the Data Preparation section in the [data folder](data/README.md).** -### Data Statistics +### Insights -As part of our dataset analysis, we have focused on understanding the distribution of images across the 30 selected fruit classes in our dataset. Now this dataset is organized into train, dev, and test subsets, and our analysis aims to provide a clear overview of the image distribution across these classes and subsets. +The dataset is much cleaner than other fruit datasets we found on Kaggle, which often contain images of cooked food or fruit juices. -With a total image count of **29430** images across the training, development, and testing subsets, our dataset is comprehensive and representative of the 30 selected fruit classes. +<img align="right" width="50%" height="auto" src="figures/examples_from_dataset/bananas-different-stages.png" title="test"> + +The images in the dataset depict fruits in +- varying stages of their life cycle +- different quantities of fruits, and +- different colors of fruits (e.g. green, red and yellow apples) +- some fruits are peeled, cut in half or in slices + +This diversity in the dataset is beneficial for our project, as it allows our models to learn from a wide range of fruit images, making them more robust and adaptable to different real-world scenarios. + +### Data Distribution Across Classes + +As part of our dataset analysis, we have focused on understanding the distribution of images across the 30 selected fruit classes in our dataset. To visually represent the **class-wise distribution**, we plotted a histogram: @@ -105,18 +100,26 @@ To visually represent the **class-wise distribution**, we plotted a histogram: <img src="figures/class_distribution_histogram.png" alt= "Histogram" width="800" height="auto"> -*Balance of Dataset*: The histogram provides visual and easy to see insights into whether the dataset is balanced or unbalanced. In our case, the dataset is mostly balanced. The nectarine, orange and jostaberry classes may have insufficient datapoints. We will keep an eye on the performance of our models on these classes to see if the imbalance has an impact on the model's ability to learn and predict these classes :mag:. +*Balance of Dataset*: The histogram provides visual and easy to see insights into whether the dataset is balanced or unbalanced. In our case, the dataset is mostly balanced. The apple class is slightly overrepresented and the nectarine, orange and jostaberry classes may have insufficient datapoints. We will keep an eye on the performance of our models on these classes to see if the imbalance has an impact on the model's ability to learn and predict these classes ðŸ”. ## Metrics -- **Accuracy**: The ratio of correctly predicted observations to the total predictions. +To evaluate classification models, the following metrics are commonly used: + +- **Accuracy**: The ratio of correctly predicted observations to the total predictions. + - â© *How many of the predicted classes are correct?* - **Precision**: The ratio of correctly predicted positive observations to the total predicted positives. High precision relates to a low false positive rate. + - â© e.g. *How many of the predicted bananas are actually bananas?* - **Recall**: The ratio of correctly predicted positive observations to the all observations in the actual class. It is also known as sensitivity or true positive rate. + - â© e.g. *How many of the actual bananas were predicted as bananas?* - **F1-Score**: The weighted average of Precision and Recall. It takes both false positives and false negatives into account. Useful for uneven class distribution. + - **Macro Average**: Averages the metric independently for each class before taking the average. This treats all classes equally, regardless of their frequency. + - treats all classes equally, regardless of their size - **Micro Average**: Calculates the metric globally by counting the total true positives, false negatives, and false positives. This is influenced by class frequency. + - treats all images equally, regardless of their class -We are setting our focus on the accuracy metric. The accuracy is a suitable choice for our multi-class classification problem, as it provides a clear and intuitive measure of the model's overall performance. This metric is particularly effective for our balanced dataset, as it provides a reliable measure of the model's success in classifying the images across the 30 different fruit classes. +We are setting our focus on the **accuracy** metric. The accuracy is a suitable choice for our multi-class classification problem, as it provides a clear and intuitive measure of the model's overall performance. This metric is particularly effective for our balanced dataset, as it provides a reliable measure of the model's success in classifying the images across the 30 different fruit classes. ## Baseline ### Overview -- GitLab