From 54ed1e2abac26e86cc83b2dc9f7e0380f4332eda Mon Sep 17 00:00:00 2001 From: igraf <igraf@cl.uni-heidelberg.de> Date: Fri, 23 Feb 2024 22:11:10 +0100 Subject: [PATCH] Fix some typos --- project/README.md | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/project/README.md b/project/README.md index 6a60dcd..7368616 100644 --- a/project/README.md +++ b/project/README.md @@ -47,7 +47,7 @@ The dataset we are using for our project is based on the Fruit-262 **fruit image ### Train / Dev / Test Split -In our project, we have focused on **30** specific **classes** out of the 262 available fruit classes, constituting a total of **29,430** images. This decision was based on the relevance and diversity we aimed to achieve in our model's learning scope. We selected classes that are typically found in supermarkets in Germany (e.g. apples, bananas, oranges, mandarins, strawberries, etc.) but also included some exotic fruits (e.g. passion fruits, buddha's hand). We deliberately included fruits that are similar in appearance (e.g. mandarins and oranges) to challenge the model's ability to distinguish between them. A comprehensive list of the selected classes can be found [here](data/class_counts.md). +In our project, we have focused on **30** specific **classes** out of the 262 available fruit classes, constituting a total of **29,430** images. This decision was based on the relevance and diversity we aimed to achieve in our model's learning scope. We selected classes that are typically found in supermarkets in Germany (e.g. apples, bananas, oranges, mandarines, strawberries, etc.) but also included some exotic fruits (e.g. passion fruits, buddha's hand). We deliberately included fruits that are similar in appearance (e.g. mandarines and oranges) to challenge the model's ability to distinguish between them. A comprehensive list of the selected classes can be found [here](data/class_counts.md). The original dataset lacks a predefined split into training, development (validation), and testing sets. To tailor our dataset for effective model training and evaluation, we implemented a custom script that methodically divides the dataset into specific proportions. @@ -261,13 +261,13 @@ While not as sophisticated as models like CNNs, Naive Bayes classifiers can stil ### Decision Tree -A decision tree classifier is a flowchart-like structure that recursively divides the dataset into smaller and smaller subsets based on the most significant features. It can also be viewed as a model that makes decisions based on asking a series of questions. +A Decision Tree classifier is a flowchart-like structure that recursively divides the dataset into smaller and smaller subsets based on the most significant features. It can also be viewed as a model that makes decisions based on asking a series of questions. #### How Decision Trees Work -Based on the features of the input data, a decision tree model asks a series of yes/no questions and leads down a path in the tree that eventually results in a decision (in this case, fruit classification). These models are intuitive and easy to visualize but can become complex and prone to overfitting, especially with a large number of features. +Based on the features of the input data, a Decision Tree model asks a series of yes/no questions and leads down a path in the tree that eventually results in a decision (in this case, fruit classification). These models are intuitive and easy to visualize but can become complex and prone to overfitting, especially with a large number of features. -A simplified example of a decision tree for our fruit classification task with a maximum depth of 2 is shown below. In each node, the tree compares a feature of the input image to a threshold and decides which branch to follow based on the result. The final decision is made at the leaf nodes. +A simplified example of a Decision Tree for our fruit classification task with a maximum depth of 2 is shown below. In each node, the tree compares a feature of the input image to a threshold and decides which branch to follow based on the result. The final decision is made at the leaf nodes. <img align="center" width="100%" height="auto" src="figures/decision_tree/plotted_tree_50x50_standard_max_depth_2_max_features_sqrt_min_samples_leaf_1_min_samples_split_2.png" title="test"> @@ -304,7 +304,7 @@ We have conducted a series of experiments to evaluate the performance of differe 🔠For our basic classifiers (Naive Bayes, Decision Tree, Random Forest), we have utilized scikit-learn's `GridSearchCV` to optimize the hyperparameters. This approach allows us to systematically test different hyperparameter combinations and identify the best configuration for each classifier. In order to use the development set for hyperparameter tuning, we have used the `PredefinedSplit` function to create a fixed split between the training and development set instead of using a default cross-validation split. -For the random forest classifier, we have conducted additional experiments involving various picture sizes and number of estimators. This exploration was motivated by the notably promising results obtained from this classifier compared to the other basic classifiers. +For the Random Forest classifier, we have conducted additional experiments involving various picture sizes and number of estimators. This exploration was motivated by the notably promising results obtained from this classifier compared to the other basic classifiers. â†ªï¸ To reproduce the results, please refer to the [README](src/README.md) in the `src` folder for instructions on how to run the experiments. @@ -361,7 +361,7 @@ The following table shows an excerpt of the feature and size combination used. ### Decision Tree -To find the best hyperparameters for the decision tree classifier, we have used the following parameter grid for the grid search: +To find the best hyperparameters for the Decision Tree classifier, we have used the following parameter grid for the grid search: ``` { @@ -374,7 +374,7 @@ To find the best hyperparameters for the decision tree classifier, we have used ``` -The best accuracy on the develepment set is **0.23** with the **HSV + Sobel filters** on 50x50 images. The best parameters for the decision tree classifier, which can also be seen in the plot below, are: +The best accuracy on the develepment set is **0.23** with the **HSV + Sobel filters** on 50x50 images. The best parameters for the Decision Tree classifier, which can also be seen in the plot below, are: ``` { @@ -392,7 +392,7 @@ The best accuracy on the develepment set is **0.23** with the **HSV + Sobel filt ### Random Forest -To find the best hyperparameters for the random forest classifier, we have used the following parameter grid for the grid search: +To find the best hyperparameters for the Random Forest classifier, we have used the following parameter grid for the grid search: ``` { @@ -404,7 +404,7 @@ To find the best hyperparameters for the random forest classifier, we have used } ``` -The best accuracy on the development set is **0.469** with the **HSV filters** (and also with **HSV + Sobel filters**) on 50x50 images. The best parameters for the random forest classifier, which can also be seen in the plot below, are: +The best accuracy on the development set is **0.469** with the **HSV filters** (and also with **HSV + Sobel filters**) on 50x50 images. The best parameters for the Random Forest classifier, which can also be seen in the plot below, are: ``` { @@ -477,7 +477,8 @@ Confusion Matrix - No filters - best parameters | Confusion Matrix -  |  - Observations: - - Classifiers both make the same mistakes, e.g. confusing raspberries, redcurrants and strawberries :strawberry: (see bottom right corner of confusion matrix) + - medium to high accuracy for most classes + - classifiers both make the same mistakes, e.g. confusing raspberries, redcurrants and strawberries :strawberry: (see bottom right corner of confusion matrix) @@ -491,15 +492,16 @@ We have implemented a Convolutional Neural Network (CNN) for our fruit image cla The CNN model achieved an accuracy of **0.68** on the development set. This is a significant improvement over the baseline models and the basic classifiers. The model's performance on the training set was a little higher, with an accuracy of **0.83**. This suggests that the model is not overfitting, and it is learning effectively from the training data. The model's performance on the test set is expected to be similar to the development set, given the balanced nature of our dataset. -The learning curve of the CNN model shows that the model is learning effectively from the training data, with the training and validation loss decreasing and the accuracy increading over time. Also we can see that after about **25-30** Epochs the model reaches its best performance and the training stagnates. +The learning curve of the CNN model shows that the model is learning effectively from the training data, with the training and validation loss decreasing and the accuracy increasing over time. Also we can see that after about **25-30** epochs the model reaches its best performance and the training stagnates.  -The confusion matrix of the CNN model shows that the model is performing well across all classes, with high accuracy for most classes. The model's performance is consistent across most classes, with some problems in distinguishing between similar classes, such as mandarins, oranges and grapefruits. Also the mandarine class is the one with the lowest accuracy because it is underrepresented in the dataset. +The confusion matrix of the CNN model shows that the model is performing well across all classes, with high accuracy for most classes. The model's performance is consistent across most classes, with some problems in distinguishing between similar classes, such as mandarines and oranges. Also the nectarine class is the one with the lowest accuracy because it is underrepresented in the dataset.  -With looking at some of the images that were misclassified, we can see that the model is sometimes confused by the similar shape and color of the fruits. For example, as stated before, the model often confuses mandarins, oranges and grapefruits. +With looking at some of the images that were misclassified, we can see that the model is sometimes confused by the similar shape and color of the fruits. For example, as stated before, the model often confuses mandarines and oranges. +   @@ -512,7 +514,10 @@ Having tested different feature combinations, hyperparameters and picture sizes  -The CNN model achieved the highest accuracy of **0.68**, followed by the Random Forest model with an accuracy of **0.48**. The Naive Bayes and Decision Tree models achieved the lowest accuracies, with **0.18** and **0.39**, respectively. The Random Forest model's performance is particularly noteworthy, as it achieved an accuracy of **0.48** using the HSV feature combination on 50x50 images. This is a good improvement over the baseline models and the other basic classifiers, demonstrating the effectiveness of the Random Forest model for our fruit image classification task. The CNN model's performance is also impressive, achieving an accuracy of **0.68**. This is a confirmation of the effectiveness of CNNs for image classification tasks, and it is a significant improvement over the baseline models and the basic classifiers. +The CNN model achieved the highest accuracy of **0.68**, followed by the R +andom Forest model with an accuracy of **0.48**. The Naive Bayes and Decision Tree models achieved the lowest accuracies, with **0.18** and **0.23**, respectively. The Random Forest model's performance is particularly noteworthy, as it achieved an accuracy of **0.48** using the HSV feature combination on 50x50 images. + +This is a good improvement over the baseline models and the other basic classifiers, demonstrating the effectiveness of the Random Forest model for our fruit image classification task. The CNN model's performance is also impressive, achieving an accuracy of **0.68**. This is a confirmation of the effectiveness of CNNs for image classification tasks, and it is a significant improvement over the baseline models and the basic classifiers. The performance on the dev and test set is (as expected) nearly the same, which is a good sign that the model is not overfitting and again shows the balanced nature of our dataset. ### Feature Importance @@ -523,7 +528,7 @@ To answer this question, we can use the `feature_importances_` attribute of the When using the RGB or HSV values as features, we have three features for each pixel. In order to visualize the feature importance, we **sum the feature importances for each pixel** and reshape the resulting array to the original image shape that was used for training the model. This way, we can visualize the feature importance for each pixel in the image. -As can be seen in the following plot, the **pixels in the middle** have higher values and are thus more important for the classification than the pixels near the edges. The same pattern is found for all decision tree and random forest models that we have trained. This meets our expectations, as the middle of the image is **where the fruit is typically located** and the edges are often just the background. +As can be seen in the following plot, the **pixels in the middle** have higher values and are thus more important for the classification than the pixels near the edges. The same pattern is found for all Decision Tree and Random Forest models that we have trained. This meets our expectations, as the middle of the image is **where the fruit is typically located** and the edges are often just the background. <img src="figures/random_forest/RandomForestClassifier_50x50_hsv_feature_importances_max_depth_70_max_features_sqrt_min_samples_leaf_2_min_samples_split_2_n_estimators_100.png" alt= "Random Forest Feature Importance" width="500" height="auto"> @@ -532,7 +537,7 @@ As can be seen in the following plot, the **pixels in the middle** have higher v *How well do our models perform with a reduced dataset size for training?* -We have also tested training a random forest model and a CNN model on a reduced dataset size. In the Diagram below you can see the results of the tests. The results show that the performance of the models decreases with a reduced dataset size but we can still achieve a good performance with 50% or more of the original dataset size. +We have also tested training a Random Forest model and a CNN model on a reduced dataset size. In the Diagram below you can see the results of the tests. The results show that the performance of the models decreases with a reduced dataset size but we can still achieve a good performance with 50% or more of the original dataset size.  -- GitLab