Im currently struggling to understand the use of the IoU. Is the IoU just a Metric to monitor the quality of a network, or is used as a loss function where the value has some impact on the backprop?
>Solution :
For a measure to be used as a loss function, it must be differentiable, with non-trivial gradients.
For instance, in image classification, accuracy is the most common measure of success. However, if you try to differentiate accuracy, you’ll see that the gradients are zero almost everywhere and therefore one cannot train a model with accuracy as a loss function.
Similarly, IoU, in its naive form, also has meaningless gradients and cannot be used as a loss function. However, extensions to IoU that preserve gradients exist and can be effectively used as loss function for training.