Mediocre-Bullfrog686
Mediocre-Bullfrog686 t1_jb71qkj wrote
Reply to [D] The MMSegmentation library from OpenMMLab appears to return the wrong results when computing basic image segmentation metrics such as the Jaccard index (IoU - intersection-over-union). It appears to compute recall (sensitivity) instead of IoU, which artificially inflates the performance metrics. by florinandrei
Pixels with ignore_index mean that they should be ignored (e.g., pixels in the ground-truth image that the annotators are not sure about). It does not mean that they are from a "negative class". It is correct to ignore those pixels during IoU computation.
Mediocre-Bullfrog686 t1_jb7n704 wrote
Reply to comment by florinandrei in [D] The MMSegmentation library from OpenMMLab appears to return the wrong results when computing basic image segmentation metrics such as the Jaccard index (IoU - intersection-over-union). It appears to compute recall (sensitivity) instead of IoU, which artificially inflates the performance metrics. by florinandrei
>If there is some index you want to ignore altogether, because you are not sure about the quality of the labels, it is best to just exclude it from the calculation of the average metric.
Isn't this what the ignore_index is doing? How else should we exclude them from the average metric? By applying ignore_index we effectively ignore those pixels.
>If some users set
ignore_indexto the value of the background pixels, that will cut very large holes in everything, therefore discarding a lot of pixels from performance evaluation, and will severely skew the results.Well the users definitely should not do that. This is then a matter of documentation. We cannot just get rid of ignore_index because (I think) it is used in some existing segmentation datasets.