Efficiency is key in machine learning and deep learning. One of the major challenges faced is relating to the volume of data, and in some scenarios, the volume of labelled data. This is where active learning finds its relevance. As an integral part of machine learning, active learning allows the learner to decide which observation requires labeling.
To achieve this, the learner leans on a strategy that permits it to choose only certain observations that will henceforth be labeled. Among several strategies proposed so far, uncertainty sampling remains one of the most renowned.

Uncertainty Sampling

Uncertainty sampling, a method in active learning, involves choosing instances for labeling for which the learner is most uncertain. To measure this uncertainty, entropy, a probabilistic measure, is often used. However, this article makes an argument for a broader framework of uncertainty that extends beyond probabilities.

A couple of recent papers have suggested that uncertainty can be split into two distinct aspects: aleatoric and epistemic uncertainties. Aleatoric uncertainty is inherent and arises from the natural variability or randomness of the event and is not reducible. On the other hand, epistemic uncertainty, tied to the lack of knowledge, can be reduced.

Practically, the labeling process, often undertaken by humans, does not account for the variance in hesitation or certainty in assigning labels, thus hinting at an inbuilt uncertainty within the labels. This element has been widely ignored in most models and sampling strategies, which is a problem that the proposed strategies intend to fix.

Proposed Strategies

Two uncertainty sampling strategies have been proposed in this article. Both of them cater to an understanding of a model's uncertainties with regard to the inherent uncertainty already present in the labels. The first strategy calculates two different uncertainties - self-conflicting properties and ignorance - based on the model output.

The second strategy broadens the reducible uncertainty to cover an evidential framework and multiple classes, thereby simplifying computation.

Imperfect Labeling

One crucial issue in the dataset used for classification is the common reliance on hard labels, a binary membership where it is clearly defined whether an observation belongs to a class or not. This paper, however, proposes an alternative in the form of rich labels. These labels, powered by the theory of belief functions, offer a broader scope than hard labels by including degrees of imprecision.

Future Direction

This article makes an ardent case for the utilization of rich labels and their modeling using belief functions theory. Including these in active learning can better equip the process to handle the inherent uncertainty in the labels, resulting in more comprehensive classification models. Furthermore, the active learning objective aims to find its future application in these uncertainty mappings on 2D representations.

Conclusion

Undoubtedly, the need for efficiency in classifying vast troves of data is paramount. As discussed, active learning and uncertainty sampling provide solutions to this need. Furthermore, the elaboration and proposal of uncertainty decomposition into epistemic and aleatoric types put forth a compelling approach - one that acknowledges humans' inherent uncertainty during labeling.