Introduction

Probe-based Confocal Laser Endomicroscopy (pCLE) has seen a surge of interest with its utility in a range of clinical indications and organ systems, including gastrointestinal, urological, and respiratory tracts. The pCLE probe relies on a coherent fibre bundle comprising many (>10k) cores that are irregularly distributed across the field of view (FoV) leading to an inherent limitation which negatively impacts the image quality.

Existing Reconstruction Approaches

Existing pCLE image reconstruction approaches typically use Delaunay triangulation to linearly interpolate irregularly sampled signals onto a Cartesian grid. These interpolation methods allow for the reconstruction of the Cartesian image, yet they do not enhance image quality nor take into account any prior knowledge of the image space and are prone to generating artifacts.

However, it has been shown that state-of-the-art CNN-based single-image super-resolution (SISR) techniques can improve the quality of pCLE images. A potential limitation in the current CNN approaches is that the analysis starts from already reconstructed pCLE images, including reconstruction artefacts.

Sparse CNN Inputs

A few research studies have been focusing on allowing sparse data as CNN input, suggesting that applying CNNs directly to irregularly sampled pCLE data is far from trivial. We propose a solution that facilitates using sparse images as the input of the SR CNN directly, without the need for prior reconstruction, and also eliminating edge artefacts from input images and compare it to the classical SR methods and reconstruction algorithm.

While convolution layers are widely used, they have been identified as sub-optimal for dealing with sparse data. Much of the available literature on exploring sparsity in the context of CNN input deals with the irregular data in an intuitive but ad-hoc way: non-informative pixels are assigned zero, creating an artificial Cartesian image.

Materials and Methods

Considering that common Image Quality Assessment (IQA) relies on ground-truth images used as a reference in metrics such as the Peak signal-to-noise ratio (PSNR), the lack of ground-truth high-resolution pCLE images makes it difficult to evaluate the quality of SR reconstructions. We propose a method to address this issue by using triangulation-based reconstruction algorithm to simulate synthetic HR and LR endomicroscopy.

In order to compare pCLE image reconstructions obtained from standard interpolation methods and custom-crafted deep learning architectures, we create a novel trainable convolutional layer, an NW layer, which integrates Nadaraya-Watson (NW) kernel regression into the DL framework to effectively handle irregularly sampled data in the CNN network.

Conclusion

The findings of our study indicate that both dense and sparse CNNs outperform the reconstruction method currently used in the clinic. Moreover, the study provides an innovative comparison of sparse and dense approach in pCLE image reconstruction, implementing trainable generalized NW kernel regression, and adaptation of synthetic data for training pCLE SR.