RGB to Spectral Reconstruction via Learned Basis Functions and Weights

Single RGB image hyperspectral reconstruction has seen a boost in performance and research attention with the emergence of CNNs and more availability of RGB/hyperspectral datasets. This work proposes a CNN-based strategy for learning RGB to hyperspectral cube mapping by learning a set of basis functions and weights in a combined manner and using them both to reconstruct the hyperspectral signatures of RGB data. Further to this, an unsupervised learning strategy is also proposed which extends the supervised model with an unsupervised loss function that enables it to learn in an end-to-end fully self supervised manner. The supervised model outperforms a baseline model of the same CNN model architecture and the unsupervised learning model shows promising results. Code will be made available online here