Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System

Ma, Chenguang; Cao, Xun; Tong, Xin; Dai, Qionghai; Lin, Stephen
November 2014
International Journal of Computer Vision;Nov2014, Vol. 110 Issue 2, p141
Academic Journal
We present a hybrid camera system for capturing video at high spatial and spectral resolutions. Composed of an red, green, and blue (RGB) video camera, a grayscale video camera and a few optical elements, the hybrid camera system simultaneously records two video streams: an RGB video with high spatial resolution, and a multispectral (MS) video with low spatial resolution. After registration of the two video streams, our system propagates the MS information into the RGB video to produce a video with both high spectral and spatial resolution. This propagation between videos is guided by color similarity of pixels in the spectral domain, proximity in the spatial domain, and the consistent color of each scene point in the temporal domain. The propagation algorithm, based on trilateral filtering, is designed to rapidly generate output video from the captured data at frame rates fast enough for real-time video analysis tasks such as tracking and surveillance. We evaluate the proposed system using both simulations with ground truth data and on real-world scenes. The accuracy of spectral capture is examined through comparisons with ground truth and with a commercial spectrometer. The utility of this high resolution MS video data is demonstrated on the applications of dynamic white balance adjustment, object tracking, and separating the appearance contributions of different illumination sources. The various high resolution MS video datasets that we captured will be made publicly available to facilitate research on dynamic spectral data analysis.


Related Articles

  • Marsobot: design and performance characterization of a low-cost, ground-based multispectral camera on an open source rover. Hobbs, S.W.; Paull, D.J.; Clarke, J.D.A. // International Journal of Remote Sensing;Jun2017, Vol. 38 Issue 12, p3635 

    Open source, low cost sensors, and robotic systems have developed to the point of being able to produce meaningful, repeatable results in real-life applications. We developed a low-cost, open source multispectral camera mounted on a small custom-built robotic rover. We compared the performance...

  • Fully spatially adaptive smoothing parameter estimation for Markov random field super-resolution mapping of remotely sensed images. Aghighi, H.; Trinder, J.; Lim, S.; Tarabalka, Y. // International Journal of Remote Sensing;Jun2015, Vol. 36 Issue 11, p2851 

    This article presents a fully spatially adaptive Markov random field (MRF)-based super-resolution mapping (SRM) technique to produce land-cover maps at a finer spatial resolution than the original coarse-resolution image. MRF combines the spectral and spatial energies; hence, an MRF-SRM...

  • An Empirical Investigation on Thematic Accuracy of Landuse/Landcover Classification Using Fused Images. Jeya Kumari, N.; Purushothaman, B. M.; Suresh Babu, S. // International Journal of Computer Science & Information Technolo;2014, Vol. 5 Issue 2, p1155 

    The satellite images at different spectral and spatial resolutions with the aid of image processing techniques can improve the quality of information. Especially image fusion is very helpful to extract the spatial information from two images of different resolution images of same area. An...

  • Super-Resolution of Multispectral Images. M. Vega; J. Mateos; R. Molina; A.K. Katsaggelos // Computer Journal;Vol. 52 Issue 1, p153 

    In this paper we propose and analyze a globally and locally adaptive super-resolution Bayesian methodology for pansharpening of multispectral images. The methodology incorporates prior knowledge on the expected characteristics of the multispectral images uses the sensor characteristics to model...

  • A regularized pan-sharpening approach based on self-similarity and Gabor prior. Upla, Kishor P.; Joshi, Manjunath V.; Khatri, Nilay // International Journal of Remote Sensing;Feb2017, Vol. 38 Issue 4, p949 

    In this article, we propose a new regularization-based approach for pan-sharpening based on the concepts of self-similarity and Gabor prior. The given low spatial resolution (LR) and high spectral resolution multi-spectral (MS) image is modelled as degraded and noisy version of the unknown high...

  • Pan-sharpening using a guided filter. Liu, Junmin; Liang, Shunlin // International Journal of Remote Sensing;Apr2016, Vol. 37 Issue 8, p1777 

    Pan-sharpening aims to integrate the spatial details of a high-resolution panchromatic (Pan) image with the spectral information of low-resolution multispectral (MS) images to produce high-resolution MS images. The key is to appropriately estimate the missing spatial details of the MS images...

  • Pansharpening by exploiting sharpness of the spatial structure. Rong, Kaixuan; Wang, Shuang; Yang, Shuyuan; Jiao, Licheng // International Journal of Remote Sensing;Sep2014, Vol. 35 Issue 18, p6662 

    High-frequency injection (HFI)-based methods are proved to be powerful in pansharpening multispectral (MS) images. In this article, based on one of the low-rank and sparse (LRS) decomposition algorithms, i.e. Go Decomposition (GoDec), a HFI-based pansharpening method exploiting spatial structure...

  • Multispectral and Texture Feature Application in Image-Object Analysis of Summer Vegetation in Eastern Tajikistan Pamirs. Salas, Eric Ariel L.; Boykin, Kenneth G.; Valdez, Raul // Remote Sensing;Jan2016, Vol. 8 Issue 1, p78 

    We tested the Moment Distance Index (MDI) in combination with texture features for the summer vegetation mapping in the eastern Pamir Mountains, Tajikistan using the 2014 Landsat OLI (Operational Land Imager) image. The five major classes identified were sparse vegetation, medium-dense...

  • Multispectral Radiometric Analysis of Façades to Detect Pathologies from Active and Passive Remote Sensing. Del Pozo, Susana; Herrero-Pascual, Jesús; Felipe-García, Beatriz; Hernández-López, David; Rodríguez-Gonzálvez, Pablo; González-Aguilera, Diego // Remote Sensing;Jan2016, Vol. 8 Issue 1, p80 

    This paper presents a radiometric study to recognize pathologies in façades of historical buildings by using two different remote sensing technologies covering part of the visible and very near infrared spectrum (530-905 nm). Building materials deteriorate over the years due to different...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics