International Journal of Image, Graphics and Signal Processing (IJIGSP)
IJIGSP Vol. 17, No. 3, 8 Jun. 2025
Cover page and Table of Contents: PDF (size: 992KB)
LiteDVDNet: Optimizing FastDVDNet for High-Speed Video Denoising
PDF (992KB), PP.1-11
Views: 0 Downloads: 0
Author(s)
Andrii Ilchenko
1,2,*
Sergii Stirenko
1,2
Index Terms
Video Denoising, Efficient Inference, Deep Neural Networks, Deep Learning
Abstract
The growing demand for high-quality video processing in real-time applications demands efficient denoising techniques that can operate swiftly while maintaining visual fidelity. Conventional approaches often struggle to balance these competing requirements, especially when dealing with high-resolution video streams or resource-constrained environments. This study aims to develop methods for accelerating video data denoising using deep convolutional neural networks while maintaining acceptable output quality. We selected the popular FastDVDNet denoising network, which operates on a sliding window principle, as our baseline for comparison and a starting point for our research. This paper proposes several modifications of FastDVDNet that significantly enhance computational efficiency. We introduce four key optimizations: caching intermediate denoising results, reducing intermediate channels in input block, simplifying convolutional blocks, and halving the number of channels. We evaluated these modifications on the Set8 dataset and compared the results with the original model at various noise levels. Finally, we introduce LiteDVDNet, a fine-tuned version of FastDVDNet model that achieves the optimal balance between processing speed, and denoising performance. We developed two model variants: LiteDVDNet-32, which is 3× faster than the original model with only 0.18 dB average PSNR reduction, and the more lightweight LiteDVDNet-16, which delivers a 5× speed improvement at the cost of 0.61 dB average PSNR reduction.
Cite This Paper
Andrii Ilchenko, Sergii Stirenko, "LiteDVDNet: Optimizing FastDVDNet for High-Speed Video Denoising", International Journal of Image, Graphics and Signal Processing(IJIGSP), Vol.17, No.3, pp. 1-11, 2025. DOI:10.5815/ijigsp.2025.03.01
Reference
[1]Matias Tassano, Julie Delon, and Thomas Veit. 2019. DVDNet: A fast network for deep video denoising. In ICIP
[2]Matias Tassano, Julie Delon, and Thomas Veit. 2020. FastDVDNet: Towards real-time deep video denoising without flow estimation. In CVPR
[3]Kelvin C.K. Chan, Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. 2021. BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond. In CVPR.
[4]Matteo Maggioni, Yibin Huang, Cheng Li, Shuai Xiao, Zhongqian Fu, and Fenglong Song. 2021. Efficient Multi-Stage Video Denoising with Recurrent Spatio Temporal Fusion. In CVPR.
[5]Liuyu Xiang, Jundong Zhou, Jirui Liu, Zerun Wang, Haidong Huang, Jie Hu, Jungong Han, Yuchen Guo, and Guiguang Ding. 2022. ReMoNet: Recurrent Multi-output Network for Efficient Video Denoising. In AAAI.
[6]Jingyun Liang, Jiezhang Cao, Yuchen Fan, Kai Zhang, Rakesh Ranjan, Yawei Li, Radu Timofte, and Luc Van Gool. 2022. VRT: A Video Restoration Transformer. arXiv:2201.12288 (2022).
[7]Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In MICCAI.
[8]Chenyang Qi, Junming Chen, Xin Yang, Qifeng Chen. 2022 Real-time Streaming Video Denoising with Bidirectional Buffers, In ACM Multimedia
[9]Wenzhe Shi, Jose Caballero, Ferenc Huszar, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. 2016. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In IEEE
[10]Matias Tassano, Julie Delon, and Thomas Veit. An analysis and implementation of the FFDnet image denoising method. Image Processing On Line, 9:1–25, Jan 2019.
[11]Sergey Ioffe and Christian Szegedy. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In International Conference on Machine Learning (ICML), pages 448–456. JMLR.org, 2015.
[12]Anna Khoreva, Anna Rohrbach, and Bernt Schiele. Video object segmentation with language referring expressions. In ACCV, 2018.
[13]Adam Paszke, Gregory Chanan, Zeming Lin, Sam Gross, Edward Yang, Luca Antiga, and Zachary Devito. Automatic differentiation in PyTorch. 2017. In NIPS
[14]D.P. Kingma and J.L. Ba. ADAM: A Method for Stochastic Optimization. Proc. ICLR, 2015.
[15]Kai Zhang, Wangmeng Zuo, and Lei Zhang. FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Transactions on Image Processing, Sep 2018.
[16]Rajiv Soundararajan and Alan C. Bovik. Video quality assessment by reduced reference spatio-temporal entropic differencing. IEEE Transactions on Circuits and Systems for Video Technology, 2013
[17]Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600-612
[18]Axel Davy, Thibaud Ehret, Jean Michel Morel, Pablo Arias, and Gabriele Facciolo. 2018. Non-Local Video Denoising by CNN. arXiv abs/1811.12758 (2018).
[19]Gregory Vaksman, Michael Elad, and Peyman Milanfar. 2021. Patch Craft: Video Denoising by Deep Modeling and Patch Matching. In ICCV.
[20]Song Han, Huizi Mao, and William J. Dally. 2016. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. International Conference on Learning Representations (ICLR).
[21]Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the Knowledge in a Neural Network. arXiv preprint arXiv:1503.02531.
[22]Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. 2017. Learning Efficient Convolutional Networks through Network Slimming. In IEEE International Conference on Computer Vision (ICCV).
[23]Michal Drozdzal, Eugene Vorontsov, Gabriel Chartrand, Samuel Kadoury, and Chris Pal. 2016. The Importance of Skip Connections in Biomedical Image Segmentation. In Deep Learning and Data Labeling for Medical Applications.
[24]Dumitru Erhan, Christian Szegedy, Alexander Toshev, and Dragomir Anguelov. 2014. Scalable Object Detection Using Deep Neural Networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[25]Sergei Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer. 2014. cuDNN: Efficient Primitives for Deep Learning. arXiv preprint arXiv:1410.0759
[26]Andrew Lavin and Scott Gray. 2016. Fast Algorithms for Convolutional Neural Networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
[27]Tal Ben-Nun and Torsten Hoefler. 2019. Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis. ACM Computing Surveys, 52(4)
[28]Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021). Pruning and quantization for deep neural network acceleration: A survey. Neurocomputing, 461, 370-390.