TFRC-Based Selective Retransmission for Free-Viewpoint Video Streaming

Árpád Huszák

Authors

Árpád Huszák Budapest University of Technology and Economics https://orcid.org/0000-0002-4756-9283

Keywords:

Free-Viewpoint Video, TFRC, Selective Retransmission, Streaming

Abstract

Free-viewpoint video may become the next big step in media technology that allows users to change the displayed viewpoint and synthesize custom views of a dynamic scene from a user-controlled perspective. The user-specific views are generated from two or more color and depth camera stream pairs that must be successfully delivered to the clients according to their continuously changing perspectives. Besides the view synthesis distortions, the higher error rate and limited bandwidth can cause quality degradation. We proposed an adaptive retransmission-based error recovery scheme allowing retransmissions according to the network congestion state and the allowed latency. The selective retransmission (TbSR) scheme for free-viewpoint video is based on the TCP Friendly Rate Control utilized as a bandwidth estimation algorithm. The TbSR approach was evaluated using the NS-2 network simulator and VSRS reference view synthesis tool in order to prove the efficiency of the proposed method.

Downloads

Download data is not yet available.

References

E. Kohler, M. Handley, and S. Floyd, “Datagram congestion control protocol (DCCP),” RFC 4340, March 2006.

S. Floyd, E. Kohler, and J. Padhye, “Profile for Datagram Congestion Control Protocol (DCCP) Congestion Control ID 3: TCP-Friendly Rate Control (TFRC),” RFC 4342, March 2006.

M. Domański, M. Gotfryd, and K. Wegner, “View synthesis for multiview video transmission,” in The 2009 International Conference on Image Processing, Computer Vision, and Pattern Recognition, Las Vegas, USA, vol. 13, 2009, p. 16.

S. Jo, D. Lee, Y. Kim, and C. D. Yoo, “Development of a simple free viewpoint video system,” in 2008 IEEE International Conference on Multimedia and Expo. IEEE, 2008, pp. 1577–1580.

S. Lu, T. Mu, and S. Zhang, “A survey on multiview video synthesis and editing,” Tsinghua Science and Technology, vol. 21, no. 6, pp. 678–695, 2016.

C. Neupane, A. Koirala, Z. Wang, and K. B. Walsh, “Evaluation of depth cameras for use in fruit localization and sizing: Finding a successor to kinect v2,” Agronomy, vol. 11, no. 9, p. 1780, 2021.

C. Fehn, “Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV,” in Stereoscopic Displays and Virtual Reality Systems XI, vol. 5291. SPIE, 2004, pp. 93–104.

Z. Ni, D. Tian, S. Bhagavathy, J. Llach, and B. S. Manjunath, “Improving the quality of depth image based rendering for 3D video systems,” in IEEE International Conference on Image Processing (ICIP). IEEE, 2009, pp. 513–516.

P. Merkle, Y. Morvan, A. Smolic, D. Farin, K. Mueller, P. De With, and T. Wiegand, “The effects of multiview depth video compression on multiview rendering,” Signal Processing: Image Communication, vol. 24, no. 1-2, pp. 73–88, 2009.

E. Bosc, R. Pepion, P. Le Callet, M. Koppel, P. Ndjiki-Nya, M. Pressigout, and L. Morin, “Towards a new quality metric for 3-D synthesized view assessment,” IEEE Journal of Selected Topics in Signal Processing, vol. 5, no. 7, pp. 1332–1343, 2011.

Y. Chen, M. M. Hannuksela, T. Suzuki, and S. Hattori, “Overview of the mvc+d 3d video coding standard,” Journal of Visual Communication and Image Representation, vol. 25, no. 4, pp. 679–688, 2014.

M. M. Hannuksela, Y. Yan, X. Huang, and H. Li, “Overview of the multiview high efficiency video coding (mv-hevc) standard,” in 2015 IEEE International Conference on Image Processing (ICIP). IEEE, 2015, pp. 2154–2158.

Á. Huszák, “Advanced free viewpoint video streaming techniques,” Multimedia Tools and Applications, vol. 76, no. 1, pp. 373–396, 2017.

T.-Y. Ho, Y.-N. Yeh, and D.-N. Yang, “Multi-view 3d video delivery for broadband ip networks,” in 2015 IEEE International Conference on Communications (ICC). IEEE, 2015, pp. 5796–5802.

P. Carballeira, C. Carmona, C. Díaz, D. Berjón, D. Corregidor, J. Cabrera, F. Morán, C. Doblado, S. Arnaldo, M. del Mar Martín et al., “Fvv live: A real-time free-viewpoint video system with consumer electronics hardware,” IEEE Transactions on Multimedia, vol. 24, pp. 2378–2391, 2021.

T.-Y. Chung, S. Sull, and C.-S. Kim, “Frame loss concealment for stereoscopic video based on interview similarity of motion and intensity difference,” in 2010 IEEE International Conference on Image Processing. IEEE, 2010, pp. 441–444.

X. Liu, Q. Peng, and X. Fan, “Frame loss concealment for multi-view video plus depth,” in 2011 IEEE 15th International Symposium on Consumer Electronics (ISCE). IEEE, 2011, pp. 208–211.

B. Oztas, M. T. Pourazad, P. Nasiopoulos, and V. C. Leung, “Adaptive 3d-hevc video streaming over congested networks through layer prioritization,” in 2017 24th International Conference on Telecommunications (ICT). IEEE, 2017, pp. 1–5.

E. Kurutepe, A. Aksay, C. Bilen, G. Gurler, T. Sikora, G. B. Akar, and A. M. Tekalp, “A standards-based, flexible, end-to-end multi-view video streaming architecture,” in Packet Video 2007. IEEE, 2007, pp. 302–307.

N. Feamster and H. Balakrishnan, “Packet loss recovery for streaming video,” in 12th International Packet Video Workshop. PA: Pittsburgh, 2002, pp. 9–16.

G. Li, S. Liu, H. Li, W. Lei, and W. Zhang, “An adaptive retransmission-based multipath transmission mechanism for conversational video,” International Journal of Communication Systems, vol. 31, no. 15, p. e3778, 2018.

S. Liu, W. Lei, W. Zhang, and Y. Guan, “Cmt-sr: A selective retransmission based concurrent multipath transmission mechanism for conversational video,” Computer Networks, vol. 112, pp. 360–371, 2017.

L. Han, S.-S. Kang, H. Kim, and H. P. In, “Adaptive retransmission scheme for video streaming over content-centric wireless networks,” IEEE Communications Letters, vol. 17, no. 6, pp. 1292–1295, 2013.

M. T. Piecuch, K. French, G. Oprica, and M. Claypool, “Selective retransmission protocol for multimedia on the internet,” in Multimedia Systems and Applications III, vol. 4209. SPIE, 2001, pp. 79–90.

Á. Huszák and S. Imre, “Source controlled semi-reliable multimedia streaming using selective retransmission in dccp/ip networks,” Computer Communications, vol. 31, no. 11, pp. 2676–2684, 2008.