Rate-Distortion-Complexity Optimization of Video Encoders with Applications to Sign Language Video Compression

Rate-Distortion-Complexity Optimization of Video Encoders with Applications to Sign Language Video Compression Rahul Vanam Co-Chairs of the Supervisory Committee: Professor Eve A. Riskin Electrical Engineering Professor Richard E. Ladner Computer Science and Engineering Applications for video compression have been growing over the years due to the availability of higher network bandwidth, lower cost of memory, and faster processor speed. Some new applications include real-time videoconferencing and video streaming. Most current cell phones come equipped with a video camera, and have the ability to capture a video and playback a recorded video. An emerging area of video compression is mobile videoconferencing, which is enabling the Deaf to communicate in American Sign Language (ASL) using video cell phones. Video encoders developed for PCs cannot be readily used for mobile phones, due to mobile phones’ low processor speeds. In this thesis, we present algorithms for improving the speed of the H.264 video encoder on both PC and cell phone platforms by selecting encoder parameters that jointly trade off encoding speed and video quality at different bitrates. We also apply our algorithms to a region-of-interest based video encoder specific to ASL. This encoder jointly optimizes for ASL intelligibility and bitrate. The parameters chosen by our algorithms are demonstrated to significantly improve the encoding speed at a given ASL intelligibility for different bitrates, on both PC and cell phone platforms. ASL videoconferencing on a cell phone drastically reduces its battery life. To extend the cell phone battery life, we detect the signer’s activity on the cell phone, and use it to control the backlight of the cell phone and the encoding frame rate. This approach improves the battery life by up to 54 minutes. Finally, we present a rate control method that allows the encoder bitrate to adapt to time-varying network bandwidth. We show that when this encoder operates with suitably chosen parameters, it yields both higher encoding speed and ASL intelligibility on a cell phone over a constant bitrate encoder.