Real-time human-centric segmentation for complex video scenes

Most existing video tasks related to “human” focus on the segmentation of salient humans, ignoring unspecified others in video. Few studies have focused segmenting and tracking all humans a complex video, including pedestrians other states (e.g., seated, riding, or occluded). In this paper, we propose novel framework, abbreviated as HVISNet, that segments tracks presented people given videos based one-stage detector. To better evaluate scenes, offer new benchmark called HVIS (Human Video Instance Segmentation), which comprises 1447 human instance masks 805 high-resolution diverse scenes. Extensive experiments show our proposed HVISNet outperforms state-of-the-art methods terms accuracy at real-time inference speed (30 FPS), especially We also notice using center bounding box distinguish different individuals severely deteriorates accuracy, heavily occluded conditions. This common phenomenon is referred ambiguous positive samples problem. alleviate problem, mechanism named Inner Center Sampling improve segmentation. Such plug-and-play inner sampling can be incorporated any model detector performance. particular, it gains 4.1 mAP improvement method case humans. Code data are available https://github.com/IIGROUP/HVISNet.