Residual Regression with Semantic Prior for Crowd Counting

Crowd counting is a challenging task due to factors such as large variations in crowdedness and severe occlusions. Although recent deep learning based counting algorithms have achieved a great progress, the correlation knowledge among samples and the semantic prior have not yet been fully exploited. In this paper, a residual regression framework is proposed for crowd counting harnessing the correlation information among samples. By incorporating such information into our network, we discover that more intrinsic characteristics can be learned by the network which thus generalizes better to unseen scenarios. Besides, we show how to effectively leverage the semantic prior to improve the performance of crowd counting. We also observe that the adversarial loss can be used to improve the quality of predicted density maps, thereby leading to an improvement in crowd counting. Experiments on public datasets demonstrate the effectiveness and generalization ability of the proposed method.

In detail, residual regression predicts a residual map (i.e., the difference between density maps) between the input image and the support image. All the residual predictions are fused, and the final predicted map is calculated based on the fused residual prediction and the appearance-based prediction from the input image.

Selected Publications

Results/Code