About
Welcome to the Video, Image, and Sound Analysis Lab (VISAL) at the City University of Hong Kong! The lab is directed by Dr. Antoni Chan in the Department of Computer Science.
Our main research activities include:
- Computer Vision, Surveillance
- Machine Learning, Pattern Recognition
- Computer Audition, Music Information Retrieval
- Eye Gaze Analysis
For more information about our current research, please visit the projects and publication pages.
Opportunities for graduate students and research assistants – if you are interested in joining the lab, please check this information.
Latest News [more]
- [Nov 28, 2019]
Congratulations to Weihong for defending his thesis!
- [Nov 28, 2019]
Congratulations to Tianyu for defending his thesis!
- [Aug 23, 2019]
Xueying Zhan receives the “Outstanding Academic Performance Award”, and Xueying Zhan and Jia Wan receive the “Research Tuition Scholarship” from the School of Graduate Studies. Congratulations!
- [May 6, 2019]
Congratulations to Di for defending his thesis!
Recent Publications [more]
- Understanding visual attention to face emotions in social anxiety using hidden Markov models.
,
Cognition and Emotion, to appear 2020. - Incorporating Side Information by Adaptive Convolution.
,
International Journal of Computer Vision (IJCV), to appear 2020. - On Diversity in Image Captioning: Metrics and Methods.
,
IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), to appear 2020. - Fully Nested Neural Network for Adaptive Compression and Quantization.
,
In: International Joint Conf. on Artificial Intelligence - Pacific Rim International Conference on Artificial Intelligence (IJCAI-PRICAI 2020), Yokohama, to appear Jan 2021. - Compare and Reweight: Distinctive Image Captioning Using Similar Images Sets.
,
In: European Conference on Computer Vision (ECCV), to appear Aug 2020 (oral). - Interpretation biases and visual attention in the processing of ambiguous information in chronic pain.
,
European Journal of Pain, 24(7):1242-1256, Aug 2020. - The role of eye movement consistency in learning to recognise faces: Computational and experimental examinations.
,
In: 42nd Annual Conference of the Cognitive Science Society (CogSci), Jul 2020. - ROAM: Recurrently Optimizing Tracking Model.
,
In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, Jun 2020. [code] - Eye movement analysis with switching hidden Markov models.
,
Behavior Research Methods, 52:1026-1043, June 2020. - 3D Crowd Counting via Multi-View Fusion with 3D Gaussian Kernels.
,
In: AAAI Conference on Artificial Intelligence, AAAI, New York, Feb 2020. [supplemental]
Recent Project Pages [more]
We propose to offline train a recurrent neural optimizer to update a tracking model in a meta-learning setting, which can converge the model in a few gradient steps during online training.
- , "ROAM: Recurrently Optimizing Tracking Model." In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, Jun 2020. [code]
Recently, an end-to-end multi-view crowd counting method called multi-view multi-scale (MVMS) has been proposed, which fuses multiple camera views using a CNN to predict a 2D scene-level density map on the ground-plane. Unlike MVMS, we propose to solve the multi-view crowd counting task through 3D feature fusion with 3D scene-level density maps, instead of the 2D ground-plane ones.
- , "3D Crowd Counting via Multi-View Fusion with 3D Gaussian Kernels." In: AAAI Conference on Artificial Intelligence, AAAI, New York, Feb 2020. [supplemental]
In the sense of end-to-end training, the hand-crafted methods used for generating the density maps may not be optimal for the particular network or dataset used. To address this issue, we propose an adaptive density map generator, which takes the annotation dot map as input, and learns a density map representation for training a counter. The counter and generator are trained jointly within an end-to-end framework.
- , "Adaptive Density Map Generation for Crowd Counting." In: Intl. Conf. on Computer Vision (ICCV), Seoul, Oct 2019.
We use a switching hidden Markov model (EMSHMM) approach to analyze eye movement data in cognitive tasks involving cognitive state changes. A high-level state captures a participant’s cognitive state transitions during the task, and eye movement patterns during each high-level state are summarized with a regular HMM.
- , "Eye movement analysis with switching hidden Markov models." Behavior Research Methods, 52:1026-1043, June 2020.
We propose a ParametRIc MAnifold Learning (PRIMAL) algorithm for Gaussian Mixtures Models (GMM), assuming that GMMs lie on or near to a manifold that is generated from a low-dimensional hierarchical latent space through parametric mappings. Inspired by Principal Component Analysis (PCA), the generative processes for priors, means and covariance matrices are modeled by
their respective latent space and parametric mapping.
- , "Parametric Manifold Learning of Gaussian Mixture Models." In: International Joint Conference on Artificial Intelligence (IJCAI), Macau, Aug 2019.
Recent Datasets and Code [more]
Eye Movement analysis with Switching HMMs (EMSHMM) Toolbox
This is a MATLAB toolbox for analyzing eye movement data using switching hidden Markov models (SHMMs), for analyzing eye movement data in cognitive tasks involving cognitive state changes. It includes code for learning SHMMs for individuals, as well as analyzing the results.
- Files: download here
- Project page
- If you use this toolbox please cite:
Eye movement analysis with switching hidden Markov models.
,
Behavior Research Methods, 52:1026-1043, June 2020.
EgoDaily – Egocentric dataset for Hand Disambiguation
Egocentric hand detection dataset with variability on people, activities and places, to simulate daily life situations.
- Files: download page
- If you use this dataset please cite:
Is that my hand? An egocentric dataset for hand disambiguation.
,
Image and Vision Computing, 89:131-143, Sept 2019.
CityStreet: Multi-view crowd counting dataset
Datasets for multi-view crowd counting in wide-area scenes. Includes our CityStreet dataset, as well as the counting and metadata for multi-view counting on PETS2009 and DukeMTMC.
- Files: download page
- Project page
- If you use this dataset please cite:
Wide-Area Crowd Counting via Ground-Plane Density Maps and Multi-View Fusion CNNs.
,
In: IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), Long Beach, June 2019.
CityUHK-X: crowd dataset with extrinsic camera parameters
Crowd counting dataset of indoor/outdoor scenes with extrinsic camera parameters (camera angle and height), for use as side information.
- Files: zip (1.8GB) | readme
- Project page
- If you use this dataset please cite:
Incorporating Side Information by Adaptive Convolution.
,
In: Neural Information Processing Systems, Long Beach, Dec 2017.
DPHEM toolbox for simplifying GMMs
Toolboxes for density-preserving HEM algorithm for simplifying mixture models.
- Files: Python toolbox (compatible with sklearn.mixture.GaussianMixture) and Matlab toolbox
- If you use this code please cite:
Density-Preserving Hierarchical EM Algorithm: Simplifying Gaussian Mixture Models for Approximate Inference.
,
IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 41(6):1323-1337, June 2019.