About

Welcome to the Video, Image, and Sound Analysis Lab (VISAL) at the City University of Hong Kong! The lab is directed by Prof. Antoni Chan in the Department of Computer Science.

Our main research activities include:

  • Computer Vision, Surveillance
  • Machine Learning, Pattern Recognition
  • Computer Audition, Music Information Retrieval
  • Eye Gaze Analysis

For more information about our current research, please visit the projects and publication pages.

Opportunities for graduate students and research assistants – if you are interested in joining the lab, please check this information.

Latest News [more]

  • [Apr 27, 2026]

    Congratulations to Weibo Shu for defending his thesis!

  • [Jan 13, 2026]

    Congratulations to Wei Lin for defending his thesis!

  • [May 27, 2025]

    Congratulations to Chenyang for defending her thesis!

  • [Feb 11, 2025]

    Congratulations to Jiuniu for defending his thesis!

Recent Publications [more]

  • SinkRouter: Sink-Aware Routing for Efficient Long-Context Decoding in Large Language and Multimodal Models.
    Junnan Liu, Xinyan Liu, Peifeng Gao, Zhaobo Qi, Beichen Zhang, Weigang Zhang, and Antoni B. Chan,
    In: ACM Multimedia (MM), Rio de Janeiro, Nov 2026. [github]
  • Depth-Guided Class-Agnostic Individual Counting in Videos.
    Yuanjing Xu, Xinyan Liu, Weidong Chen, Zixuan Zou, Linhao Zhang, Zhuangzhe Meng, Antoni B. Chan, and Weigang Zhang,
    In: ACM Multimedia (MM), Nov 2026. [github]
  • A Multi-Modal Per-Pair Pipeline for UAV Pose Estimation in Orbital Image Sets.
    Xinyan Liu, Weigang Zhang, Weidong Chen, Zhaobo Qi, Beichen Zhang, and Antoni B. Chan,
    In: ACM Multimedia Workshop UAVs in Multimedia, Rio de Janeiro, Nov 2026.
  • SMANet: Probabilistic Gating and Neighborhood Attention for Video Individual Counting.
    Pengqi Huang, Xinyan Liu, Difan Zou, Weidong Chen, Weigang Zhang, Qingming Huang, and Antoni B. Chan,
    In: Intl Conf on Image and Graphics (ICIG), Singapore, Oct 2026. [github]
  • DMBG-RWKV: Adjacent Depth Mixing and Boundary-guided Feature Harmonization for Medical Image Segmentation.
    Tianzheng Xu, Xinyan Liu, Pengqi Huang, Xinfeng Zhang, Weidong Chen, Weigang Zhang, Qingming Huang, and Antoni B. Chan,
    In: Intl Conf on Image and Graphics (ICIG), Singapore, Oct 2026. [github]
  • Aligning Prototypes and Updating Null Spaces for Continual WSI Learning.
    Xianrui Li, Kaiwen Xiao, and Antoni B. Chan,
    In: 29th Intl Conf on Medical Image Computing and Computer Assisted Intervention (MICCAI), Strasbourg, Sept 2026.
  • Exclusivity-Guided Mask Learning for Semi-Supervised Crowd Instance Segmentation and Counting.
    Jiyang Huang, Hongru Chen, Wei Lin, Jia Wan, and Antoni B. Chan,
    In: European Conference on Computer Vision (ECCV), Malmö, Sweden, Sept 2026 (oral).
  • Grad-ECLIP: Gradient-based Visual and Textual Explanations for CLIP.
    Chenyang Zhao, Kun Wang, Janet H. Hsiao, and Antoni B. Chan,
    IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), accepted 2026 (online July 2026). [github]
  • Semantic bias in image-text matching in humans versus vision-language pretraining AI models.
    Jinhan Zhang, Qichun Duan, Chenyang Zhao, Antoni B. Chan, and Janet H. Hsiao,
    In: Annual Conference of the Cognitive Science Society (CogSci), Rio de Janeiro, Jul 2026.
  • Identifying Mind Wandering Episodes during Virtual Cognitive Stimulation Therapy through Gaze Estimation from Videos.
    Xiaoru Teng, Yong Li, Gloria H.Y. Wong, Antoni B. Chan, and Janet H. Hsiao,
    In: Annual Conference of the Cognitive Science Society (CogSci), Rio de Janeiro, Jul 2026.

Recent Project Pages [more]

Continual Learning MIL

We pinpoint catastrophic forgetting to the attention layers of attention-MIL models for whole-slide images and introduce two remedies: Attention Knowledge Distillation (AKD) to retain attention weights across tasks and a Pseudo-Bag Memory Pool (PMP) that keeps only the most informative patches. Combined, AKD and PMP achieve state-of-the-art continual-learning accuracy while sharply cutting memory usage on diverse WSI datasets.

Image Editing with Diffusion Model from Frequency Perspective

We introduce a novel fine-tuning free approach that employs progressive Frequency truncation to refine the guidance of Diffusion models for universal editing tasks (FreeDiff).

DistinctAD: Distinctive Audio Description Generation in Contexts

We propose a two-stage framework DistinctAD for automatically generating audio descriptions in movies or tv series. DistinctAD targets at generating distinctive and interesting ADs in similar contextual video clips.

P2R Loss for Semi-Supervised Counting

We introduce a Point-to-Region (P2R) loss to address the over-activation and pseudo-label propagation issues inherent in semi-supervised crowd counting. By replacing pixel-level matching with region-level supervision, P2R suppresses background noise and achieves state-of-the-art results with significantly higher training stability.

Proximal Mapping Loss for Crowd Counting

We propose the Proximal Mapping Loss (PML), a theoretically grounded framework that discards the unrealistic “non-overlap” assumption common in crowd counting. By leveraging proximal operators from convex optimization, PML accurately recovers density in highly congested scenes where severe occlusions and overlapping objects are prevalent.

Recent Datasets and Code [more]

Modeling Eye Movements with Deep Neural Networks and Hidden Markov Models (DNN+HMM)

This is the toolbox for modeling eye movements and feature learning with deep neural networks and hidden Markov models (DNN+HMM).

Dolphin-14k: Chinese White Dolphin detection dataset

A dataset consisting of  Chinese White Dolphin (CWD) and distractors for detection tasks.

Crowd counting: Zero-shot cross-domain counting

Generalized loss function for crowd counting.

CVCS: Cross-View Cross-Scene Multi-View Crowd Counting Dataset

Synthetic dataset for cross-view cross-scene multi-view counting. The dataset contains 31 scenes, each with about ~100 camera views. For each scene, we capture 100 multi-view images of crowds.

Crowd counting: Generalized loss function

Generalized loss function for crowd counting.