Query-level gating
Classification-guided thresholding removes low-confidence queries in both training and inference. L2-normalized embeddings decouple similarity from feature magnitude.
Online Video Instance Segmentation (VIS) relies on per-frame instance queries for causal detection, segmentation, and temporal association. However, in realistic videos with occlusion, blur, and clutter, these predictions often become noisy, exhibiting false positives, missed instances, or inaccurate masks. Such noisy queries destabilize cross-frame association, contaminate contrastive supervision, and propagate drift through memory-based matching.
We introduce MiNQVIS, the first noise-aware framework for online VIS that stabilizes both training and inference. It combines query-level gating with embedding-norm stabilization, video-level prototypical contrastive learning (VPCL) from reliable matches, and dual-memory association that fuses momentum embeddings with history-best retrieval. Experiments on YouTubeVIS 2019/2021 and OVIS show consistent improvements over strong online VIS baselines, with large gains in challenging scenes.
MiNQVIS builds on a standard Mask2Former-based online VIS pipeline and introduces three complementary components to prevent unreliable queries from affecting supervision and temporal association.
Classification-guided thresholding removes low-confidence queries in both training and inference. L2-normalized embeddings decouple similarity from feature magnitude.
VPCL aggregates reliable cross-frame matches into stable instance prototypes, reducing the influence of noisy frames on contrastive supervision.
Momentum memory captures short-term continuity, while history-best retrieval preserves a reliable reference when recent observations are corrupted.
Query-level gating explicitly filters unreliable predictions and stabilizes similarity through embedding normalization.
MiNQVIS is evaluated on YouTubeVIS 2019, YouTubeVIS 2021, and OVIS with ResNet-50 and Swin-L backbones. It achieves the best OVIS AP under both backbones, demonstrating robust association in crowded and heavily occluded scenes.
| Backbone | YTVIS19 AP | YTVIS21 AP | OVIS AP |
|---|---|---|---|
| ResNet-50 | 55.6 | 51.7 | 38.2 |
| Swin-L | 66.2 | 61.6 | 49.1 |
| Query-level gating | VPCL | Dual-memory fusion | YTVIS21 AP | OVIS AP |
|---|---|---|---|---|
| 47.5 | 34.7 | |||
| ✓ | 49.8 | 35.9 | ||
| ✓ | 48.7 | 36.3 | ||
| ✓ | 48.1 | 35.5 | ||
| ✓ | ✓ | 51.4 | 37.8 | |
| ✓ | ✓ | ✓ | 51.7 | 38.2 |
Most filtered queries are false positives or fragmented masks. In this OVIS example, the gate removes 193 unreliable queries while retaining all valid foreground instances.
@article{qiao2026minqvis,
author = {Qiao, Jie and Chen, Jianxu and Xu, Xiaowei},
title = {MiNQVIS: Mitigating Noisy Queries for Robust Online Video Instance Segmentation},
year = {2026}
}