IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2022

Large-Field Contextual Feature Learning for Glass Detection

Haiyang Mei¹ Xin Yang^1,* Letian Yu¹ Qiang Zhang¹ Xiaopeng Wei^1,* Rynson W.H. Lau^2,*

¹ Dalian University of Technology ² City University of Hong Kong

Contact us: xinyang@dlut.edu.cn mhy666@mail.dlut.edu.cn

1. Abstract

Glass is very common in our daily life. Existing computer vision systems neglect it and thus may have severe consequences, e.g., a robot may crash into a glass wall. However, sensing the presence of glass is not straightforward. The key challenge is that arbitrary objects/scenes can appear behind the glass. In this paper, we propose an important problem of detecting glass surfaces from a single RGB image. To address this problem, we construct the first large-scale glass detection dataset (GDD) and propose a novel glass detection network, called GDNet-B, which explores abundant contextual cues in a large field-of-view via a novel large-field contextual feature integration (LCFI) module and integrates both high-level and low-level boundary features with a boundary feature enhancement (BFE) module. Extensive experiments demonstrate that our GDNet-B achieves satisfying glass detection results on the images within and beyond the GDD testing set. We further validate the effectiveness and generalization capability of our proposed GDNet-B by applying it to other vision tasks, including mirror segmentation and salient object detection. Finally, we show the potential applications of glass detection and discuss possible future research directions.

2. Boundary Feature Enhancement (BFE)

Since the boundary is a strong cue for humans to distinguish a glass region, we further introduce the boundary cue to help boost the glass detection accuracy. Integrating boundary information can help glass detection in both segmentation and localization. In BFE, the input features are passed through four parallel branches, and the outputs of all branches are fused to generate glass boundary features, which are then used to predict the glass boundary map and complement the input features.

3. Downloads

Model	: [ Google Drive ] [ Baidu Disk, fetch code: pami ]
Results	: [ Google Drive ] [ Baidu Disk, fetch code: pami ]
Code	: [ Github ]

4. Applications

Automatic glass detection has various possible applications. Here, we envision two potential ones: obstacle avoidance for drones and intelligence photography/editing. If a drone is equipped with the ability to detect glass, it can easily find out where the glass is and avoid crashing into the glass (e.g., Figure (b)). In addition, if a phone camera is equipped with the ability to detect glass, it can identify the glass and then remove the reflections from the glass region to obtain the desired result (e.g., Figure (d)).

5. BibTex

@article{Haiyang:GDNet-B:2022,
    author = {Mei, Haiyang and Yang, Xin and Yu, Letian and Zhang, Qiang and Wei, Xiaopeng and Lau, Rynson W. H.},
    title = {Large-Field Contextual Feature Learning for Glass Detection},
    booktitle = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
    pages={1-17},,
    year = {2022}
}