QSRNet: Estimating Qualitative Spatial Representations from RGB-D Images
Author(s)
Lee, Sang Uk; Hong, Sungkweon; Hofmann, Andreas; Williams, Brian
DownloadAccepted version (3.784Mb)
Open Access Policy
Open Access Policy
Creative Commons Attribution-Noncommercial-Share Alike
Terms of use
Metadata
Show full item recordAbstract
© 2020 IEEE. Humans perceive and describe their surroundings with qualitative statements (e.g., "Alice's hand is in contact with a bottle."), rather than quantitative values (e.g., 6-D poses of Alice's hand and a bottle). Qualitative spatial representation (QSR) is a framework that represents the spatial information of objects in a qualitative manner. Region connection calculus (RCC), qualitative trajectory calculus (QTC), and qualitative distance calculus (QDC) are some popular QSR calculi. With the recent development of computer vision, it is important to compute QSR calculi from the visual inputs (e.g., RGB-D images). In fact, many QSR application domains (e.g., human activity recognition (HAR) in robotics) involve visual inputs. We propose a qualitative spatial representation network (QSRNet) that computes the three QSR calculi (i.e., RCC, QTC, and QDC) from the RGB-D images. QSRNet has the following novel contributions. First, QSRNet models the dependencies among the three QSR calculi. We introduce the dependencies as kinematics for QSR because they are analogous to the kinematics in classical mechanics. Second, QSRNet applies the 3-D point cloud instance segmentation to compute the QSR calculi. The experimental results show that QSRNet improves the accuracy in comparison to the other state-of-the-art techniques.
Date issued
2020Department
Massachusetts Institute of Technology. Department of Aeronautics and AstronauticsJournal
IEEE International Conference on Intelligent Robots and Systems
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Lee, Sang Uk, Hong, Sungkweon, Hofmann, Andreas and Williams, Brian. 2020. "QSRNet: Estimating Qualitative Spatial Representations from RGB-D Images." IEEE International Conference on Intelligent Robots and Systems.
Version: Author's final manuscript