Object-Aware SLAM Based on Efficient Quadric Initialization and Joint Data Association

ZhenZhong Cao; Yunzhou Zhang; Rui Tian; Rong Ma; Xinggang Hu; Sonya Coleman; Dermot Kerr

doi:10.1109/lra.2022.3190622

Object-Aware SLAM Based on Efficient Quadric Initialization and Joint Data Association

ZhenZhong Cao
, Yunzhou Zhang
, Rui Tian
, Rong Ma
, Xinggang Hu
, Sonya Coleman
, Dermot Kerr

Northeastern University China

Research output: Contribution to journal › Article › peer-review

24 Citations (Scopus)

417 Downloads (Pure)

Abstract

Semantic simultaneous localization and mapping (SLAM) is a popular technology enabling indoor mobile robots to sufficiently perceive and interact with the environment. In this paper, we propose an object-aware semantic SLAM system, which consists of a quadric initialization method, an object-level data association method, and a multi-constraint optimization factor graph. To overcome the limitation of multi-view observations and the requirement of dense point clouds for objects, an efficient quadric initialization method based on object detection and surfel construction is proposed, which can efficiently initialize quadrics within fewer frames and with small viewing angles. The robust object-level joint data association method and the tightly coupled multi-constraint factor graph for quadrics optimization and joint bundle adjustment enable the accurate estimation of constructed quadrics and camera poses. Extensive experiments using public datasets show that the proposed system achieves competitive performance with respect to accuracy and robustness of object quadric estimation and camera localization compared with state-of-the-art methods.

Original language	English
Pages (from-to)	9802-9809
Number of pages	8
Journal	IEEE Robotics and Automation Letters
Volume	7
Issue number	4
Early online date	13 Jul 2022
DOIs	https://doi.org/10.1109/lra.2022.3190622
Publication status	Published (in print/issue) - Oct 2022

Bibliographical note

Funding Information:
This work was supported by National Natural Science Foundation of China under Grant 61973066, in part by theMajor Science and Technology Projects of Liaoning Province under Grant 2021JH1/10400049, in part by the Fundation of Key Laboratory of Aerospace System Simulation underGrant 6142002200301, in part by the Fundation ofKey Laboratory of Equipment Reliability under Grant WD2C20205500306, and in part by theFundamental Research Funds for the CentralUniversities underGrant N2004022.

Publisher Copyright:
© 2022 IEEE.

Funding

Funding Information: This work was supported by National Natural Science Foundation of China under Grant 61973066, in part by theMajor Science and Technology Projects of Liaoning Province under Grant 2021JH1/10400049, in part by the Fundation of Key Laboratory of Aerospace System Simulation underGrant 6142002200301, in part by the Fundation ofKey Laboratory of Equipment Reliability under Grant WD2C20205500306, and in part by theFundamental Research Funds for the CentralUniversities underGrant N2004022. Publisher Copyright: © 2022 IEEE.

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 9 Industry, Innovation, and Infrastructure

Keywords

Artificial Intelligence
Control and Optimization
Computer Science Applications
Computer Vision and Pattern Recognition
Mechanical Engineering
Human-Computer Interaction
Biomedical Engineering
Control and Systems Engineering

Access to Document

10.1109/lra.2022.3190622

Object-Aware_SLAM_Based_on_Efficient_Quadric_Initialization_and_Joint_Data_AssociationAuthor Accepted Manuscript, 1.12 MB

Cite this

@article{409228d1aec040b688a52a8f5fae01d3,

title = "Object-Aware SLAM Based on Efficient Quadric Initialization and Joint Data Association",

abstract = "Semantic simultaneous localization and mapping (SLAM) is a popular technology enabling indoor mobile robots to sufficiently perceive and interact with the environment. In this paper, we propose an object-aware semantic SLAM system, which consists of a quadric initialization method, an object-level data association method, and a multi-constraint optimization factor graph. To overcome the limitation of multi-view observations and the requirement of dense point clouds for objects, an efficient quadric initialization method based on object detection and surfel construction is proposed, which can efficiently initialize quadrics within fewer frames and with small viewing angles. The robust object-level joint data association method and the tightly coupled multi-constraint factor graph for quadrics optimization and joint bundle adjustment enable the accurate estimation of constructed quadrics and camera poses. Extensive experiments using public datasets show that the proposed system achieves competitive performance with respect to accuracy and robustness of object quadric estimation and camera localization compared with state-of-the-art methods.",

keywords = "Artificial Intelligence, Control and Optimization, Computer Science Applications, Computer Vision and Pattern Recognition, Mechanical Engineering, Human-Computer Interaction, Biomedical Engineering, Control and Systems Engineering",

author = "ZhenZhong Cao and Yunzhou Zhang and Rui Tian and Rong Ma and Xinggang Hu and Sonya Coleman and Dermot Kerr",

note = "Funding Information: This work was supported by National Natural Science Foundation of China under Grant 61973066, in part by theMajor Science and Technology Projects of Liaoning Province under Grant 2021JH1/10400049, in part by the Fundation of Key Laboratory of Aerospace System Simulation underGrant 6142002200301, in part by the Fundation ofKey Laboratory of Equipment Reliability under Grant WD2C20205500306, and in part by theFundamental Research Funds for the CentralUniversities underGrant N2004022. Publisher Copyright: {\textcopyright} 2022 IEEE.",

year = "2022",

month = oct,

doi = "10.1109/lra.2022.3190622",

language = "English",

volume = "7",

pages = "9802--9809",

journal = "IEEE Robotics and Automation Letters",

issn = "2377-3766",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "4",

}

TY - JOUR

T1 - Object-Aware SLAM Based on Efficient Quadric Initialization and Joint Data Association

AU - Cao, ZhenZhong

AU - Zhang, Yunzhou

AU - Tian, Rui

AU - Ma, Rong

AU - Hu, Xinggang

AU - Coleman, Sonya

AU - Kerr, Dermot

N1 - Funding Information: This work was supported by National Natural Science Foundation of China under Grant 61973066, in part by theMajor Science and Technology Projects of Liaoning Province under Grant 2021JH1/10400049, in part by the Fundation of Key Laboratory of Aerospace System Simulation underGrant 6142002200301, in part by the Fundation ofKey Laboratory of Equipment Reliability under Grant WD2C20205500306, and in part by theFundamental Research Funds for the CentralUniversities underGrant N2004022. Publisher Copyright: © 2022 IEEE.

PY - 2022/10

Y1 - 2022/10

N2 - Semantic simultaneous localization and mapping (SLAM) is a popular technology enabling indoor mobile robots to sufficiently perceive and interact with the environment. In this paper, we propose an object-aware semantic SLAM system, which consists of a quadric initialization method, an object-level data association method, and a multi-constraint optimization factor graph. To overcome the limitation of multi-view observations and the requirement of dense point clouds for objects, an efficient quadric initialization method based on object detection and surfel construction is proposed, which can efficiently initialize quadrics within fewer frames and with small viewing angles. The robust object-level joint data association method and the tightly coupled multi-constraint factor graph for quadrics optimization and joint bundle adjustment enable the accurate estimation of constructed quadrics and camera poses. Extensive experiments using public datasets show that the proposed system achieves competitive performance with respect to accuracy and robustness of object quadric estimation and camera localization compared with state-of-the-art methods.

AB - Semantic simultaneous localization and mapping (SLAM) is a popular technology enabling indoor mobile robots to sufficiently perceive and interact with the environment. In this paper, we propose an object-aware semantic SLAM system, which consists of a quadric initialization method, an object-level data association method, and a multi-constraint optimization factor graph. To overcome the limitation of multi-view observations and the requirement of dense point clouds for objects, an efficient quadric initialization method based on object detection and surfel construction is proposed, which can efficiently initialize quadrics within fewer frames and with small viewing angles. The robust object-level joint data association method and the tightly coupled multi-constraint factor graph for quadrics optimization and joint bundle adjustment enable the accurate estimation of constructed quadrics and camera poses. Extensive experiments using public datasets show that the proposed system achieves competitive performance with respect to accuracy and robustness of object quadric estimation and camera localization compared with state-of-the-art methods.

KW - Artificial Intelligence

KW - Control and Optimization

KW - Computer Science Applications

KW - Computer Vision and Pattern Recognition

KW - Mechanical Engineering

KW - Human-Computer Interaction

KW - Biomedical Engineering

KW - Control and Systems Engineering

UR - https://www.scopus.com/pages/publications/85134263324

U2 - 10.1109/lra.2022.3190622

DO - 10.1109/lra.2022.3190622

M3 - Article

SN - 2377-3766

VL - 7

SP - 9802

EP - 9809

JO - IEEE Robotics and Automation Letters

JF - IEEE Robotics and Automation Letters

IS - 4

ER -

Object-Aware SLAM Based on Efficient Quadric Initialization and Joint Data Association

Abstract

Bibliographical note

Funding

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Object-Aware SLAM Based on Efficient Quadric Initialization and Joint Data Association

Cite this