Autonomous Grasping of 3-D Objects by a Vision-Actuated Robot Arm using Brain-Computer Interface

Arnab Rakshit, Shraman Pramanick, Anurag Bagchi, Saugat Bhattacharyya

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)
15 Downloads (Pure)


A major drawback of a Brain–Computer Interface-based robotic manipulation is the complex trajectory planning of the robot arm to be carried out by the user for reaching and grasping an object. The present paper proposes an intelligent solution to the existing problem by incorporating a novel Convolutional Neural Network (CNN)-based grasp detection network that enables the robot to reach and grasp the desired object (including overlapping objects) autonomously using a RGB-D camera. This network uses a simultaneous object and grasp detection to affiliate each estimated grasp with its corresponding object. The subject uses motor imagery brain signals to control the pan and tilt angle of a RGB-D camera mounted on a robot link to bring the desired object inside its Field-of-view presented through a display screen while the objects appearing on the screen are selected using the P300 brain pattern. The robot uses inverse kinematics along with the RGB-D camera information to autonomously reach the selected object and the object is grasped using proposed grasping strategy. The overall BCI system outperforms other comparative systems involving manual trajectory planning significantly. The overall accuracy, steady-state error, and settling time of the proposed system are 93.4%, 0.05%, and 15.92 s, respectively. The system also shows a significant reduction of the workload of the operating subjects in comparison to manual trajectory planning based approaches for reaching and grasping.
Original languageEnglish
Article number104765
Pages (from-to)1-16
Number of pages17
JournalBiomedical Signal Processing and Control
Early online date7 Mar 2023
Publication statusPublished (in print/issue) - 31 Jul 2023

Bibliographical note

Funding Information:
We thank Prof. Amit Konar, Head of the Artificial Intelligence lab, Jadavpur University, India, for providing us the necessary setup and facilities to conduct the experiment. Authors are also thankful to Laishram Rokesh Singh, (CAE Engineer, ARTPARK) for providing us with the robot gripper.

Publisher Copyright:
© 2023 Elsevier Ltd


  • Brain-Computer Interface
  • Electroencephalogram
  • Robot
  • Convolutional Neural Network
  • Overlapping Object GraspNet
  • Brain–Computer Interface


Dive into the research topics of 'Autonomous Grasping of 3-D Objects by a Vision-Actuated Robot Arm using Brain-Computer Interface'. Together they form a unique fingerprint.

Cite this