Visual Servoing of a Redundant Manipulator with Jacobian Matrix Estimation using Self-organizing Map

Prem Kumar Patchaikani, Laxmidhar Behera

    Research output: Contribution to journalArticlepeer-review

    39 Citations (Scopus)


    Vision based redundant manipulator control with a neural network based learning strategy is discussed in this paper. The manipulator is visually controlled with stereo vision in an eye-to-hand configuration. A novel Kohonen's self-organizing map (KSOM) based visual servoing scheme has been proposed for a redundant manipulator with 7 degrees of freedom (DOF). The inverse kinematic relationship of the manipulator is learned using a Kohonen's self-organizing map. This learned map is shown to be an approximate estimate of the inverse Jacobian, which can then be used in conjunction with the proportional controller to achieve closed loop servoing in real-time. It is shown through Lyapunov stability analysis that the proposed learning based servoing scheme ensures global stability. A generalized weightupdate law is proposed for KSOM based inverse kinematic control, to resolve the redundancy during the learning phase. Unlike the existing visual servoing schemes, the proposed KSOM based scheme eliminates the computation of the pseudo-inverse of the Jacobian matrix in real-time. This makes the proposed algorithm computationally more efficient. The proposed scheme has been implemented on a 7 DOF PowerCubeTM robot manipulator with visual feedback from two cameras.
    Original languageEnglish
    Pages (from-to)978-990
    JournalRobotics and Autonomous Systems
    Issue number8
    Publication statusPublished (in print/issue) - Aug 2010

    Bibliographical note

    Reference text: [1] S. Hutchinson, G.D. Hager, P.I. Corke, A tutorial on visual servo control, IEEE
    Transactions on Robotics and Automation 12 (5) (1996) 651670.
    [2] E. Malis, S. Benhimane, A unified approach to visual tracking and servoing,
    Robotics and Autonomous Systems 52 (1) (2005) 3952.
    [3] D. Kragic, H.I. Christensen, Cue integration for visual servoing, IEEE Transac-
    tions on Robotics and Automation 17 (1) (2001) 1827.
    [4] E. Marchand, F. Chaumette, Statistically robust 2-D visual servoing, IEEE
    Transactions on Robotics and Automation 22 (2) (2006) 415420.
    [5] W.-C. Chang, Precise positioning of binocular eye-to-hand robotic manipula-
    tors, Journal of Intelligent and Robotics Systems 49 (2007) 219236.
    [6] L. Behera, N. Kirubanandan, A hybrid neural control scheme for visual-motor
    coordination, IEEE Control System Magazine 19 (4) (1999) 3441.
    [7] F. Chaumette, S. Hutchinson, Visual servo control part I: basic approaches, IEEE
    Robotics and Automation Magazine 13 (4) (2006) 8290.
    [8] F. Chaumette, S. Hutchinson, Visual servo control part II: advanced approaches
    (tutorial), IEEE Robotics and Automation Magazine 14 (1) (2007) 109118.
    [9] D. Kragic, H. Christensen, Survey on visual servoing for manipulation,
    [10] F. Chaumette, Potential problems of stability and convergence in image based
    and position based visual servoing, in: D. Kreigman, G. Hager, S. Morse (Eds.),
    T. Confluence of Vision, Control, in: Lecture notes in Control and Information
    Sciences, vol. 237, Springer-Verlag, New York, 1998, pp. 6678.
    [11] J.-T. Lapreste, F. Jurie, M. Dhome, F. Chaumette, An efficient method to
    compute the inverse Jacobian matrix in visual servoing, in: Proceedings of
    IEEE International Conference on Robotics and Automation, vol. 1, IEEE, New
    Orleans, LA, 2004, pp. 727732.
    [12] B. Siciliano, Kinematic control of redundant robot manipulators: a tutorial,
    Journal of Intelligent and Robotic systems 4 (4) (1990) 201212.
    [13] D.E. DeMers, K.K. Kreutz-Delgado, Solving the inverse kinematics problem for
    robots with excess degrees-of-freedom,
    [14] F. Chaumette, E. Marchand, A redundancy-based iterative approach for
    avoiding joint limits: application to visual servoing, IEEE Transactions on
    Robotics and Automation 17 (5) (2001) 719730.

    [15] T. Martinetz, H. Ritter, K. Schulten, Learning of visuomotor-coordination of a
    robot arm with redundant degrees of freedom, in: Proceedings of the Int. Conf.
    on Parallel Processing in Neural Systems and Computers, 1990, pp. 431434.
    [16] M. Han, N. Okada, E. Kondo, Coordination of an uncalibrated 3-D visuo-motor
    system based on multiple self-organizing maps, JSME International Journal,
    Series C 49 (1) (2006) 230239.
    [17] S. Kumar, P. Prem Kumar, A. Dutta, L. Behera, Visual motor control of a
    7DOF redundant manipulator using redundancy preserving learning network,
    Robotica (2009) 116 (first view).
    [18] G.D.A. Barreto, A.F.R. Araujo, Self-organizing feature maps for modeling and
    control of robtoic manipulators, Journal of Intelligent and Robotic Systems 36
    (2003) 407450.
    [19] F. Chaumette, Image moments: a general and useful set of features for
    visual servoing, IEEE Transactions on Robotics and Automation 20 (4) (2004)
    [20] Amtec robotics,
    [21] Unibrain,
    [22] R.Y. Tsai, A versatile camera calibration technique for high-accuracy 3D
    machine vision metrology using off-the-shelf TV cameras and lenses, IEEE
    Journal of Robotics and Automation RA-3 (4) (1987) 323344.
    [23] T. Hesselroth, K. Sarkar, P.P. Smagt, K. Schulten, Neural network control of a
    pneumatic robot arm, IEEE Transactions on Systems, Man and Cybernetics 24
    (1) (1994) 2838.
    [24] Y. Zhang, J. Wang, Y. Xu, A dual neural network bi-criteria kinematic control
    of redundant manipulators, IEEE Transactions on Robotics and Automation 18
    (6) (2002) 923931.
    [25] T.F. Chan, R.V. Dubey, A weighted least-norm based solution scheme for
    avoiding joint limits for redundant joint manipulators, IEEE Transactions on
    Robotics and Automation 11 (2) (1995) 286292.
    [26] R. Wilson, Tsai camera calibration software,


    • Redundant manipulator
    • Visual servoing
    • Inverse Jacobian
    • Self-organizing map
    • Kinematic control
    • Redundancy resolution


    Dive into the research topics of 'Visual Servoing of a Redundant Manipulator with Jacobian Matrix Estimation using Self-organizing Map'. Together they form a unique fingerprint.

    Cite this