The IntelliMedia Workbench - an environment for building multimodal systems

T Broendsted, P Dalsgaard, LB Larsen, M Manthey, P McKevitt, TB Moeslund, KG Olesen

Research output: Chapter in Book/Report/Conference proceedingChapter

1 Citation (Scopus)

Abstract

Intelligent MultiMedia (IntelliMedia) focuses on the computer processing and understanding of signal and symbol input from at least speech, text and visual images in terms of semantic representations. We have developed a general suite of tools in the form of a software and hardware platform called “CHAMELEON” that can be tailored to conducting IntelliMedia in various application domains. Chameleon has an open distributed processing architecture and currently includes ten agent modules: blackboard, dialogue manager, domain model, gesture recogniser, laser system, microphone array, speech recogniser, speech synthesiser, natural language processor, and a distributed Topsy learner. Most of the modules are programmed in C and C++ and are glued together using the DACS communications system. In effect, the blackboard, dialogue manager and DACS form the kernel of Chameleon. Modules can communicate with each other and the blackboard which keeps a record of interactions over time via semantic representations in frames. Inputs to Chameleon can include synchronised spoken dialogue and images and outputs include synchronised laser pointing and spoken dialogue. An initial prototype application of CHAMELEON is an IntelliMedia WorkBench where a user will be able to ask for information about things (e.g. 2D/3D models, pictures, objects, gadgets, people, or whatever) on a physical table. The current domain is a Campus Information System for 2D building plans which provides information about tenants, rooms and routes and can answer questions like Whose office is this? and Show me the route from Paul Mc Kevitt’s office to Paul Dalsgaard’s office. in real time. Chameleon and the IntelliMedia WorkBench are ideal for testing integrated signal and symbol processing of language and vision for the future of SuperinformationhighwayS.
LanguageEnglish
Title of host publicationCooperative Multimodal Communication: Second International Conference, CMC'98, Tilburg, The Netherlands, January 1998, Selected Papers
EditorsH Bunt, R-J Beun
Place of PublicationBerlin, Germany
Pages217-233
VolumeLNAI 2
DOIs
Publication statusPublished - 28 Nov 2001

Fingerprint

Managers
Processing
Semantics
Lasers
Microphones
Communication systems
Information systems
Hardware
Testing

Cite this

Broendsted, T., Dalsgaard, P., Larsen, LB., Manthey, M., McKevitt, P., Moeslund, TB., & Olesen, KG. (2001). The IntelliMedia Workbench - an environment for building multimodal systems. In H. Bunt, & R-J. Beun (Eds.), Cooperative Multimodal Communication: Second International Conference, CMC'98, Tilburg, The Netherlands, January 1998, Selected Papers (Vol. LNAI 2, pp. 217-233). Berlin, Germany. https://doi.org/10.1007/3-540-45520-5
Broendsted, T ; Dalsgaard, P ; Larsen, LB ; Manthey, M ; McKevitt, P ; Moeslund, TB ; Olesen, KG. / The IntelliMedia Workbench - an environment for building multimodal systems. Cooperative Multimodal Communication: Second International Conference, CMC'98, Tilburg, The Netherlands, January 1998, Selected Papers. editor / H Bunt ; R-J Beun. Vol. LNAI 2 Berlin, Germany, 2001. pp. 217-233
@inbook{6cedd8324abb44aeb6ff4b02dd9573b5,
title = "The IntelliMedia Workbench - an environment for building multimodal systems",
abstract = "Intelligent MultiMedia (IntelliMedia) focuses on the computer processing and understanding of signal and symbol input from at least speech, text and visual images in terms of semantic representations. We have developed a general suite of tools in the form of a software and hardware platform called “CHAMELEON” that can be tailored to conducting IntelliMedia in various application domains. Chameleon has an open distributed processing architecture and currently includes ten agent modules: blackboard, dialogue manager, domain model, gesture recogniser, laser system, microphone array, speech recogniser, speech synthesiser, natural language processor, and a distributed Topsy learner. Most of the modules are programmed in C and C++ and are glued together using the DACS communications system. In effect, the blackboard, dialogue manager and DACS form the kernel of Chameleon. Modules can communicate with each other and the blackboard which keeps a record of interactions over time via semantic representations in frames. Inputs to Chameleon can include synchronised spoken dialogue and images and outputs include synchronised laser pointing and spoken dialogue. An initial prototype application of CHAMELEON is an IntelliMedia WorkBench where a user will be able to ask for information about things (e.g. 2D/3D models, pictures, objects, gadgets, people, or whatever) on a physical table. The current domain is a Campus Information System for 2D building plans which provides information about tenants, rooms and routes and can answer questions like Whose office is this? and Show me the route from Paul Mc Kevitt’s office to Paul Dalsgaard’s office. in real time. Chameleon and the IntelliMedia WorkBench are ideal for testing integrated signal and symbol processing of language and vision for the future of SuperinformationhighwayS.",
author = "T Broendsted and P Dalsgaard and LB Larsen and M Manthey and P McKevitt and TB Moeslund and KG Olesen",
year = "2001",
month = "11",
day = "28",
doi = "10.1007/3-540-45520-5",
language = "English",
isbn = "978-3-540-42806-2",
volume = "LNAI 2",
pages = "217--233",
editor = "H Bunt and R-J Beun",
booktitle = "Cooperative Multimodal Communication: Second International Conference, CMC'98, Tilburg, The Netherlands, January 1998, Selected Papers",

}

Broendsted, T, Dalsgaard, P, Larsen, LB, Manthey, M, McKevitt, P, Moeslund, TB & Olesen, KG 2001, The IntelliMedia Workbench - an environment for building multimodal systems. in H Bunt & R-J Beun (eds), Cooperative Multimodal Communication: Second International Conference, CMC'98, Tilburg, The Netherlands, January 1998, Selected Papers. vol. LNAI 2, Berlin, Germany, pp. 217-233. https://doi.org/10.1007/3-540-45520-5

The IntelliMedia Workbench - an environment for building multimodal systems. / Broendsted, T; Dalsgaard, P; Larsen, LB; Manthey, M; McKevitt, P; Moeslund, TB; Olesen, KG.

Cooperative Multimodal Communication: Second International Conference, CMC'98, Tilburg, The Netherlands, January 1998, Selected Papers. ed. / H Bunt; R-J Beun. Vol. LNAI 2 Berlin, Germany, 2001. p. 217-233.

Research output: Chapter in Book/Report/Conference proceedingChapter

TY - CHAP

T1 - The IntelliMedia Workbench - an environment for building multimodal systems

AU - Broendsted, T

AU - Dalsgaard, P

AU - Larsen, LB

AU - Manthey, M

AU - McKevitt, P

AU - Moeslund, TB

AU - Olesen, KG

PY - 2001/11/28

Y1 - 2001/11/28

N2 - Intelligent MultiMedia (IntelliMedia) focuses on the computer processing and understanding of signal and symbol input from at least speech, text and visual images in terms of semantic representations. We have developed a general suite of tools in the form of a software and hardware platform called “CHAMELEON” that can be tailored to conducting IntelliMedia in various application domains. Chameleon has an open distributed processing architecture and currently includes ten agent modules: blackboard, dialogue manager, domain model, gesture recogniser, laser system, microphone array, speech recogniser, speech synthesiser, natural language processor, and a distributed Topsy learner. Most of the modules are programmed in C and C++ and are glued together using the DACS communications system. In effect, the blackboard, dialogue manager and DACS form the kernel of Chameleon. Modules can communicate with each other and the blackboard which keeps a record of interactions over time via semantic representations in frames. Inputs to Chameleon can include synchronised spoken dialogue and images and outputs include synchronised laser pointing and spoken dialogue. An initial prototype application of CHAMELEON is an IntelliMedia WorkBench where a user will be able to ask for information about things (e.g. 2D/3D models, pictures, objects, gadgets, people, or whatever) on a physical table. The current domain is a Campus Information System for 2D building plans which provides information about tenants, rooms and routes and can answer questions like Whose office is this? and Show me the route from Paul Mc Kevitt’s office to Paul Dalsgaard’s office. in real time. Chameleon and the IntelliMedia WorkBench are ideal for testing integrated signal and symbol processing of language and vision for the future of SuperinformationhighwayS.

AB - Intelligent MultiMedia (IntelliMedia) focuses on the computer processing and understanding of signal and symbol input from at least speech, text and visual images in terms of semantic representations. We have developed a general suite of tools in the form of a software and hardware platform called “CHAMELEON” that can be tailored to conducting IntelliMedia in various application domains. Chameleon has an open distributed processing architecture and currently includes ten agent modules: blackboard, dialogue manager, domain model, gesture recogniser, laser system, microphone array, speech recogniser, speech synthesiser, natural language processor, and a distributed Topsy learner. Most of the modules are programmed in C and C++ and are glued together using the DACS communications system. In effect, the blackboard, dialogue manager and DACS form the kernel of Chameleon. Modules can communicate with each other and the blackboard which keeps a record of interactions over time via semantic representations in frames. Inputs to Chameleon can include synchronised spoken dialogue and images and outputs include synchronised laser pointing and spoken dialogue. An initial prototype application of CHAMELEON is an IntelliMedia WorkBench where a user will be able to ask for information about things (e.g. 2D/3D models, pictures, objects, gadgets, people, or whatever) on a physical table. The current domain is a Campus Information System for 2D building plans which provides information about tenants, rooms and routes and can answer questions like Whose office is this? and Show me the route from Paul Mc Kevitt’s office to Paul Dalsgaard’s office. in real time. Chameleon and the IntelliMedia WorkBench are ideal for testing integrated signal and symbol processing of language and vision for the future of SuperinformationhighwayS.

U2 - 10.1007/3-540-45520-5

DO - 10.1007/3-540-45520-5

M3 - Chapter

SN - 978-3-540-42806-2

VL - LNAI 2

SP - 217

EP - 233

BT - Cooperative Multimodal Communication: Second International Conference, CMC'98, Tilburg, The Netherlands, January 1998, Selected Papers

A2 - Bunt, H

A2 - Beun, R-J

CY - Berlin, Germany

ER -

Broendsted T, Dalsgaard P, Larsen LB, Manthey M, McKevitt P, Moeslund TB et al. The IntelliMedia Workbench - an environment for building multimodal systems. In Bunt H, Beun R-J, editors, Cooperative Multimodal Communication: Second International Conference, CMC'98, Tilburg, The Netherlands, January 1998, Selected Papers. Vol. LNAI 2. Berlin, Germany. 2001. p. 217-233 https://doi.org/10.1007/3-540-45520-5