Tutorial 7: Distributed data fusion for interactive cognitive environments

Presented by

Carlo Regazzoni, Lucio Marcenaro


The tutorial aims at providing an overview of data fusion techniques necessary for representing, modeling and automatically interpreting complex interaction situations occurring in interactive cognitive environments starting from observations provided by a distributed network of embedded systems interacting with the observed environment. A Bayesian oriented viewpoint is assumed in this tutorial.

The main objective is to describe a common framework based on distributed data fusion, for multi-level joint tracking and classification of objects and interactions. The distributed data fusion theory is applied to the problem of recognizing multiple interacting objects from multi-camera video sequences. In particular, the attention will be focused on those actions involving more than one subject, or a subject and elements in the environment, i.e. interactions. This approach is quite novel in the domain of human activity recognition since most methods rely just on the analysis of body motion of single subjects but they don't take into account the context where these situations take place. To this goal, video processing algorithms aiming at improving tracking performances will also be described, both for detection of subjects and body parts. Collaborative trackers, being designed to exchange information between each other in order to produce more reliable models, are a promising approach to tackle typical issues due to occlusions happening during interaction among multiple subjects.

The tutorial will demonstrate how the same distributed data fusion theories can be successfully applied at a higher level for classifying interactions between subjects on a larger scale. Object classification is a fundamental step in automatic video tracking which allows improved tracking and a more accurate description of events. The hypothesis for object classification is based on feature selection method which gives a good subset of features while the machine learns the classification task and use these selected features for object classification.

The tutorial is divided into the following parts:

  1. Introduction on Interactive and Cognitive Environments (C.Regazzoni)
  2. Data fusion architectures and models (L.Marcenaro)
  3. Probabilistic Graphical Models and Dynamic Bayesian Networks for interactions modeling (C.Regazzoni)
  4. Examples and case studies (L.Marcenaro)

Target audience:

This tutorial is specifically intended for researchers studying signal processing in interactive and cognitive environments. Data fusion plays an important role for developing this kind of systems. Typical audience of this tutorial comprehends experts studying methods that involve distributed data sources from small to large sensors. The audience should have a working knowledge of mathematics and probability theory. Some familiarity with data fusion and tracking is desirable but not required.

Speaker Biography

Carlo S. Regazzoni received the the Laurea degree in Electronic Engineering and the Ph.D. in Telecommunications and Signal Processing from the University of Genoa (UniGE), in 1987 and 1992, respectively. Since 2005 he is Full Professor of Telecommunications Systems.

Dr.Regazzoni is involved in research on Signal and Video processing and Data Fusion in Cognitive Telecommunication Systems since 1988. His main current research interests are: Bio-inspired Signal and Video Processing and Recognition, Distributed Data Fusion, Signal Processing for Wireless Communications and Localization, Ambient Intelligence, Cognitive Radio, Multimodal Intelligent Interfaces, Pervasive adaptation in embodied cognitive systems. Since 1998 he is responsible of the Video and Signal Processing for Telecommunications (ISIP40, http://www.isip40.it) Research Group at the Department of Biophysical and Electronic Engineering (DIBE), within the Engineering Faculty of UniGE. Intelligent Distributed Video Surveillance and Wireless Mobile Communications/ Interaction Systems are the main focus of applications studied in this group.

Dr. Regazzoni has been project responsible for DIBE in several EU research and development projects and of several research contracts with italian industries. He is responsible of joint research labs with Technoaware (A2lab /Ambient Awareness Lab), Telecom Italia, and Selex Communications and of the Cognitive Radio Lab (CorLab) at DIBE.

Dr. Regazzoni since 2009 is coordinator of the Erasmus Mundus Joint Doctorate on Interactive and Cognitive Environments (ICE), one of the 13 courses international PhD Courses selected in 2009 by European agency EACEA providing joint PhD titles. ICE is composed by a consortium of five top European Universities.

Since 1997 he has served as certified quality system reviewer of Higher Education Courses in the context of the Campus and Campusone projects organized by Italian Conference of Deans (CRUI). He has evaluated more than 40 courses at the Master and Bachelor level. He has also served as quality system reviewer under the European EURACE project.

Dr Regazzoni is the academic responsible of the security lab section of the Region Liguria industrial- academic consortium on Intelligent Integrated Systsems (SIIT) as well as member of the Scientifical Technical Committee of SIIT.

He was in the funding board of the International Conferences on Advanced Signal and Video Based Surveillance Systems, IEEE AVSS, now at its 6th edition. He serves as General Chair of Genova edition AVSS 2009. He served as Technical Program chairman for the IEEE SPS ICIP05 Conference on Image Processing. He is currently external reviewer of a EU-FP7 project (Sense) and of several research programmes in Italy. Dr. Regazzoni is Associate Editor of several international journals: IEEE Signal Processing Letters, IEEE Transactions on Mobile Computing, Eurasip Journal on Information Security, International Journal on Image and Graphics. He serves as associate editor for the IEEE Transactions on Mobile Computing, and the International Journal on Image and Graphics. He has been Guest editor of many special issues on Proceedings of the IEEE, IEEE Signal Processing Magazine, and other journals. He has been co-editor of 4 edited books (Kluwer) on intelligent video surveillance from 1999 to 2003. He has been awarded of best IEEE Vehicular Electronics VT paper award in 2002. Dr. Regazzoni is author or co-author of 70 papers on International Scientific Journals and of more than 250 papers presented at peer reviewed International Conferences. He is member of Image and Multidimensional Signal Processing (IMDSP) and Multimedia Signal Processing (MMSP) committees of the IEEE Signal Processing Society.

Lucio Marcenaro enjoys over 10 years of experience in image and video sequence analysis, and authored over 30 technical papers related to signal and video processing for computer vision. An Electronic Engineering graduate from Genova University in 1999, he received his PhD in Computer Science and Electronic Engineering from University of Genova in 2003. From 2003 to 2010 he was CEO and development manager at TechnoAware srl. From March 2011, he became Assistant Professor in Telecommunications for the Faculty of Engineering at the Department of Biophysical and Electronic Engineering at the University of Genova. He is the principal scientific and technical coordinator of the Ambient Awareness Lab (A2Lab), with TechnoAware srl.

His main current research interests are: video processing for event recognition, detection and localization of objects in complex scenes, distributed heterogeneous sensors ambient awareness systems, ambient intelligence and bio-inspired cognitive systems.

Lucio Marcenaro was involved in many video-surveillance projects including: "Sistemi intelligenti per l'elaborazione e la trasmissione di segnali multidimensionali per applicazioni di video-sorveglianza in tempo reale" (1999-2000) funded by the Italian Ministry of University and Scientific Research; UE-FP5 REOST (2002-2004): Railway Electro Optical System for Safe Transportation; UE-FP6 INMOVE (EU IST-2001-37422) (2002-2004): Intelligent Mobile Video Environments; VICOM-FIRB (2002-2006) VICom (Virtual Immersive Communications); Elsag PSA (2001-2004); Architetture distribuite ed eterogenee per sistemi di sorveglianza multi-sensoriali (Prot. N. 7280/297 MIUR) (2002-2005); Context Awarness and Autonomic Network (2007-2009), joint lab with Telecom Italia spa; Sintesis (Sistema integrato per la sicurezza ad intelligenza distribuita) (2008-2010).

Lucio Marcenaro was Industrial Chair of the 6th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS2009), Genoa, Italy September 2-4 2009. He is in the technical committee of many surveillance related international conferences (IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), IEEE International Conference on Imaging for Crime Detection and Prevention (ICDP), IEEE International Conference on Image Processing (ICIP), IEEE International Workshop on Visual Surveillance (VS) 2009) and international journals (IET Image Processing, IET Computer Vision, IET Communications, Machine Vision and Applications, Circuit and Systems for Video Technology, Pattern Recognition Letters, Transactions on Sensor Networks, Signal Image and Video Processing).