Tutorial 12: Bio-Inspired Cognition, Adaptation, and Learning over Networks

Presented by

Ali H. Sayed

Abstract

Self-organized and complex patterns of behavior are common in many biological networks, where no single agent is in command and yet forms of self-organization and decentralized intelligence are evident. Examples include fish joining together in schools, birds flying in formation, bees swarming towards a new hive, and bacteria diffusing towards a nutrient source. While each individual agent in these biological networks is not capable of complex behavior, it is the combined coordination among multiple agents that leads to the manifestation of sophisticated order at the network level. The study of these phenomena opens up opportunities for collaborative research across several domains including economics, life sciences, biology, and information processing, in order to address and clarify several relevant questions such as: (a) how and why organized behavior arises at the group level from interactions among agents without central control? (b) What communication topologies enable the emergence of order at the higher level from interactions at the lower level? (c) How is information processed during the diffusion of knowledge through the network? And (d) how does mobility influence the learning abilities of the agents and the network? Several disciplines have been concerned in elucidating different aspects of these questions including evolutionary biology, animal behavior studies, physical biology, and even computer graphics. In the realm of signal processing, these questions motivate the need to study and develop decentralized strategies for information processing that are able to endow networks with real-time adaptation and learning abilities. This tutorial examines several patterns of decentralized intelligence in biological networks, and describes powerful diffusion adaptation and learning strategies that are able to model and reproduce these kinds of behavior.

Although biological networks provide inspiration for the design of powerful engineered networks, the resulting theory and algorithms will be applicable to a broader context including machine learning applications, distributed optimization problems, and cooperative processing designs. The material presented in this tutorial can be used to design general cognitive networks that possess adaptation and learning abilities. Cognitive networks are defined as consisting of spatially distributed agents that are linked together through a connection topology. The topology may vary with time and the agents may also move. The agents cooperate with each other through local interactions and by means of in-network processing. Diffusion adaptation strategies are embedded into the nodes and allow them to perform various distributed tasks rather effectively. Such cognitive networks are well-suited to perform decentralized information processing and inference tasks. They are also well-suited to model complex behavior encountered in nature and in social and economic networks. While it is generally possible to find centralized or hierarchical processing mechanisms that can be more accurate in performing a given task, cognitive networks are generally more robust, scalable, adaptable, and resilient. Biological networks provide an excellent example of how resilient cognitive and adaptive networks can be.

Pre-requisites

The tutorial's objective is to introduce interested participants to this exciting field of exploration. The presentation of the material is designed to be accessible to a broad audience of students, researchers, and practitioners, including participants without prior background in adaptation and learning. Some basic understanding of matrix theory and probability theory can be helpful.

Outline

  1. Self-organized and complex behavior in biological networks.
  2. Local interactions that lead to global patterns.
  3. Strategies for information processing over networks.
  4. Strategies for distributed optimization over networks.
  5. Diffusion adaptation and learning over networks.
  6. Performance limits and learning behavior.
  7. Adaptive topologies; mobile networks.
  8. Fish schooling behavior.
  9. Bird flight formations.
  10. Bee swarming behavior.
  11. Bacteria motility.
  12. Stripe patterns in animals and visual cortex.
  13. Herding and crowd control.

Speaker Biography

Ali H. Sayed is Professor of Electrical Engineering at the University of California, Los Angeles (UCLA), and Principal Investigator of the UCLA Adaptive Systems Laboratory (http://www.ee.ucla.edu/asl). He is the author and coauthor of over 350 publications and 5 books. He is the author of the textbooks Adaptive Filters (New York: Wiley, 2008), Fundamentals of Adaptive Filtering (New York: Wiley, 2003), and co-author of Linear Estimation (Prentice-Hall, 2000). Dr. Sayed's research interests span several areas including adaptation and learning, adaptive and cognitive networks, bio-inspired networks, flocking and swarming behavior, cooperative behavior, distributed processing, self-healing circuitry, and statistical signal processing. His research has been awarded several recognitions, including the 1996 IEEE Donald G. Fink Prize, a 2002 Best Paper Award from the IEEE Signal Processing Society, the 2003 Kuwait Prize in Basic Science, the 2005 Frederick E. Terman Award, and a 2005 Young Author Best Paper Award from the IEEE Signal Processing Society. He served as Editor-in-Chief of the IEEE Transactions on Signal Processing (2003-2005) and the EURASIP Journal on Advances in Signal Processing (2006-2007). He also served as 2005 Distinguished Lecturer of the IEEE Signal Processing Society, and as General Chairman of the 2008 International Conference on Acoustics, Speech, and Signal Processing (ICASSP). He served as Vice-President-Publications of the IEEE Signal Processing Society (2009-2011), and as member of the Board of Governors (2007-2011) of the same society.