Intelligent Systems
Complex problem settings in widely differing technical, commercial, and financial fields evoke an increasing need for computer applications that must show “intelligent behavior.” These applications are desired to support decision-making, to control processes, to recognize and interpret patterns, or to maneuver vehicles or robots autonomously in unknown environments. Novel approaches, methods, tools and programming environments have been developed to accomplish such tasks.
Seen from a higher level of abstraction, the general requirements for developing such an “intelligent system” are ultimately always the same, namely simulating intelligent thinking and actions in a certain field of application (Russel and Norvig 2009). For this purpose, the knowledge about this field must be represented and processed. The quality of the resulting system mainly depends on how well the knowledge representation problem is solved in the development process. There is no such thing as the “best” method. Rather one has to sift through many available approaches to find those that fit the application area of the intelligent system best.
The mechanisms and processes that underlie intelligent behavior are examined in the research area of artificial intelligence. Similar to most other areas of computer science (or science in general), computational intelligence (CI) comprises both theoretical aspects (how and why do these system work?) and application-oriented aspects (where and when can these systems be used?).
At the beginning of the development of intelligent systems, researchers often focused the idea of seeing “a human being as a machine”—an idea that stems from the age of enlightenment. They ventured to create an (artificial) intelligence that can both think creatively and solve problems in the way a human can. This intelligence was also meant to exhibit forms of both consciousness and emotions. In the infancy of the field, the typical way to design an artificial intelligence was to describe a symbolic basis of the relevant mechanisms. This includes the top-down perspective of problem solving, which mainly addresses the question why these systems work. The answer to this question is usually given with the help of a symbolic representation and a logic-based inference mechanism. Techniques that are examples of these approaches include rule-based expert systems, automatic theorem provers, and many operations research techniques which underlie modern planning and scheduling software. Although these traditional approaches have been very successful in some cases, they do have limitations, especially when it comes to scalability. A tiny complication of the problem to solve often causes an increase of complexity that cannot be handled feasibly. As a consequence, although these approaches usually guarantee an optimal, precise, or correct solution, they are rarely applicable to practical problems.
Consequently, efficient method to represent and process knowledge are still a research topic. For certain types of problems, techniques that are inspired by natural or biological processes proved successful. These approaches signify a paradigm change away from symbolic representations and towards inference strategies for adaptation and learning. Among such methods we find artificial neural networks, evolutionary algorithms, bayes networks, and fuzzy systems. These novel methods have demonstrated their usefulness in many application areas, often in combination with traditional problem-solving techniques.
Computational Intelligence
The research area of computational intelligence (CI) comprises concepts, paradigms, algorithms, and implementations to develop systems that exhibit intelligent behavior in complex environments. Typically, sub-symbolic and nature-analogous methods are adopted that tolerate incomplete, imprecise, and uncertain knowledge. As a consequence, the resulting approaches allow for approximate, manageable, robust, and resource-efficient solutions (Kacprzyk and Pedrycz 2015).
The general strategy that is adopted in the area of computational intelligence is to apply approximation techniques and methods that can find coarse, incomplete, or only partially valid solutions to given problems. As a reward for dispensing with guaranteed correctness and completeness, solutions are found in a tolerable time frame and within a bearable budget. Such solutions often consist of relatively simple sub-functions, which, through interaction, lead to complex and self-organized behavior. As a consequence, these heuristic approaches can usually not be analyzed in a classical fashion, but, in exchange, they offer the possibility to quickly find approximate solutions to problems that are difficult to solve in other ways.
It is obvious that an area as diverse as computational intelligence cannot be covered exhaustively in about 570 pages. Therefore we confine ourselves to four core techniques that are frequently used in practice. In the first two parts of this book, so-called nature-analogous or natured-inspired methods are discussed. Here the governing idea is to analyze problem-solving strategies as they occur in nature. Certain aspects of such solution strategies are then mimicked or simulated in a computer, usually without striving to model the original systems correctly and exhaustively, nor even ensuring the biological plausibility of the simulation. Particularly successful and thus practically relevant representatives of this kind are artificial neural networks (Haykin 2008), evolutionary algorithms (Rozenberg et al. 2012) ant colony optimization (Dorigo and Stützle 2004).
Many ideas and principles in the area of artificial neural networks are inspired by neuroscience. Artificial neural networks are information processing systems whose structure and functionality simulates the nervous systems and particularly the brain of animals and human beings. They consist of a large number of fairly simple processing units, the so-called neurons , which work in parallel. These neurons send information in the form of action potentials via directed links to other neurons. Based on knowledge about the functionality of biological neural networks, one tries to model and mimic them, especially to achieve learning capability.
Evolutionary algorithms draw on ideas from biological evolution, in which organisms, over many generations, get adapted to environmental conditions. They address certain classes of optimization problems and belong to the family of metaheuristics, which offers algorithms to approximately solve may types of optimization problems. Metaheuristics are defined by an abstract sequence of steps that are applicable to more or less arbitrary problem descriptions. However, every single step must be implemented in a problem-specific fashion. Metaheuristics are often applied to problems for which no efficient solution algorithm is known. Although finding an optimal solution is usually not guaranteed and thus a found solution can, in principle, be arbitrarily bad compared to the optimal solution, metaheuristics offer the possibility to obtain (sufficiently) good solutions in a reasonable time frame. Here biologically inspired methods such as particle swarm optimization (or computational swarm intelligence) have proven themselves to be noteworthy candidates to employ in theory and practice.
The latter two parts of the book focus on integrating uncertain, vague and incomplete knowledge into the problem-solving strategy. The governing idea in these parts is that for human beings even imperfect knowledge can be very valuable. Therefore it is desirable to enable computers to work with such knowledge as well and not just precise and certain knowledge. Particularly successful approaches that can handle such knowledge are fuzzy systems (Kruse et al. 1994; Michels et al. 2006) and Bayesian and Markov networks (Borgelt et al. 2009; Kjaerulff and Madsen 2008).
In fuzzy systems, vague knowledge, which may be provided by a human expert or formulated intuitively by a system developer, is formalized with the help of fuzzy logic and fuzzy set theory. Fuzzy approaches can also be used to derive inference mechanisms, thus giving rise to approximate reasoning methods. Fuzzy systems are routinely applied in control engineering because in many application scenarios a precise and complete modeling of the system is impractical or even impossible.
Bayesian networks are means to efficiently store and reason with uncertain knowledge in complex application areas. Formally, a Bayesian network is a probabilistic graphical model that represents a set of random variables and their conditional dependences by a directed acyclic graph. Due to the probabilistic representation, one can easily draw inferences based on new information. In addition, Bayesian networks are well suited for dependence analysis and learning from data.
In many applications, hybrid computational intelligence systems, for instance, neuro-fuzzy systems, have proven to be highly useful. Sometimes these techniques are also combined with related methods, e.g., from the area of machine learning or case-based reasoning (Hüllermeier et al. 2010).
About the Second Edition of This Book
Our main objective with this textbook is to give a methodical introduction to the field of computational intelligence. Therefore we focus on fundamental concepts and their implementation and strive to explain the theoretical background of proposed solutions to certain problems. We hope to convey to a reader all that is necessary for a profound application of the discussed methods. This book requires only fairly basic knowledge of mathematics, as we tried to introduce all necessary concepts and tools in this book in order to make it as self-contained as possible. Furthermore, the four parts about artificial neural networks, evolutionary algorithms, fuzzy systems, and Bayesian networks can be studied independently of each other as we tried to avoid dependences between the parts or requiring prerequisites from earlier parts.
Chapters on deep learning, computational swarm intelligence, and fuzzy data analysis complement the second version of this book and accomodate for recent developments of different aspects of computational intelligence. This book would not be the same without the many contributions by Frank Klawonn and Christian Moewes for the parts on fuzzy systems and evolutionary algorithms from the first edition.
This book is intended as an accompanying book for lectures about the field of computational intelligence and is fundamentally based on written notes about lectures that the first author has given periodically for students of different areas for almost 20 years.
On the website module descriptions, lecture slides matching the book, exercise sheets with solutions, sample exams, software demos, literature references, references to organizations, journals, software tools, and additional material can be found for all four parts, i.e., artificial neural networks, evolutionary algorithms, fuzzy systems, and Bayesian networks. This additional material covers a total of four modules, i.e., lectures with corresponding exercise lessons.