Science Series Part II: Synchronisation of Chaotic Networks

This is the second edition of my three-part series about the science behind my work at the University of Maryland TREND program (Training and Research Experience in Nonlinear Dynamics). I am currently working on the experimental side of the field of nonlinear dynamics and chaos. These are my posts:

Part 1: Nonlinear and chaotic systems

Part 2: Synchronisation of Chaotic Networks

Part 3: Nonlinear networks at my lab

In this post I would like to talk about the development of the field of research in synchronisation of chaotic systems. I would like to do this by talking about the work of Lou Pecora and highlighting some of his important papers.

Lou Pecora
Lou Pecora is a researcher at the NRL (Naval Research Laboratory) in Washington DC. He has done a lot of good work on synchronisation of chaotic systems and has worked a lot with Thomas Carroll (Also from NRL) and Francesco Sorrentino (University of New Mexico) on this subject. My Professors Thomas Murphy and Rajarshi Roy have also collaborated often with him and he even came to give a talk here during my program.

The nice thing about his work and the way that I will highlight it here, is the line followed of increasing generality and solving broader and broader problems. Of course these papers are not solely the result of his own work, but were in collaboration with others, but since his name is the only one appearing in all of these papers, it is reasonable to focus on him. Here is an overview of the papers I would like to expose:

• [1] "Synchronization in Chaotic Systems", Louis M. Pecora and Thomas L. Carroll (1990)
• [2] "Master Stability Functions for Synchronized Coupled Systems", Louis M. Pecora and Thomas L. Carroll (1997)
• [3] "Cluster synchronization and isolated desynchronization in complex networks with symmetries", Louis M. Pecora, Francesco Sorrentino, Aaron M. Hagerstrom, Thomas E. Murphy & Rajarshi Roy (2014)
• [4] "Complete characterization of the stability of cluster synchronization in complex dynamical networks", Francesco Sorrentino, Louis M. Pecora, Aaron M. Hagerstrom, Thomas E. Murphy, Rajarshi Roy (2016)
• [5] "Synchronization of chaotic systems", Louis M. Pecora, Thomas L. Carroll (2015) (Review Paper)

"Synchronization in Chaotic Systems" (1990)
This short three-page paper kicked off the field of study of synchronisation of chaotic systems. People (Russian and Japanese researchers) had actually looked at this before, as Pecora and Carroll admit in their 2015 review paper [5], but, as so often in science, the last person to discover something is usually the one who is remembered. What Pecora and Carroll did was investigate what happens if they were to take the well known chaotic Lorenz system, which has three x, y and z variables, and run one Lorenz system, then feed the x-variable of this system into the x variable of a second Lorenz system. It turned out that by driving the second system with only one variable of the three, the second system would synchronise!

Image I took from the 1990 paper [1]. It shows how a driven Lorenz system synchronises,
and approximately synchronises for small parameter mismatches.

Why is this result special? This is unexpected because chaotic systems are very sensitive to small differences, and so you would expect that small differences between the driven and driving system's conditions would always amplify and so prevent synchronisation. This synchronisation is like only telling someone your latitude during your trip, and them figuring out your longitude and exact route as well, and after that they are suddenly walking next to you, pushing you off the sidewalk! They did some mathematics on the equations underlying this, did a stability analysis, some simulations, and invented a thing called a sub-Lyapunov exponent. I mentioned Lyapunov exponents towards the end of my previous post. This sub-Lyapunov exponent quantifies the rate at which a state in the driven system will approach or go away from the nearby state of the driving system. If all sub-Lyapunov exponents are negative, the system will synchronise, if one is positive, then the systems will never synchronise since whenever their states get close, they will diverge again. Note that the maximal sub-Lyapunov exponent is approximately the steepness of the line in the figure I enclosed above. If it is positive, differences will increase, if it is negative, solutions converge.

I don't have many good graphs or pictures at hand of examples
of synchronising chaotic systems, but here is the best I have:
two oscilloscopes from my experiment displaying the outputs of two nodes.
As you can see, their signals have synchronised, although if you look
closely, you will notice that they are in antiphase on the lower oscilloscope.

Lou Pecora and Thomas Carroll were initially not just interested in this for purely scientific purposes, but they also justified their work with the idea of finding a new method of encoding messages in these chaotic signals for Military or Naval communication purposes. See a demonstration here (at approx 1:50). This turned out to be an over-complicated way of achieving this and there are now better ways to do this. An interesting thing about their encryption though is that it could be achieved with analogue circuits and since digital communications were still not common place at the time, this was a useful thing. In their paper they continued to demonstrate this synchronisation in a real experiment with two electronic circuits. This is very common in literature on synchronising chaotic systems: showing something works on paper and simulations, and then proving that it is a real-world effect by using an experiment, which is always has imperfections compared to theory.

"Master Stability Functions for Synchronized Coupled Systems" (1998)
Soon people began to consider more general problems. Where Pecora's 1990 paper initially considered one system synchronising to another, questions were asked such as "What happens if we couple a system both ways?", or "What if we couple multiple systems?", or "Can we build networks of systems?". From this all stemmed the question: how can we generally predict global synchronisation for a coupled system? Global synchronisation means that we consider a network of multiple chaotic systems, and seeing if by coupling them, we can get all systems to synchronise together, i.e. global synchrony. To get an idea of global synchrony, watch this video of metronomes synchronising. Pecora and Carroll's 1998 paper gives a definitive answer to this question. They considered the following general system:

Here x represents the ith node, F is a function indicating each node has the same internal dynamics, G indicates that the network is connected by a matrix, and H is the function that determines how each node influences the rest.

These equations describe how a system consisting of identical nodes, coupled through connections defined in the matrix G, behaves. The question is whether this can synchronise. It turns out you can predict this with theory. What Pecora and Carroll did is invent a thing called the Master Stability Function (MSF). This allows us to separately analyse the shape of the network and the chaotic system which the network nodes represent. By analysing the network we can calculate so-called eigenvalues of the network. By analysing the system which makes up each node, we can get the master stability function. If all the eigenvalues of our network are to be found in the area where the master stability function is negative, then all sub-Lyapunov exponents will be negative, and our system can synchronise.

Example of a star network, which may, or may not synchronise

Below you can see that all the round dots, which represent the eigenvalues of a circular network, lie in the area that the MSF is negative, so that system will synchronise. One of the stars which represent the eigenvalues of a star-shaped network lies in the positive part, and so that system will not synchronise. By changing the coupling strength we can get these values to lie in or outside these negative areas, and so by making a system couple too strongly, it will not synchronise, and by coupling it too weakly, it will not synchronise either. The coupling has to be just right.

Picture from 1998 paper [2]. It shows the graph of the Master Stability Function

Cluster synchronisation (2014 and 2016)
Lou Pecora went further, and wrote a series of papers with Francesco Sorrentino, going beyond general synchronisation. He considered: What if clusters within our network synchronise? When does that happen? This is detailed in two papers from 2014 and 2016, which I will admit, I did not fully understand, contrary to the previous papers, but I got the general point...!

It turns out that networks can support clusters of synchrony, or even clusters of synchrony with at the same time other sections being desynchronised! The conditions for this are determined by the symmetries of a network. To understand this better, you need to know some group theory. To put it simply, symmetries can induce so-called orbits in networks.

Orbits are collections of nodes that can be permuted to each other through a symmetry. These orbits can form groups that synchronise, because these nodes are essentially 'equivalent', since they are symmetric to each other. Although these networks in theory support synchrony, these synchronised solutions may be unstable. That is where the stability analysis from Lou's work comes in again, to show that these clusters can be stable. Here is a nice paper from Joe Hart, the graduate student at our lab who helped me so much, where he demonstrates all possible clusters in a four-node network!

Example of a network, where I gave the nodes in the same orbits, the same colours.
There are therefore 4 orbits. Can you find all 8 symmetries?