Science Series Part II: Synchronisation of Chaotic Networks

This is the second edition of my three-part series about the science behind my work at the University of Maryland TREND program (Training and Research Experience in Nonlinear Dynamics). I am currently working on the experimental side of the field of nonlinear dynamics and chaos. These are my posts:

Part 1: Nonlinear and chaotic systems

Part 2: Synchronisation of Chaotic Networks

Part 3: Nonlinear networks at my lab

In this post I would like to talk about the development of the field of research in synchronisation of chaotic systems. I would like to do this by talking about the work of Lou Pecora and highlighting some of his important papers.

Lou Pecora
Lou Pecora is a researcher at the NRL (Naval Research Laboratory) in Washington DC. He has done a lot of good work on synchronisation of chaotic systems and has worked a lot with Thomas Carroll (Also from NRL) and Francesco Sorrentino (University of New Mexico) on this subject. My Professors Thomas Murphy and Rajarshi Roy have also collaborated often with him and he even came to give a talk here during my program.

The nice thing about his work and the way that I will highlight it here, is the line followed of increasing generality and solving broader and broader problems. Of course these papers are not solely the result of his own work, but were in collaboration with others, but since his name is the only one appearing in all of these papers, it is reasonable to focus on him. Here is an overview of the papers I would like to expose:

• [1] "Synchronization in Chaotic Systems", Louis M. Pecora and Thomas L. Carroll (1990)
• [2] "Master Stability Functions for Synchronized Coupled Systems", Louis M. Pecora and Thomas L. Carroll (1997)
• [3] "Cluster synchronization and isolated desynchronization in complex networks with symmetries", Louis M. Pecora, Francesco Sorrentino, Aaron M. Hagerstrom, Thomas E. Murphy & Rajarshi Roy (2014)
• [4] "Complete characterization of the stability of cluster synchronization in complex dynamical networks", Francesco Sorrentino, Louis M. Pecora, Aaron M. Hagerstrom, Thomas E. Murphy, Rajarshi Roy (2016)
• [5] "Synchronization of chaotic systems", Louis M. Pecora, Thomas L. Carroll (2015) (Review Paper)

"Synchronization in Chaotic Systems" (1990)
This short three-page paper kicked off the field of study of synchronisation of chaotic systems. People (Russian and Japanese researchers) had actually looked at this before, as Pecora and Carroll admit in their 2015 review paper [5], but, as so often in science, the last person to discover something is usually the one who is remembered. What Pecora and Carroll did was investigate what happens if they were to take the well known chaotic Lorenz system, which has three x, y and z variables, and run one Lorenz system, then feed the x-variable of this system into the x variable of a second Lorenz system. It turned out that by driving the second system with only one variable of the three, the second system would synchronise!

Image I took from the 1990 paper [1]. It shows how a driven Lorenz system synchronises,
and approximately synchronises for small parameter mismatches.

Why is this result special? This is unexpected because chaotic systems are very sensitive to small differences, and so you would expect that small differences between the driven and driving system's conditions would always amplify and so prevent synchronisation. This synchronisation is like only telling someone your latitude during your trip, and them figuring out your longitude and exact route as well, and after that they are suddenly walking next to you, pushing you off the sidewalk! They did some mathematics on the equations underlying this, did a stability analysis, some simulations, and invented a thing called a sub-Lyapunov exponent. I mentioned Lyapunov exponents towards the end of my previous post. This sub-Lyapunov exponent quantifies the rate at which a state in the driven system will approach or go away from the nearby state of the driving system. If all sub-Lyapunov exponents are negative, the system will synchronise, if one is positive, then the systems will never synchronise since whenever their states get close, they will diverge again. Note that the maximal sub-Lyapunov exponent is approximately the steepness of the line in the figure I enclosed above. If it is positive, differences will increase, if it is negative, solutions converge.

I don't have many good graphs or pictures at hand of examples
of synchronising chaotic systems, but here is the best I have:
two oscilloscopes from my experiment displaying the outputs of two nodes.
As you can see, their signals have synchronised, although if you look
closely, you will notice that they are in antiphase on the lower oscilloscope.

Lou Pecora and Thomas Carroll were initially not just interested in this for purely scientific purposes, but they also justified their work with the idea of finding a new method of encoding messages in these chaotic signals for Military or Naval communication purposes. See a demonstration here (at approx 1:50). This turned out to be an over-complicated way of achieving this and there are now better ways to do this. An interesting thing about their encryption though is that it could be achieved with analogue circuits and since digital communications were still not common place at the time, this was a useful thing. In their paper they continued to demonstrate this synchronisation in a real experiment with two electronic circuits. This is very common in literature on synchronising chaotic systems: showing something works on paper and simulations, and then proving that it is a real-world effect by using an experiment, which is always has imperfections compared to theory.

"Master Stability Functions for Synchronized Coupled Systems" (1998)
Soon people began to consider more general problems. Where Pecora's 1990 paper initially considered one system synchronising to another, questions were asked such as "What happens if we couple a system both ways?", or "What if we couple multiple systems?", or "Can we build networks of systems?". From this all stemmed the question: how can we generally predict global synchronisation for a coupled system? Global synchronisation means that we consider a network of multiple chaotic systems, and seeing if by coupling them, we can get all systems to synchronise together, i.e. global synchrony. To get an idea of global synchrony, watch this video of metronomes synchronising. Pecora and Carroll's 1998 paper gives a definitive answer to this question. They considered the following general system:

Here x represents the ith node, F is a function indicating each node has the same internal dynamics, G indicates that the network is connected by a matrix, and H is the function that determines how each node influences the rest.

These equations describe how a system consisting of identical nodes, coupled through connections defined in the matrix G, behaves. The question is whether this can synchronise. It turns out you can predict this with theory. What Pecora and Carroll did is invent a thing called the Master Stability Function (MSF). This allows us to separately analyse the shape of the network and the chaotic system which the network nodes represent. By analysing the network we can calculate so-called eigenvalues of the network. By analysing the system which makes up each node, we can get the master stability function. If all the eigenvalues of our network are to be found in the area where the master stability function is negative, then all sub-Lyapunov exponents will be negative, and our system can synchronise.

Example of a star network, which may, or may not synchronise

Below you can see that all the round dots, which represent the eigenvalues of a circular network, lie in the area that the MSF is negative, so that system will synchronise. One of the stars which represent the eigenvalues of a star-shaped network lies in the positive part, and so that system will not synchronise. By changing the coupling strength we can get these values to lie in or outside these negative areas, and so by making a system couple too strongly, it will not synchronise, and by coupling it too weakly, it will not synchronise either. The coupling has to be just right.

Picture from 1998 paper [2]. It shows the graph of the Master Stability Function

Cluster synchronisation (2014 and 2016)
Lou Pecora went further, and wrote a series of papers with Francesco Sorrentino, going beyond general synchronisation. He considered: What if clusters within our network synchronise? When does that happen? This is detailed in two papers from 2014 and 2016, which I will admit, I did not fully understand, contrary to the previous papers, but I got the general point...!

It turns out that networks can support clusters of synchrony, or even clusters of synchrony with at the same time other sections being desynchronised! The conditions for this are determined by the symmetries of a network. To understand this better, you need to know some group theory. To put it simply, symmetries can induce so-called orbits in networks.

Orbits are collections of nodes that can be permuted to each other through a symmetry. These orbits can form groups that synchronise, because these nodes are essentially 'equivalent', since they are symmetric to each other. Although these networks in theory support synchrony, these synchronised solutions may be unstable. That is where the stability analysis from Lou's work comes in again, to show that these clusters can be stable. Here is a nice paper from Joe Hart, the graduate student at our lab who helped me so much, where he demonstrates all possible clusters in a four-node network!

Example of a network, where I gave the nodes in the same orbits, the same colours.
There are therefore 4 orbits. Can you find all 8 symmetries?

Weeks 6,7,8,9,9.5: Travelling around, but no wheels, Duncan comes

I am writing this post by my bedroom window which, despite the airconditioning in my flat, is wide open, so that I can hear and enjoy the outside summer noises. I have now been here for almost eight weeks and I am settling in here well, getting used to living in the USA. My research on the other hand, is going a bit pear-shaped. Duncan is coming on Tuesday and I will be heading off in two weeks time!
...time passes...
I am now writing this post on Wednesday evening, and Duncan has come. Things are speeding along and here is an overview of what I have been up to!

New York + Visiting Jutta
The week before last I headed off from work early on Friday, to Union Station in DC, took the "Washington Deluxe", a pretty standard but otherwise good bus, straight to New York. In New Jersey, as the sun set, the Big Apple started to loom in the distance, with my bus gliding towards the city, marked by the towers which, planted in the city, extended high into the night sky. After going through the tunnel to Manhattan, stepping out of the bus, I found myself on the corner of 36th St and 7th Ave, and made a dash to the Port Authority bus station. On the way I was impressed by the atmosphere of the old, brick Manhattan buildings, with modern cafés in between, and the eclectic business of 8th Avenue. I jumped into my commuter bus, swooping me out of the city, to Demarest, New Jersey, on the other side of the Hudson River, to see Jutta, my lovely host. As soon as I saw the sign "Demarest", I pressed the stop button, confirmed with the bus driver this was Demarest, and jumped out.

I discovered that although this was definitely Demarest, I was nowhere near the Duck Pond bus stop, where I was meeting Jutta-I had jumped out way too early. Some walking and guessing remedied this and soon I found her and we headed off to her place. Jutta is an old family friend. I last met her in 2010, when I travelled to New York for the first and last time since I was born there! She is from Westfalen, in Germany and emigrated to the USA in 1960, and has known my grandparents for a long, long time. My father was even sent to stay with her in 1978 as a fifteen-year old! She made some lovely dinner-sandwiches for me and supplied me with ample dessert. After having a lovely post-dinner chat with her, I headed to bed, and slept like a log.

This bowl I came across just comes to demonstrate that drawing technique has not changed
much over the course of 3000 years.

On Saturday I headed into town, sauntered around on the busy 7th avenue, took a subway to get to the Metropolitan Museum of Art, accidentally went 50 streets too far, made my way back, traversed Central Park, and entered this museum with a member's ticket under the guise of Jutta Greweldinger, a patroness of the Met since 1978. Apparently very convincing, because I got in! They had many interesting things to mention, a vast collection, but two things I remember well was their section with Cypriot artifacts from pre-hellenic times, which I was very fond of, and their portraits and landscapes from Dutch artists in the 14th, 15th and 16th centuries. Even though the Dutch golden age was in the 17th century, these paintings were already very refined and impressive.

View from Brooklyn Bridge

After this, I headed down to southern Manhattan, to where the Staten Island Ferry leaves, and had a good look across the sound bay, then went on, marched across the Brooklyn bridge, and went to a small little barge by the water, which was currently being shouted over by some extremely loud concert nearby. There I attended the a chamber music performance at Bargemusic, with the performers being the director of the place, Peskov himself, a Ukrainian violinst, and a piano player. Together the played various duets. Peskov had quite an informal, jokey manner, and even needed to check the program to see what he was going to play 'Let's see what I will play today...'. I then headed off to Port Authority, got very lost in the metro along the way, and came back to Jutta.

Sunday
That morning I got up and watched the football world cup final with Jutta. Good game, pity France won, we were both routing for Croatia! Afterwards I said goodbye, went back to NY, and thoroughly investigated Times Square, which turned out to be quite a small square, with way too many tourists. Not much there, definitely not the center of the world. I then visited St. Patrick's Cathedral, which was quite stunning. St. Patrick's Cathedral is where I was baptised in 1998! It was really cool being there. They happened to be playing an organ concert at that moment.

St. Patrick's Cathedral from the inside

After this I passed the Empire State Building, got some reeeally cheap pizza slices, scoffed it down, and bussed back down to DC. 't Was a good weekend. I much prefer NY to DC. The people are more warm there, although they can also be outright. Better to show what you feel, than be cold I think! I could imagine returning to work in the future! On top this, I also think that the New Yorker accent is adorable. Once back at home I said farewell to John, my good flatmate, who was off to Alaska and finishing up in DC. He was thrilled to hear about the huge buildings in NY and even more thrilled to go shoot guns and see his girlfriend again!

Farewell to the Empire State of Mind

Disaster strikes: Bicycle Dismembered

Things were going all well the following week...until Tuesday morning I discovered someone had taken my bleeding bike wheels!! Both bloody wheels!! Parked right outside my bleeding bloody door! I took it as it came and the next day, I found myself doing the walk of shame with my considerably lighter bike across campus to the bike shop to explain what happened. They replaced my bike and only charged 80$ so good on them, thanks for the kindness!

The last thing you want to see on your Tuesday morning

Weekend with Verwiel Family
The weekend after this, I headed off to Baltimore Airport, where I met Frank Verwiel, an old friend of my father's, who was picking up his daughter Laurine. Together we headed off to their stunning weekend house at the east shore of Chesapeake Bay in Royal Oak, MD. Over there I met his wife Francine. Unfortunately, it was rainy the whole weekend and so we weren't able to enjoy the outdoors too much. I had a pleasant time talking to them. Even though Frank and Francine emigrated to the USA a long time ago, they have managed to completely raise their children in Dutch and so I actually only spoke Dutch the whole weekend!

Seeing Cecil
I have an old friend. Quite an old friend. To be exactly, I met Cecil in Brazil when I was 7 :-p. Since then we have seen each other a few times. He happens to study at Georgetown University in DC and was in the area, and so we met up, we met up twice! We had a good conversation, caught up, reminded each other of our parents' names etc. and generally had a good time inspecting Georgetown during my first visit, and DC monuments the second time.

Some people stand on shoulders of giants.......I sit on their lap [Einstein Monument]

Cecil-Francis and I at the WWII monument

Cycle to DC and Greek cooking
Next weekend I cycled to Washington DC on my proud refitted bike which had wheels again! I took a long bicycle trail which leads all the way from where I live to the East of DC and from there, headed on to the capitol. There I hung around, had a pleasant time, got a milk shake, ascertained that I was satisfied, and headed back. It's an 11.8 mile cycle there. At home, I baked Cypriot Halloumi bread according to the recipe that my friend Christos Kourris from Edinburgh gave me: mix halloumi cheese with dough and onions, and bake it. It was pretty good, although there is space for improvement, which is a good thing!

Ma' dow

Ma' loaf

On Sunday I also went to the National Shrine of the Immaculate Conception in Washington DC, which is a long name for the biggest Cathedral in North America! It was a very impressive church and the organ so loud, and the setup so overwhelming, that mass almost felt like a movie, or a show!

Outside

Inside

Chapel dedicated by Irish to St. Patrick

Duncan visits
Last week Tuesday, Duncan came to visit me! My phone had just broken down, and so all the coordination from then on had to be done with old fashioned e-mail and co-ordination. We looked at monuments in DC, went swimming at the pool, enjoyed diving, in the weekend looked at DC again, swam and dove more, and had a good time in each other's company. I even took him to an astronomy talk and he saw my lab and met some of my research mates! Monday morning he left off and I will be seeing him Sunday, when I return to Ireland, which I am really looking forward to. Since my phone broke down, I have no pictures as evidence. Duncan came just at the right moment because that weekend, my good flatmates Nishad (from Caltech) and Josh (Alaska/Arizona) left off, as well as Jessie (Alaska/Washington). Now it's just Kwazi and me camping out in our flat.

How have I been lately
I have been pretty good. On the one hand I am settling in here and finding my rhythm even better, on the other hand I am really looking forward to having some holiday and seeing family & friends again. I suppose that is double good! My work has been progressing steadily and I have pretty much rounded it up by now. I will be presenting on Friday at a Symposium with the other REU's.

Last Friday I started thinking about how symmetries in dynamical networks induce symmetries in their statistics, and how looking at the statistics allows us to infer symmetries! I have been thinking about it and I think it is pretty exciting. It needs a lot more work but it would be a new way of finding something out about the shapes of a network by looking at the nodes. I am not sure if other people have done this before though, I need to verify that. My professor seemed to be pretty interested. I have also made a research poster about my work, find it below. I posted it on a separate website so you can zoom in.

Also I just found a gem of a music video. It is the song "Freight Train", written and played by Elizabeth Cotton. This song has been covered many times by country music players. She wrote it around 1906 and in this video she is actually 90+ and performing in her own home! Take note of the fact that she is left-handed and so is playing this right-handed guitar upside down, plucking the basses with her fingers!