Hydrogel Learns to Play Pong: A Breakthrough in Soft Robotics and AI

A lifeless hydrogel blob can play Pong

Inspired by recent advancements in brain organoid systems, researchers have designed a simple hydrogel-electrode array that not only can "play" Pong, but improve its gameplay over time. Debuted by Atari in 1972, Pong is one of the most rudimentary but influential video games of all time. Although it just features two player paddles and a pixelated "ball" ricocheting between them, it still serves as a helpful benchmark for training not just artificial intelligence and neural networks, but also organoid intelligence, or OI. Grown from stem cells into rudimentary "brains," these OI systems may one day provide promising alternatives to more traditional hardware. But both AI and OI are extremely complex, costly industries -- what if much simpler arrays could achieve similar results? A team at the UK's University of Reading set out to do just that, and published their surprising findings on August 22 in the journal, Cell Reports Physical Science. As New Scientist highlighted in a video on Thursday, the results may one day influence new, simpler forms of algorithmic design. Led by biomedical engineering professor Yoshikatsu Hayashi, researchers hooked up a blob of nonliving, electroactive materials known as Belousov-Zhabotinsky (BZ) hydrogels to a computer simulation of Pong using a custom multi-electrode array. When subjected to small bursts of electricity, the BZ hydrogels ionize. This, in turn, causes their water molecules to oscillate and swell, changing the sample's overall shape in the process. But the hydrogels reduce swelling at a much slower rate than they do to initially enlarge, which means each movement influences the next. "[It] is sort of like memory occurring," robotics engineer and study first author Vincent Strong, explained in an accompanying statement. "The continued rearrangement of ions within the hydrogel is based off of previous rearrangements within the hydrogel, continuing back to when it was first made and had a homogeneous distribution of ions." "The basic principle in both neurons and hydrogels is that ion migration and distributions can work as a memory function that can correlate with sensory-motor loops in the Pong world," Hayashi added. "In neurons, ions run within the cells; in the gel, they run outside." [Related: Lab-grown human brain tissue used to control robot.] Once their system was in place, Hayashi and collaborators started a round of Pong by sending a ball in a random direction. Electrical stimulation then "informed" the hydrogel of the ball's location while measuring ionic movement in the smart polymer to "determine" its virtual paddle position. As each round progressed, the team recorded the gel's accuracy rate to see if it managed to improve over time -- which, in fact, it did. "Ionic hydrogels can achieve the same kind of memory mechanics as more complex neural networks," said Strong. "We showed that hydrogels are not only able to play Pong, they can actually get better at it over time." The hydrogel, however, never became a Pong pro. According to the team, their ionized blob never truly beat its computer opponent and improved its accuracy by a maximum of around 10 percent. The polymer also reached its peak performance after 20 minutes of play, versus previous brain organoid tests doing the same in just 10 minutes. That said, such advances allowed the hydrogel to compete over longer rallies against the game. Given most AI algorithms currently rely on neural networks to function, Hayashi's team now theorizes that hydrogels may offer a different form of "intelligence" that can be harnessed to create simpler algorithmic systems. Going forward, researchers hope to further investigate the physical properties behind the rudimentary memory formations, as well as test it in other applications.

Scientists enable hydrogel to play and improve at Pong video game

Researchers say their creation has memory, which it can use to perform better by gaining experience Researchers have found a soft and squidgy water-rich gel is not only able to play the video game Pong, but gets better at it over time. The findings come almost two years after brain cells in a dish were taught how to play the 1970s classic, a result the researchers involved said showed "something that resembles intelligence". The team behind the latest study said that while they were inspired by that work, they were not claiming their hydrogel was sentient. "We are claiming that it has memory, and through that memory it can improve in performance by gaining experience," said Dr Vincent Strong, the first author of the research, from the University of Reading. Strong said the work could offer a simpler way to develop algorithms for neural networks - models that underpin AI systems including Chat GPT - noting that at present they are based on how biological structures work. Released in 1972, Pong was one of the first video games and has a simple premise: two paddles on a court can be moved up and down to hit a ball back and forth between them. The longer the rally, the higher the score. Strong's study focused on a single-player version in which a paddle is moved along one wall of a court to keep a ball bouncing around. Writing in the journal Cell Reports Physical Science, he and his colleagues describe how they sandwiched an electroactive polymer hydrogel between two plates, each bearing a 3x3 array of electrodes hooked up to a computer system that simulated Pong. Six of the electrode pairs, in a 3x2 arrangement, were then stimulated to represent the movement of the ball within the game's court. Across the other three electrode pairs - representing the wall along which the paddle is located - the team applied a small voltage and the current was measured with sensors. The position of the paddle was defined as the point where the current was highest. Crucially, the type of hydrogel used in the experiment contains charged ions. These move in response to electrical stimulation and linger where they end up. As a result, the point along the "wall" with the highest current could shift as the ball moved, meaning the paddle could change position. "At the beginning the ions are equally and randomly distributed so the paddle hits and misses the ball," said Strong. But as the ball bops around the court, the gel receives more and more electrical stimulation. "Over time the ion concentrations increase where the ball is most, acting as a kind of muscle memory, as with the higher concentrations there are higher electric current readings and the paddle is able to act more accurately," said Strong. In other words, the paddle is able to hit the ball more often, resulting in longer rallies. "Our research shows that even very simple materials can exhibit complex, adaptive behaviours typically associated with living systems or sophisticated AI," said Dr Yoshikatsu Hayashi, another author of the research at the University of Reading. Dr Brett Kagan, the chief scientific officer at Cortical Labs, who worked on the Pong-playing brain cells but was not involved in the latest study, said the hydrogel system demonstrates a basic form of memory similar to the way a riverbed records a memory of the river. That, he said, can be useful to understand how changes within a medium may help electrical signals travel through it better. But he said significantly more work would be needed to show hydrogels can "learn". "The performance and the improvement was tied to a specific location of stimulation. When this was changed in any way the system was not able to reorganise to still show performance," Kagan said. "This is different to our tests in neural systems where we showed that regardless of how you presented the information, learning still occurred."

Gooey gel can play video game Pong and learns how to improve over time, scientists find

A gooey gel made by scientists can play the video game Pong and gets better over time as it learns, new research has shown. The experts claim the "muscle memory" on display might be useful for people developing artificial intelligence. Inspired by a study that used brain cells in a dish to play Pong, the team from Reading University decided to try playing the "tennis like" game with a simple water-based gel. The soft, flexible material contained charged ions that respond to electricity. When electricity was passed through the gel, the ions moved to the source of the current, dragging water with them and causing the gel to swell. It was hooked up to electrodes that mimicked the classic Pong board, dividing it into six rectangles with walls around the edges. The aim of Pong, which was one of the first video games ever released, is to pass a ball between two paddles and keep a rally going. The scientists passed a small voltage into the hydrogel where the ball hit the wall and measured where the ions gathered. The computer interpreted this as where to move the paddle. Over time, the charged ions gathered more effectively where the ball hit. Its accuracy improved by up to 10% and the length of rallies increased. Muscle memory Signs of swelling remained in the gel, acting as a sort of memory, say the team. The scientists are not claiming the gel is sentient, however, the "muscle memory" on display might be useful for those developing AI. Read more from Sky News: Lung cancer patient is first in UK to receive experimental vaccine 'Don't sit' on mpox vaccines, WHO's Europe chief says Alzheimer's drug given green light in Britain - but won't be available on NHS "Instead of it just knowing what's immediately happened, it has a memory of the ball's motion over the entirety of the game," robotics engineer Vincent Strong, of the University of Reading, told the New Scientist magazine. "It sort of gains an experience of the ball's general motion, not just its current position. It sort of becomes a black-box neural network that has a memory of the ball's behaviour, how it behaves and how it moves." Dr Yoshikatsu Hayashi, a biomedical engineer at the University of Reading who led the research, said: "Our research shows that even very simple materials can exhibit complex, adaptive behaviours typically associated with living systems or sophisticated artificial intelligence." Most AI algorithms are inspired by the complex way the human brain works but the researchers say that hydrogels represent a different kind of "intelligence" that could be used to develop new, simpler algorithms. "Ionic hydrogels can achieve the same kind of memory mechanics as more complex neural networks," said Mr Strong. In future, the researchers plan to further probe the hydrogel's "memory" by examining the mechanisms behind its memory and by testing its ability to perform other tasks.

Hydrogels can learn to play Pong

Work could lead to new "smart" materials that can learn and adapt to their environment. Pong will always hold a special place in the history of gaming as one of the earliest arcade video games. First introduced in 1972, it was a table tennis game featuring very simple graphics and game play. In fact, it's simple enough that even non-living materials known as hydrogels can "learn" to play the game by "remembering" previous patterns of electrical stimulation, according to a new paper published in the journal Cell Reports Physical Science. "Our research shows that even very simple materials can exhibit complex, adaptive behaviors typically associated with living systems or sophisticated AI," said co-author Yoshikatsu Hayashi, a biomedical engineer at the University of Reading in the UK. "This opens up exciting possibilities for developing new types of 'smart' materials that can learn and adapt to their environment." Hydrogels are soft, flexible biphasic materials that swell but do not dissolve in water. So a hydrogel may contain a large amount of water but still maintain its shape, making it useful for a wide range of applications. Perhaps the best known use is soft contact lenses, but various kinds of hydrogels are also used in breast implants, disposable diapers, EEG and ECG medical electrodes, glucose biosensors, encapsulating quantum dots, solar-powered water purification, cell cultures, tissue engineering scaffolds, water gel explosives, actuators for soft robotics, supersonic shock-absorbing materials, and sustained-release drug delivery systems, among other uses. Further Reading Back in April, Hayashi co-authored a paper showing that hydrogels can "learn" to beat in rhythm with an external pacemaker, something previously only achieved with living cells. They exploited the intrinsic ability of the hydrogels to convert chemical energy into mechanical oscillations, using the pacemaker to apply cyclic compressions. They found that when the oscillation of a gel sample matched the harmonic resonance of the pacemaker's beat, the system kept a "memory" of that resonant oscillation period and could retain that memory even when the pacemaker was turned off. Such hydrogels might one day be a useful substitute for heart research using animals, providing new ways to research conditions like cardiac arrhythmia. For this latest work, Hayashi and co-authors were partly inspired by a 2022 study in which brain cells in a dish -- dubbed DishBrain -- were electrically stimulated in such a way as to create useful feedback loops, enabling them to "learn" to play Pong (albeit badly). As Ars Science Editor John Timmer reported at the time: Pong proved to be an excellent choice for the experiments. The environment only involves a couple of variables: the location of the paddle and the location of the ball. The paddle can only move along a single line, so the motor portion of things only needs two inputs: move up or move down. And there's a clear reward for doing things well: you avoid an end state where the ball goes past the paddles and the game stops. It is a great setup for testing a simple neural network. Put in Pong terms, the sensory portion of the network will take the positional inputs, determine an action (move the paddle up or down), and then generate an expectation for what the next state will be. If it's interpreting the world correctly, that state will be similar to its prediction, and thus the sensory input will be its own reward. If it gets things wrong, then there will be a large mismatch, and the network will revise its connections and try again. There were a few caveats -- even the best systems didn't play Pong all that well -- but the approach mostly worked. Those systems comprising either mouse or human neurons saw the average length of Pong rallies go up over time, indicating they might be learning the game's rules. Systems based on non-neural cells, or those lacking a reward system, didn't see this sort of improvement. The findings provided some evidence that neural networks formed from actual neurons spontaneously develop the ability to learn. And that could explain some of the learning capabilities of actual brains, where smaller groups of neurons are organized into functional units.

AI made of jelly 'learns' to play Pong -- and improves with practice

A basic artificial intelligence (AI) system made of a jelly-like material hooked up to electrodes can 'learn' how to play the classic video game Pong and improve over time, according to a study published today. The results are a first step towards demonstrating that synthetic materials can use a basic form of 'memory' to boost performance, says Brett Kagan, chief scientific officer at Cortical Labs in Melbourne, Australia. "The system demonstrates memory in a similar way that a river bed records a memory of a river," he says. In 2022, Kagan and his colleagues showed that a system made of neurons in a dish -- known as DishBrain -- can learn to play the table-tennis-like video game through electrical stimulation. Inspired by this work, Yoshikatsu Hayashi, a biomedical engineer at the University of Reading, UK, and his colleagues wondered whether a non-biological material could also master Pong. Hayashi and his colleagues turned to hydrogels -- jelly-like materials that are used for a variety of applications, such as components for soft robots -- that contained charged particles called ions. When this type of hydrogel is electrically stimulated, the ions move through the material and drag water molecules along with them, causing the hydrogel to change its shape. This change in the distribution of ions influences the next set of particle arrangements, says Hayashi. "It's like a physical memory." To test whether this 'memory' could enable the hydrogel to play Pong, the researchers used electrodes to connect the material to the game on a computer. The game was divided into a grid of six squares that corresponded to six pairs of electrodes. Every time the ball moved through one of the squares, the corresponding electrodes sent an electrical signal to the hydrogel, causing the ions to change position. Then, sensing electrodes measured the electrical current of the rearranged ions and relayed this information back to the computer, which it interpreted as a command to move the game paddle into a new position. Over time, this formed a basic 'memory' because the ions' movement was affected by their past rearrangements. At first, the hydrogel hit the ball about half of the time, but it increased its hit rate to 60% in around 24 minutes, indicating that the material updates its 'memory' of the ball's movement using the pattern of ions. The improved performance also resulted in longer rallies -- the periods when the ball is in play. The researchers conducted control experiments that involved feeding the hydrogel the wrong information about the ball's position or making it operate 'blind' by not stimulating it at all. That meant the positions of the gel's ions did not accurately reflect the game on the screen. The hydrogel's Pong play showed no signs of improvement under these conditions, suggesting that it gets better only when fed the correct information. The hydrogel didn't master Pong as quickly as DishBrain, which took less than 20 minutes to perform at its best. "Hydrogels are a much simpler system," says Hayashi. But he adds that the results suggest that hydrogels have further computational abilities that could help researchers to develop more efficient algorithms. "The authors took a creative approach to try to adapt concepts from neuroscience to a more physical-based system," says Kagan. But more work needs to be done to show that hydrogels can truly 'learn', he adds.