Study Reveals How Perception Shapes AI Creativity Assessment

Is AI truly creative? Study shows how visibility of process shapes perception

What makes people think an AI system is creative? New research shows that it depends on how much they see of the creative act. The findings have implications for how we research and design creative AI systems, and they also raise fundamental questions about how we perceive creativity in other people. The work is published in the journal ACM Transactions on Human-Robot Interaction. "AI is playing an increasingly large role in creative practice. Whether that means we should call it creative or not is a different question," says Niki Pennanen, the study's lead author. Pennanen is researching AI systems at Aalto University and has a background in psychology. Together with other researchers at Aalto and the University of Helsinki, he did experiments to find out whether people think a robot is more creative if they see more of the creative act. In the study, participants were initially asked to evaluate the creativity of robots based only on still life drawings they had made. They were told the robots were driven by AI, but in fact it had been programmed to reproduce drawings that the researchers had commissioned from an artist. This deception made it possible to measure people's perception of creativity without requiring the robot to be creative, which would have introduced too much variability between the drawings. Next, the study participants evaluated how creative the drawings were when they saw not only the final product but also a video of the drawing process--the lines appearing on the page, but not the robot creating them. In the final stage, participants scored the drawings when they could see all three elements: the final product, the process, and the robot making the drawing. The findings showed that the drawings were seen as more creative as more elements of the creative act were revealed. "The more people saw, the more creative they judged it to be," says Christian Guckelsberger, assistant professor of creative technologies at Aalto and the study's senior author. "As far as I'm aware, we're the first to study the effects of perceiving product, process and producer in a separate and controlled manner, not only in the context of AI but also more generally." The power of perception Understanding how people assess the creativity of robots or other artificial systems is important in thinking about how to design them--but it's not entirely clear what the appropriate design choices would be. "The study suggests that revealing more about the process and producer can be conducive to people's perception of the systems' creativity," says Guckelsberger. "But if we added elements to make AI systems seem more creative even though the system is in fact performing the same way, we could question whether that's actually a good thing." In some cases, that could be helpful--for example, it might be a way to help people stay engaged with a co-creative system. But in other contexts, it could give people a deceptive impression of how creative an artificial system really is. "Our findings help address this conflict by giving us a better idea of our own human biases. This research makes them a bit more transparent, which is also important from the user's perspective, for us to understand how a system's design affects our perception of it," says Guckelsberger. In addition to these social and design implications, the findings also have significance for research on creative AI systems. If our judgment of creativity depends on how a system is presented, then future studies should control for that factor. Likewise, existing research needs to be reevaluated in light of these findings--comparing the creativity of different systems without accounting for differences in their presentation could have led to false conclusions. Another intriguing question posed by this research is what it tells us about ourselves. "Now that we've found this about people's perception of AI creativity, does it also apply to people's perception of other people?" asks Guckelsberger. Does shape matter? The researchers also carried out the experiments with two different robot designs. Their goal was to test whether people scored the creativity differently depending on the robot's shape, because earlier work had suggested a link between shape and perceived creativity. The team tested whether people saw different levels of creativity when a still life was drawn by a sleek arm-like robot or a more mechanistic plotter robot. Keeping the drawings consistent between the robots and from one participant to another was quite challenging. "I think our biggest difficulty was the physical robots themselves. We did a lot of work with the robots and the drawing process to try to keep everything identical so we could do a scientifically rigorous comparison," says Pennanen. The researchers were surprised to find no significant difference in how people scored the two robots. They're planning future work to look further into this counterintuitive result, as well as what other elements influence our perception of creativity. "We're interested in doing more research about what kinds of biases affect our evaluation of creative and embodied AI systems and how those effects happen," says Pennanen. The findings should also be confirmed for different artistic genres, as well as other forms of art and creative expression. To make it easier for others to replicate their work and build on it, the researchers followed strict open science practices. As artificial systems become commonplace, understanding the factors shaping our perception of their creativity is vital for effective design--and it may also shed some light on how we recognize creativity in humans.

Is AI Truly Creative? Turns Out Creativity Is in the Eye of the Beholder | Newswise

Newswise -- What makes people think an AI system is creative? New research shows that it depends on how much they see of the creative act. The findings have implications for how we research and design creative AI systems, and they also raise fundamental questions about how we perceive creativity in other people. 'AI is playing an increasingly large role in creative practice. Whether that means we should call it creative or not is a different question,' says Niki Pennanen, the study's lead author. Pennanen is researching AI systems at Aalto University and has a background in psychology. Together with other researchers at Aalto and the University of Helsinki, he did experiments to find out whether people think a robot is more creative if they see more of the creative act. In the study, participants were initially asked to evaluate the creativity of robots based only on still life drawings they had made. They were told the robots were driven by AI, but in fact it had been programmed to reproduce drawings that the researchers had commissioned from an artist. This deception made it possible to measure people's perception of creativity without requiring the robot to be creative, which would have introduced too much variability between the drawings. Next, the study participants evaluated how creative the drawings were when they saw not only the final product but also a video of the drawing process -- the lines appearing on the page, but not the robot creating them. In the final stage, participants scored the drawings when they could see all three elements: the final product, the process, and the robot making the drawing. The findings showed that the drawings were seen as more creative as more elements of the creative act were revealed. 'The more people saw, the more creative they judged it to be,' says Christian Guckelsberger, assistant professor of creative technologies at Aalto and the study's senior author. 'As far as I'm aware, we're the first to study the effects of perceiving product, process and producer in a separate and controlled manner, not only in the context of AI but also more generally.' The power of perception Understanding how people assess the creativity of robots or other artificial systems is important in thinking about how to design them -- but it's not entirely clear what the appropriate design choices would be. 'The study suggests that revealing more about the process and producer can be conducive to people's perception of the systems' creativity,' says Guckelsberger. 'But if we added elements to make AI systems seem more creative even though the system is in fact performing the same way, we could question whether that's actually a good thing.' In some cases, that could be helpful -- for example, it might be a way to help people stay engaged with a co-creative system. But in other contexts, it could give people a deceptive impression of how creative an artificial system really is. 'Our findings help address this conflict by giving us a better idea of our own human biases. This research makes them a bit more transparent, which is also important from the user's perspective, for us to understand how a system's design affects our perception of it,' says Guckelsberger. In addition to these social and design implications, the findings also have significance for research on creative AI systems. If our judgment of creativity depends on how a system is presented, then future studies should control for that factor. Likewise, existing research needs to be reevaluated in light of these findings -- comparing the creativity of different systems without accounting for differences in their presentation could have led to false conclusions. Another intriguing question posed by this research is what it tells us about ourselves. 'Now that we've found this about people's perception of AI creativity... does it also apply to people's perception of other people?' asks Guckelsberger. Does shape matter? The researchers also carried out the experiments with two different robot designs. Their goal was to test whether people scored the creativity differently depending on the robot's shape, because earlier work had suggested a link between shape and perceived creativity. The team tested whether people saw different levels of creativity when a still life was drawn by a sleek arm-like robot or a more mechanistic plotter robot. Keeping the drawings consistent between the robots and from one participant to another was quite challenging. 'I think our biggest difficulty was the physical robots themselves. We did a lot of work with the robots and the drawing process to try to keep everything identical so we could do a scientifically rigorous comparison,' says Pennanen. The researchers were surprised to find no significant difference in how people scored the two robots. They're planning future work to look further into this counterintuitive result, as well as what other elements influence our perception of creativity. 'We're interested in doing more research about what kinds of biases affect our evaluation of creative and embodied AI systems and how those effects happen,' says Pennanen. The findings should also be confirmed for different artistic genres, as well as other forms of art and creative expression. To make it easier for others to replicate their work and build on it, the researchers followed strict open science practices. As artificial systems become commonplace, understanding the factors shaping our perception of their creativity is vital for effective design -- and it may also shed some light on how we recognize creativity in humans. A pre-print version of the paper is now available at: https://doi.org/10.1145/3711939. The final publication will be available from the same link on 7 April 15:30 EET.