Generative AI Produces Only Average-Level Creative Work: Study

A new theoretical study published in the Journal of Creative Behaviour has challenged the belief that artificial intelligence is close to surpassing human creativity. The research, conducted by Professor David H Cropley from the University of South Australia, argues that generative AI models such as ChatGPT are mathematically limited to amateur-level creativity.

Professor Cropley began the study to bring clarity to the ongoing debate about whether AI can truly be creative. He applied the standard definition of creativity, which requires both originality and effectiveness, to the way large language models operate. According to him, this approach helps determine whether the design of AI systems places a natural cap on their creative abilities.

According to the study, large language models work by predicting the next word or "token" based on patterns learned from their training data. Cropley explains that this predictable and statistical method allows for a mathematical evaluation of AI creativity. His analysis shows that while AI can produce outputs that seem creative, its originality is limited by the patterns it has learned, and therefore its creativity cannot rise beyond average human levels.

The study cites previous tests showing that AI-generated stories and ideas usually rank around the 40th to 50th percentile when compared with human work. This supports the conclusion that AI cannot reach expert or professional standards under current design methods.

Cropley notes that most people produce average levels of creativity, which is why AI output can appear impressive to the general public. However, professionals in writing, design, or art quickly notice that AI-generated content is repetitive and formulaic.

According to the study, truly high-level creativity involves producing ideas that are both unique and highly effective - something AI cannot achieve because it relies on existing data. Cropley says AI may assist with routine tasks, but it cannot replace skilled creators or produce groundbreaking work on its own.

The paper also acknowledges limitations. It uses simplified mathematical assumptions and focuses on standard AI settings. It does not measure how human editing or advanced prompting may improve final results. Future studies may explore whether adjusting AI parameters or using different architectures could expand its creative potential.

Cropley concludes that achieving expert-level creativity in AI would require a completely new kind of system - one that can generate ideas beyond learned statistical patterns. Until then, human creativity remains unmatched.