As inventors and intellectual property consultants, we have been super impressed with ChatGPT as a tool that can often completely answer any question with lots of backup information. ChatGPT certainly appears to replace a lot of what an internet search does today, delivering all sorts of specific text that does not require a user to figure things out for themselves by scrolling through links and reading websites.
We have been working in the field of creativity and invention for decades and have analyzed many AI-based tools that assist with, or even take a stab, at inventing. So, we decided to analyze ChatGPT to see how well it invents. Before we get started, it should be noted that ChatGPT was not designed to invent or be creative. ChatGPT will tell us that it is not inventive or creative when asked. We wanted a working view of how close ChatGPT comes to creativity and invention. Also note that we have been analyzing techniques for decades on being creative and inventive. There are no standard tests for these questions and the ones that come closest all deal with personal behaviors and personalities.
Can AI Invent?
To answer this question, we started by asking the question, Can AI invent? From the history we have followed and written on, we can never really answer the question unless we can get a patent through the patent office. If an AI-generated application became a granted patent, it would become a gold standard for saying the AI can invent. The invention question may be satisfied by proxy of getting a patent. But, except for a single instance, called the DABUS case in the South African patent office, no other AI has been named an inventor! To answer the question, then, “Can ChatGPT invent?” we need to apply what we know as the basic thinking behind invention. We have trained thousands of inventors worldwide with the fundamental basics of invention thinking, so if we apply the same fundamentals, ChatGPT has to be able to perform associative thinking tasks and invention enablement. These two high-level processes come together to create invention. So, we tested how well ChatGPT does with associative thinking and enablement. In this post, we break down our findings.
First, we need to understand some further background of associative thinking and enablement. These basics tell us that inventions need creative ideas that can be enabled. A creative idea has to be new and useful. A patentable invention must be new, useful, and non-obvious. There are some rules associated with obtaining a patentable invention. The key ones that deal with invention and creativity are the rules that deal with novelty over the prior art and obviousness. Many inventions do not have to be patentable to be considered an invention. In the case of non-patented inventions, no standards association can tell us whether a non-patented invention is an invention. But, to understand the nuances of this question, we must answer the question, does ChatGPT demonstrate an ability to contribute to the conception of invention when asked questions that illicit a creative, inventive response?
Testing ChatGPT for Invention
We did some testing using a set of creativity and enablement tests but with limited sample sizes and content areas to test. The tests we performed attempted to get us a first-pass analysis. ChatGPT could not do complex or even simple associations to get new ideas at a high-level summary. ChatGPT even explicitly stated that associations we were testing were “not suitable” or “not recommend“! ChatGPT could do associations when the associations were much more detailed, which makes sense for a great search tool to get better the more detail you add to the questions. As the questions we asked got closer to a completed association, the better ChatGPT was able to produce text that looked like an invention. So, we rate ChatGPT poor at associations in this analysis. Association was the first measure of creativity we looked at.
ChatGPT also had poor fluency in its ability to produce different responses given the same stimulus of associations. This fluency result is likely one of the root causes of ChatGPT not being creative as a language model search tool, as ChatGPT shouldn’t give a new answer each time! So, from the outside, ChatGPT likely is designed away from developing new ideas. So, we rate ChatGPT poor at fluency, which is another measure of creativity.
However, in our opinion, the power of ChatGPT could easily transition to a tremendous inventing machine if we could give ChatGPT a creative mode where it is not just searching probabilistically for a close match.
ChatGPT demonstrated that when the associations requested were on knowledge domains further away from each other, ChatGPT did even worse. So, for example, prompting an invention combining a toothbrush and toothpaste allowed ChatGPT to perform comparatively well. However, prompting an invention that combines a toothbrush with sandpaper was so disparate, ChatGPT wouldn’t even try to create an invention. ChatGPT was poor at associating novel domains. This is further evidence of poor creativity.
If ChatGPT knows that associations are far away, instead of saying it can’t combine the associations, ChatGPT could use the result as a trigger to go into inventing mode! As part of the overall patterns, ChatGPT’s response to our creativity tools showed uncreative results.
Given what we know today, we do not think ChatGPT is designed to be creative. When we asked ChatGPT if it was creative, it responded:
“As an AI language model, I do not have personal feelings, emotions, or creativity the same way a human would. However, I have been trained on a large corpus of text and can generate text based on patterns in the data. This means that I can generate novel and original text, but it is not the result of a creative process in the sense that a human would understand it. Instead, it is based on patterns and probabilities in the data I was trained on.”
ChatGPT
Promising Invention Enablement from ChatGPT
At another level, in the second part of the invention question, which is enablement thinking, ChatGPT was excellent at enablement. ChatGPT may not offer novel ideas to start with, but if you keep asking more detailed questions, ChatGPT comes up with more and more information to draw upon. We rated the enablement as excellent. In most cases where we asked ChatGPT to tell us how something could work, it would quickly provide usable results
Since ChatGPT is excellent at answering questions at precise levels, if the right questions are asked and a fair number of drill-down questions are asked, one might imagine that ChatGPT may be able to generate specific answers that seem creative to a set of drill-down questions. However, we never saw a response that would contribute to the conception of an invention. ChatGPT itself said,
“As an AI language model, I don’t have the capability to invent things in the traditional sense, but I can assist with generating creative ideas and text and providing information about various inventions.”
ChatGPT
When we tested enablement, we recognized that anyone using ChatGPT to drill down on creating inventions might walk away thinking that ChatGPT is an inventor or a co-inventor. We think this is an incorrect assumption. Although this will undoubtedly be open to debate and eventually we may see this debate make its way the Supreme court.
Conclusions About ChatGPT’s Ability to Invent
However, invention is a two-part test. It must be creative (new and useful), and it must be enabled. ChatGPT does not invent today as it simply exhibits no creative capability and no conception of invention.
We recognize that many might disagree, especially when a user keeps asking ChatGPT questions and ChatGPT keeps answering these questions in more and more detail. But inventors don’t become inventors, whether in a brainstorming session or not, by only answering specific questions without having conception of some new connection.
Based on our definition of invention, in order to be an inventor, ChatGPT must be creative and ChatGPT must enable. The first test fails. We did not see creativity, so even if ChatGPT can help enable an invention excellently, it is still not capable of inventing.
In our opinion, ChatGPT exhibits no creativity in associations and does not conceive of inventions, but ChatGPT enables them exceptionally well. It should be noted that ChatGPT itself tells us it cannot be creative or inventive in the usual sense. It wasn’t programmed to do this.
But it could be in the future!