This feature originally appeared in the Automated newsletter. Subscribe here.
Ken Goldberg certainly isn’t the first roboticist I’ve spoken to who’s expressed a kind of pleasant surprise reflecting on generative AI breakthroughs over the last couple of years. Toward the end of our recent podcast interview however, the Berkeley professor turned startup co-founder expresses a much more fundamental shift in his own thinking spurred on by these technologies.
“I didn't see any evidence of [creativity in machines] for many years,” he notes. “It seemed to me that machines couldn't do that. You need humans to be creative, to identify what was new, what was interesting. But I am changing my view, and I have admitted that I'm wrong in the terms of images. We now have systems that you can describe an image and it will generate an image that will often surprise you. And that I did not predict.”
You can view a wide cross section of Goldberg’s own multimedia work via an online portfolio hosted on his U.C. Berkeley site. The works all incorporate technology to varying degrees — some more subtly than others. There’s a 1/1 millionth scale model of one of Frank Lloyd Wright’s most iconic buildings and colorful spheres generated from seismic data derived from the Hayward Fault. There are paintings by robots dating back to the late 80s and a project in a greenhouse we toured a few years back, wherein a system is “learning how to garden.”
We’re wandering into some murky philosophical territory when considering the question of whether AI and machines can be “creative.” Googling the question, Gemini will force its way to the top of your search results and insist, “Yes, AI can be creative.” Scroll down a spot for a human take published in a recent issue of Nature.
“AI can produce a creative product, sure,” educational psychologist and verified human person, James Kaufman, says in the piece. “But it doesn’t go through a creative process. I don’t think it’s a creative entity.”
For Goldberg, those late-80s robot paintings don’t cross that particular threshold, while recent AI do. He cites a test similar to J.P. Guilford’s Alternate Uses Task (AUT), in which participants are asked to come up with as many uses as possible for an object within a given amount of time.
“The example I gave it was how many uses can you find for a guitar pick, and it generated [a] hundred uses instantly and one of them was a miniature sale for a toy boat,” the professor notes. “I would never have thought of it. And I was like, that's great, that's creative. Okay. I cannot deny that. But now, you know, it's doing lots of things. And I think that that ability to be creative is, is really exciting. That's a frontier.”
Goldberg’s students are conducting their own fascinating research into the intersection between generative AI, robotics, and, potentially, creativity. Of the numerous recent projects he sent me in the lead up to our conversation, it was Blox-Net that really stood out. The paper, which was presented late last month, describes how visual language models (VLMs) can assemble physical blocks into designs based on language designs. As evidenced by the “giraffe” at the top of this piece, things can get pretty abstract, but that’s more to do with the limits of the blocks themselves.
“Blox-Net achieved a Top-1 accuracy of 63.5% in the semantic accuracy of its designed assemblies,” the team notes in the paper. “These designs, after automated pertubation redesign, were reliably assembled by a robot, achieving near-perfect success across 10 consecutive assembly iterations with human intervention only during reset prior to assembly. Surprisingly, this entire design process from the textual word to reliable physical assembly is performed with zero human intervention.”
