Mimic and Black Forest Team to Train Robots

For robots to be truly useful in both industry and the home, they have to learn new tasks and adapt to their surroundings with the same flexibility of their human colleagues. But current training methods are slow and rigid. mimic robotics believes its methods could change that.

Zurich-based mimic announced Thursday that it has teamed up with AI research lab, Black Forest Labs, to release FLUX-mimic, a video-action model. FLUX-mimic is a video and image generation world model that uses an action decoder to break tasks into executable segments. The company claims this approach allows robots to learn new tasks in as little as 30 minutes. 

Mimic cofounder and CTO, Elvis Nava, told A3 that the team has essentially added a robot modality on top of previously trained, robust image and video model. These models are designed to predict what a robot will do the same way image generation LLMs generate pictures. 

Per Nava, training robots with video generation separates mimic from competitors that use vision-language-action models, which are trained on image and text. Because VLA models aren’t trained on video, they can require additional (usually extensive) amounts of said data to teach a robot a new task. 

“Fundamentally it accelerates a lot of our timeline, and we are able to kind of leapfrog [to] state-of-the-art in robot learning,” Nava said about the company’s approach. “We are able to get a model that is already very good and requires minimal amount of finetuning. Yhe better the model already is, the less fine-tuning data we will need when going to factories.”

Nava says he has followed Black Forest Labs founders’ research on stable diffusion since his PhD program at ETH Zürich, prior to befriending co-founder and CEO Robin Rombach.

“You could argue that they fundamentally invented modern AI image generation,” Nava said about Rombach and his co-founders. “I applied a lot of that research into robotics and that's kind of how we got started in inventing the whole application of this aspect to robotics. When we [got] into contact with them again this year, it became very obvious that it was a very nice match.”

FLUX-mimic is already deployed out in the wild with automaker, Audi, which has been partnered with mimic from the beginning. Nava said that the company was very intentional about taking on customer projects from the outset, so the deep tech lab’s long-term bets will result in products companies will actually buy.  

“The goal of these customer projects was always to frame and ground what we are doing in terms of KPIs that are useful for industry,” Nava said. “We want to benchmark the models doing useful manipulation tasks in a way that is useful for industrial adoption. It doesn’t matter to fold the laundry, if we don’t know who is going to pay for that. The point was to find tasks that are worthwhile.”

Now that this new model is out, mimic is focused on commercialization, Nava said. The startup hopes its full-stack approach, including recently released gloves and robotic hand for capturing human training data, will help it stand out in an increasingly crowded category. 

“From the customer perspective, what we do is very simple, it is just to automate the task,” Nava said. “And that means that we have to know the full stack of how to get there and how to do it.”