Engineers from the American startup Generalist AI have developed and presented a multimodal foundational model called GEN-1.5, designed for robot control.
This model has the ability to learn new physical actions from just one brief demonstration provided in the context window, or through minor fine-tuning using a gradient method in one or several steps.
During tests, the robot controlled by GEN-1.5 demonstrated improvisation skills. Specifically, it was able to use a banana as a substitute for a brush to complete the task of collecting cubes into a bowl. Details of this achievement were published on the company's official blog.