For decades, the concept of having a robot assistant walking around and carrying out our every request has been a mere figment of our imagination. But, recent breakthroughs in robotics and artificial intelligence have brought us closer than ever to realizing this sci-fi dream.
In a paper published by Microsoft on Monday, researchers explored the AI model’s potential to generate code for robotics hardware.
The experiment proved successful, with ChatGPT successfully writing the code to move a robotic arm and fly an aerial drone. Although the AI model was able to take on a lot of the heavy lifting, some assistance from human-written code was still necessary for completion.
In the experiment, researchers gave Chat-GPT a series of prompts instructing the robotic equipment to complete tasks of varying complexity.
Despite recent advances in robotics, hand-written code is still the main method for controlling robots.
But researches at Microsoft are out to change that.
By eliminating hand-written code in robotics, robots can be programmed in a faster, simpler, and more efficient way. And users with no technical know-how can interact with and command robots.
The diagram below demonstrates this concept:
Credit: Microsoft
The Experiment
Researchers provided ChatGPT with a prompt that outlined the various commands that the AI program could use to control a given robot.
This included information about task constraints and how ChatGPT should form its answers.
To demonstrate the effectiveness of the new approach, the team applied the method to an aerial drone. It was given a long prompt outlining the computer commands it could use to control the drone. Afterward, the team could make requests to ChatGPT to control the robot in various ways. For example, the bot was asked to use the drone’s camera to identify a drink such as a coconut water and a can of Coca-Cola.
ChatGPT successfully processed the commands and controlled the drone.
Conversation needed for complex tasks
Next, researchers instructed ChatGPT to write code to create a Microsoft logo out of wooden blocks.
To achieve this, the researchers taught ChatGPT how to string together APIs into more complex functions. This enabled the model to take the skills it had learned and link them to carry out tasks such as stacking blocks.
Not only was it able to remember the logo from its internal database, but it was also able to ‘draw’ the logo in the form of SVG code, and then figure out which robot actions could be used.
Credit: Microsoft
The approach does have one key limitation. It can only write computer code for a robot based on a text-based request from a human. For the AI program to generate applicable computer code, human engineers must provide a comprehensive explanation of the robot’s application programming interface.
Nevertheless, the integration of ChatGPT with robotics could potentially unleash a new era of robots that are smart enough to understand voice commands.
That’s why need to carefully consider how to safely communicate with them.
Researchers are warning that these tools should not be given full control of the robotics pipeline, especially for safety critical applications. They stress that it’s essential to ensure the quality and safety of the code before executing it on the robot. And that large language models like ChatGPT may generate incorrect responses.
Don’t get left behind by AI
As AI develops, prompting will become more complex and harder to learn. Keeping up with new developments, updates, and prompting features will keep you ahead of the curve whether you’re an artist, graphic designer, business owner, web designer, or hobbyist.
Sign up for our email list to get cutting-edge updates about prompt engineering and design.