Robot Assistants: Is Microsoft’s ChatGPT Turning Sci-Fi into Reality?

In a paper published by Microsoft on Monday, researchers explored ChatGPT’s potential to generate code for robotics hardware, and the experiment proved successful.
PROMPT​
retrofuturistic digital art of a futuristic city with gigantic buildings and flying cars, city background, golden hour, clear lines, award winning masterpiece

For decades, the concept of having a robot assistant walking around and carrying out our every request has been a mere figment of our imagination. But, recent breakthroughs in robotics and artificial intelligence have brought us closer than ever to realizing this sci-fi dream.

In a paper published by Microsoft on Monday, researchers explored the AI model’s potential to generate code for robotics hardware.

The experiment proved successful, with ChatGPT successfully writing the code to move a robotic arm and fly an aerial drone. Although the AI model was able to take on a lot of the heavy lifting, some assistance from human-written code was still necessary for completion.

In the experiment, researchers gave Chat-GPT a series of prompts instructing the robotic equipment to complete tasks of varying complexity.

microsoft expermiment with ChatGPT

Despite recent advances in robotics, hand-written code is still the main method for controlling robots.

But researches at Microsoft are out to change that.

By eliminating hand-written code in robotics, robots can be programmed in a faster, simpler, and more efficient way. And users with no technical know-how can interact with and command robots.

The diagram below demonstrates this concept:  

Credit: Microsoft

The Experiment

Researchers provided ChatGPT with a prompt that outlined the various commands that the AI program could use to control a given robot.

This included information about task constraints and how ChatGPT should form its answers.

To demonstrate the effectiveness of the new approach, the team applied the method to an aerial drone. It was given a long prompt outlining the computer commands it could use to control the drone. Afterward, the team could make requests to ChatGPT to control the robot in various ways. For example, the bot was asked to use the drone’s camera to identify a drink such as a coconut water and a can of Coca-Cola.

ChatGPT successfully processed the commands and controlled the drone.

Conversation needed for complex tasks

Next, researchers instructed ChatGPT to write code to create a Microsoft logo out of wooden blocks.

To achieve this, the researchers taught ChatGPT how to string together APIs into more complex functions. This enabled the model to take the skills it had learned and link them to carry out tasks such as stacking blocks.

Not only was it able to remember the logo from its internal database, but it was also able todraw the logo in the form of SVG code, and then figure out which robot actions could be used.

Credit: Microsoft

The approach does have one key limitation. It can only write computer code for a robot based on a text-based request from a human. For the AI program to generate applicable computer code, human engineers must provide a comprehensive explanation of the robot’s application programming interface.

Nevertheless, the integration of ChatGPT with robotics could potentially unleash a new era of robots that are smart enough to understand voice commands.

That’s why need to carefully consider how to safely communicate with them. 

Researchers are warning that these tools should not be given full control of the robotics pipeline, especially for safety critical applications. They stress that it’s essential to ensure the quality and safety of the code before executing it on the robot. And that large language models like ChatGPT may generate incorrect responses.

Don’t get left behind by AI

As AI develops, prompting will become more complex and harder to learn. Keeping up with new developments, updates, and prompting features will keep you ahead of the curve whether you’re an artist, graphic designer, business owner, web designer, or hobbyist. 

Sign up for our email list to get cutting-edge updates about prompt engineering and design.

Join Our Community Become a Prompt Engineer

Recommended from Promptlib

ministories and coloring cover, ebook

Create a Personalized Children’s Book – In Just Hours!...

Make a Custom Children’s Book With AI​ ....

Read more.

futuristic chatbot discussing with a human

Everything You Need to Know About GPT-4: Explained...

After months of anticipation, Microsoft-backed startup OpenAI ....

Read more.

An augmented woman laughing with an orb in the background

Artificial Intelligence Won’t Steal Our Jobs, It’ll Change Them...

With AI’s capability to complete generalized tasks ....

Read more.

Collage of midjourney ai bot generated photographs

Creating Characters with AI: Midjourney Guide for Artists, Writers, and Game Designers

futuristic chatbot discussing with a human

Everything You Need to Know About GPT-4: Explained

Robot Assistants: Is Microsoft’s ChatGPT Turning Sci-Fi into Reality?

Seamless Tiles, The First Real Use Case for AI-Generated Art?

An augmented woman laughing with an orb in the background

Artificial Intelligence Won’t Steal Our Jobs, It’ll Change Them

Laura Rzoska
Writer at Promptlib.
Collage of midjourney ai bot generated photographs

Creating Characters with AI: Midjourney Guide for Artists, Writers, and Game Designers

futuristic chatbot discussing with a human

Everything You Need to Know About GPT-4: Explained

Robot Assistants: Is Microsoft’s ChatGPT Turning Sci-Fi into Reality?

Seamless Tiles, The First Real Use Case for AI-Generated Art?

An augmented woman laughing with an orb in the background

Artificial Intelligence Won’t Steal Our Jobs, It’ll Change Them

Welcome back!

We're so excited to see you again!

Upload Successful

Congratulations! Your upload was successful and is now pending review.