How filming your chores could train the android butlers of the future
The dream of deploying humanoid robots in every home has created a new type of job. The only requirements are a head strap, a smartphone and a list of chores.
With the evolution of artificial intelligence, humanoid robots have become the latest frontier in the race to dominateadvanced technology. Robot makers are rolling out a succession of new models that can walk, dance and fight with increasing agility.
But the holy grail of the burgeoning industry – a general-purpose robot that can work in shops, offices and homes – needs a vast amount of data to learn how to safely and effectively replace humans. Increasingly, that data is being created by people recording themselves doing mundane household tasks.
This has created a voracious appetite for first-person footage that can be used to train robots, also known as “egocentric data” or “human data.” Over the past several months, startups have stepped in to supply that demand by collecting and annotating videos from thousands of contract workers around the world.
“Manufacturing, factory warehouses, retail, nursing homes, hospitals – you’re going to need this type of data in basically every single environment, and that’s because the movements are all different,” said Arian Sadeghi, vice president of robotics data at Micro1, which began recruiting its own army of remote videographers last year.
Each person receives headgear to attach a camera, filming instructions and a list of tasks such as cooking, cleaning, gardening and pet care. Workers are expected to alternate between assignments and submit at least 10 hours of video each week.
While the shots currently revolve around household chores, Sadeghi said the company encourages contractors to experiment with what they film, in case it could eventually help robots adapt more quickly to new environments and responsibilities.
Billions of hours’
Though Micro1 is based in Palo Alto, California, it has about 4,000 “robotics generalists” in different households across 71 countries, who send the company more than 160,000 hours of video each month. Sadeghi said that’s nowhere near enough.
You need probably billions of hours,” he said. “We haven’t even gotten to human interactions. This is just simple household chores.”
He said the growing demand for data in robotics mirrors the early trajectory of ChatGPT and other AI chatbots. Trained on hundreds of billions of words harvested from the internet, ChatGPT uses what it’s learned about text patterns to generate the likeliest responses to user prompts.
Following text, AI models evolved to churn out custom images and videos on demand by relying on readily available content online. But robot developers require a much more specific set of training data, and lack the same kind of instant library that the internet previously provided.
That’s become a multibillion-dollar opportunity for startups like Micro1, which also annotate the videos so that robots can differentiate objects, distances and physical movements. Market research firms estimate that the data collection and labeling industry will on average expand about 30% annually, led by growth in Asia, to reach at least $10 billion by 2030.
Ravi Rajalingam, founder of the data annotation company Objectways, provided audio and visual data to train AI-powered virtual assistants and self-driving cars, before shifting his focus to robotics last year. Since he started hiring contractors to collect human data, he’s found that only about half the submitted footage is usable.
Still, with 90% of his customers based in the US, and their assumption that American consumers have the spending power to adopt humanoid robots early, some are willing to pay more for data from US households, even though the hourly wage can be as much as triple that of a worker in Vietnam or India.
“The India kitchen is very different from the US kitchen. A broomstick in India is very different from a broomstick in US. So variety is important, but it depends where you are going to place your robots first,” said Rajalingam. “That’s the reason we are collecting all over the world.”