DARPA (Defense Advanced Research Projects Agency) is developing a new system to help military personnel perform complex tasks. Perception-enabled task guidance (PTG) technology uses sensors to see and hear what the user sees and hears, guiding them through AI-produced instructions displayed in augmented reality (AR).
PTG combines sensors (a microphone and a head-mounted camera) with AI and AR headsets to blend into the user’s environment. The idea is to help soldiers and other military personnel improve their skills, complete difficult tasks, and perform better. DARPA has narrowed its focus to three areas: battlefield medicine (such as untrained personnel assisting medics in the field), maintenance (keeping military equipment running), and co-piloting (especially helicopters).
However, DARPA training demos use something more pedestrian: cooking. Dr. Bruce Draper, the program’s administrator, describes it as the ideal proxy task. “[Cooking is] a good example of a complex physical task that can be done in many ways. There are many different objects, solids, liquids, things that change states, so visually it’s quite complex. There is specialized terminology, there are specialized devices, and there are many different ways to achieve this. So it’s a very good practice domain.” The team believes that PTG will eventually find uses in medical training, assessing the competency of physicians, and other healthcare services.
The staff demonstrating the technology appear to be using a variant of the Microsoft HoloLens. The government recently halted plans to buy more “AR combat glasses” from Microsoft, instead approving $40 million for the company to develop a new version. The rollback came after discovering that the current version caused issues like headaches, eyestrain, and nausea.
DARPA is the “mad science” division of the Department of Defense. Founded by President Dwight Eisenhower in 1958 to develop cutting-edge technology for the US, many of the agency’s projects have turned into non-military products, including GPS, voice recognition, driverless cars and robotics. Oh, and a lesser technology called the “Internet” also emerged from DARPA’s ARPANET (Advanced Research Projects Agency Network) program in the late 1960s. It’s easy to imagine some form of PTG eventually following suit in our lifetime. daily.
All Engadget Recommended products are curated by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you purchase something through one of these links, we may earn an affiliate commission. All prices are correct at the time of publication.