Microsoft’s Drone AI Pilot Training Lab

Sign up for daily news updates from CleanTechnica on email. Or follow us on Google News

---

It makes sense to perform the most hazardous activities in simulations when you’re testing advanced driver assistance systems (ADAS), autonomous vehicle prototypes, or simply automobiles in general. An early autonomous system may be extremely unpredictable, and Tesla warns that they might “make the wrong decision at the worst possible time.” So, before even installing a system on a car, automotive manufacturers are doing a lot of simulator work to prepare the software for reality. At its most extreme, Tesla is building a supercomputer with capabilities far beyond what’s needed.

However, sooner or later, you must validate your work on the road. A system that makes mistakes on a regular basis keeps a driver attentive, according to research, but systems that work flawlessly and just make a mistake once in a while might lull a human test driver into false security, resulting in death and destruction on the roads. As a result, you’ll need to be able to simulate driving that falls somewhere between simulation and reality.

One way companies in the automotive space bridge that gap is to built simulators that take in a whole vehicle, and not just the computer. Taking that approach helps make sure that the vehicle is ready for testing on the streets because the whole vehicle got a good test and it won’t do anything crazy.

It turns out that autonomous aircraft, including unmanned drones, face a similar set of challenges. Throwing a drone out into the air and seeing how it performs can lead to some dangerous situations for the public, but computer simulations for the software doing the flying don’t quite get a drone ready for real testing in real airspace.

So, that’s why it’s exciting to see that Microsoft is doing the same thing as the link above about car simulators, but with drones.

Microsoft’s Drone Simulator Lab

They start by telling us the story of Josh Riedy, the CEO of Airtonomy, when he visited Microsoft’s lab. Using VR glasses, he was following along behind a drone in a virtual windfarm. Throughout the Midwest, Leech is using hyper-realistic simulations to train autonomous aerial vehicles that are now inspecting wind farms, monitoring wildlife, and detecting leaks in oil tanks.

When he looked around, it felt to him like he was really there up in the air with the drone, and that drone’s software was similarly fooled into thinking it was actually in the air.

“You don’t want to fly drones into wind turbines, powerlines or really anything for that matter,” Riedy said. “Coupled with the fact that winter can literally last 7 months in North Dakota, we realized we needed something other than the physical world to design our solutions for customers.”

The data generated by Project AirSim is used to train AI models on which activities to take at each stage of flight, from takeoff to cruising to landing. It will also provide libraries of simulated 3D environments representing a variety of urban and rural settings as well as a set of sophisticated pretrained AI models to assist with the acceleration of autonomous aerial infrastructure inspection, last-mile delivery, and urban air mobility.

Gurdeep Pall, corporate vice president for Business Incubations in Technology & Research at Microsoft, stated that advancements in AI, computing and sensor technology are starting to change how we move people and goods. And this isn’t just a problem in rural areas with wind farms; gridlocked roads and highways can no longer suffice as the quickest method to get from place to place because of urban density growth. Instead, companies will turn to drones for transportation.

“Autonomous systems will transform many industries and enable many aerial scenarios, from the last-mile delivery of goods in congested cities to the inspection of downed power lines from 1,000 miles away,” Pall said. “But first we must safely train these systems in a realistic, virtualized world. Project AirSim is a critical tool that lets us bridge the world of bits and the world of atoms, and it shows the power of the industrial metaverse – the virtual worlds where businesses will build, test and hone solutions and then bring them into the real world.”

The goal of AirSim, an earlier open-source project from Microsoft Research that is being retired but inspired today’s launch, was to provide high-fidelity simulation. AirSim was a popular research tool, but it required significant programming and machine-learning expertise. Microsoft has now turned the open-source technology into an end-to-end platform that allows AAM customers to simulate 3D environments with AI aircraft more easily test and train them .

“Everyone talks about AI, but very few companies are capable of building it at scale,” said Balinder Malhi, engineering lead for Project AirSim. “We created Project AirSim with the key capabilities we believe will help democratize and accelerate aerial autonomy – namely, the ability to accurately simulate the real world, capture and process massive amounts of data and encode autonomy without the need for deep expertise in AI.”

Developers can use Project AirSim to get access to pre-trained AI building blocks, such as high-quality models for locating and avoiding obstacles and precise landings. These out-of-the-box features eliminate the need for deep machine learning expertise, allowing more people to train autonomous aircraft.

Simulink and AirSim are already available on a variety of platforms, including Windows and Android. Microsoft is also collaborating with industry partners to bring accurate simulation to weather, physics, and – most significantly – the sensors an autonomous machine uses to “see” the world. Customers may utilize Ansys’ high-fidelity physics-based sensor simulations to get rich ground truth data for autonomous vehicles through a partnership with MathWorks. Meanwhile, Microsoft and MathWorks are collaborating on ways for customers to import their own physical modeling into Simulinks using Simulink.

Technology will not, on its own, be enough to bring about the age of autonomous flight. The industry must also find a way through existing aviation monitoring systems and regulatory situations. In order to speed up this sector, the Project AirSim team is currently engaged with standards organizations, civil aviation authorities, and legal bodies in developing required standards and means of compliance.

Microsoft’s Pall said the firm wants to collaborate with global civil aviation authorities on how Project AirSim might assist in the certification of safe autonomous systems, potentially providing scenarios within AirSim that an unmanned vehicle must successfully navigate. There is heavy rain, strong winds, and GPS connectivity drops in one scenario. If the car can go from A to B every time under these conditions, Pall says it would be a significant accomplishment toward certification.

AirSim pioneer Ashish Kapoor is ecstatic to have assisted the simulation engine progress from a code-based research tool into a more robust platform that any company can use without technical expertise. An aviator himself, Kapoor cannot wait to see what this development will mean for the aviation industry.

“A ton of data gets generated when an aircraft flies through space in Project AirSim,” said Kapoor, now general manager of Microsoft’s autonomous systems research group. “Our ability to capture that data and translate it into autonomy is going to significantly change the landscape of aviation. And because of that we are going to see many more vehicles in the sky, helping to monitor farms, inspect critical infrastructure and transport goods and people to the remotest of places.”

Featured image provided by Microsoft and Airtonomy.