Microsoft collaborates with Volkswagen to bring augmented reality into cars

Microsoft is working with Volkswagen to carry augmented reality to cars through a new “Moving Platform” mode for the Hololens 2. Microsoft has officially declared a new “moving platform” feature for the HoloLens 2, which is intended to let the augmented reality headset work in places like cars. It addresses a long-standing HoloLens issue of moving environments confounding the headset’s sensors. The upgrade was created as a team with Volkswagen, which has been exploring different avenues regarding involving the headset as a heads-up display in its vehicles.

As Microsoft’s blog post explains, its augmented reality headset tracks movement using a combination of camera sensors and an inertial measurement unit (which typically incorporates accelerometers and gyroscopes). However, in a vehicle, the readings from these two sensors can struggle; the headset senses movement yet sees a static environment. In other words, it was getting car sick.

Neither one of the organizations meticulously describes the situation concerning how this will function, however, there is some sign regarding what’s in store. Microsoft, for example, said “Moving Platform” could be used to train drivers on the most proficient method to deal with adverse road conditions.

This isn’t tied in with training everyday drivers. Instead, the focus, likewise with all the other things in the HoloLens 2 orbit, is business and enterprise, and that implies this platform could be used to train, for example, fleet drivers.

That is the very thing VW found after it began researching the use of augmented reality headsets to show drivers how to get around a racetrack faster. It began working together with Microsoft to fix the sensor issue in 2018, and, eventually, the two developed a prototype system that permitted a vehicle to display real-time data on a connected headset.

The system allows virtual objects to be set both inside and outside of the vehicle. One image released by Microsoft shows the HoloLens 2 projecting a virtual map onto the dashboard of a car, with navigation arrows showing up ahead at key intersections. A second shows it making the driver aware of an upcoming passerby crossing.

Enhanced with 3D models

Taking a gander at the prototype images, the mode works by placing 3D models inside and outside a vehicle. You can see them by wearing the HoloLens 2 mixed reality headgear. For navigation, you’ll have a map for the route virtually placed on top of the dashboard with an arrow highlighting where you want to go.

The HoloLens 2 will likewise show a holographic pedestrian warning at crosswalks to advise you to watch out for individuals. It will even let you know the temperature inside the car.

Microsoft tested the Moving Platform prototype by having Volkswagen scientists associate one of its vehicles with the HoloLens. The Volkswagen team then worked with Microsoft, offering feedback on the best way to further develop the HoloLens Moving Platform mode.

This update depends on Microsoft’s current version of Moving Platform, which is intended to be involved on ships in the ocean. Ocean waves can upset the HoloLen 2, so the mode balances out the gadget’s view.

Microsoft isn’t halting with cars and boats. There are plans to carry its Moving Platform to ” …elevators, trains, [and] other moving environments.”

Rescuing HoloLens

This new partnership with Volkswagen might be a method for reviving the HoloLens project. It has gone through a turbulent time recently.

Back toward the beginning of February 2022, the HoloLens 3 was canceled. The issue was nobody knew what the motivation behind the headset was, which might be the reason a HoloLens 3 colleague escaped to work for Meta on its metaverse projects.

With this Moving Platform Volkswagen partnership, however, Microsoft is solidifying HoleLens 2’s purpose as a mixed reality tool for training and virtual information in a steadily developing array of workplace scenarios. One might dare to dream that HoloLens headsets and Moving Platform one day end up in secondary school driver’s education classes, as well.

VW’s current vehicles now integrate some augmented reality elements. Its new ID electric cars feature an augmented reality heads-up display that projects information from the vehicle — including flow speed and navigation instructions — onto the windshield, where it’s simpler so that the driver might be able to see without taking their eyes off the street.

Obviously, given Microsoft is solidly focusing on the $3,500 HoloLens 2 for enterprise users, there’s no indication of it getting a consumer rollout anytime soon. (Even though Microsoft’s Marc Pollefeys, who worked on the project, says it’s interested in consumer use cases “in the longer term.”) Instead, Microsoft proposes the first recipients of the new functionality could be maritime organizations that can connect laborers with remote mechanical specialists who can glance through a laborer’s HoloLens 2 to diagnose an issue.

Maybe in particular, obviously Microsoft is still actively developing its augmented reality headset platform notwithstanding reports from earlier this year that the project was experiencing hardships. The organization has purportedly lost 70 workers from its HoloLens team over the past year, with over 40 heading for Meta to help its attempt to turn into a metaverse organization.

HoloLens 2 was likewise at first expected for enterprise customers, not your everyday driver, because of the two its specs and its $3,500 price tag. In any case, Microsoft botched its last HoloLens deal so seriously that it seems OK the organization is more open now to elective purposes. All things considered, because an AR headset isn’t good for battle doesn’t mean it won’t work for a drive around.