The requirement of indoor mapping for localising vehicles indoors, enabling Autonomous Valet Parking in the future

The next step in autonomous and mobility services will bring the ‘blue dot’ indoors, with multi-modal route planning and Automated Valet Parking (AVP) made possible by indoor vehicle localisation, achieved without the use of the Global Positioning System (GPS). 

According to Harvard University research, parking facilities currently take up as much as one-third of ground-level space in some cities, increasing demand for space-optimising indoor and underground options. Demand is particularly high in smart cities, where on-street parking is being reduced to accommodate greener living; with wider roads built to improve traffic flow and incorporate growing last-mile services. 

Indoor and underground parking facilities maximise ground-area usage, but come with the drawback of blocking the line-of-sight to satellites, preventing traditional navigation via GPS. As such, an alternative method of vehicle localisation is needed to maintain the navigation features which drivers are accustomed to on the road, as well as enabling Mobility as a Service (MaaS) and convenience services of the future, such as AVP. 

Autonomous Valet Parking 

AVP has the potential to improve efficiency and reduce stress, emissions and time wasted on parking, with benefits such as allowing a driver to be dropped off in front of a car park or their final destination, with the vehicle then autonomously parking itself. 

AVP can improve safety as drivers will no longer need to walk around poorly lit parking facilities or navigate around moving vehicles. AVP can also avoid unnecessary congestion and pollution through real-time dissemination of parking space availability to connected autonomous vehicles.

For parking facility operators, AVP utilises parking spaces more efficiently, in some cases by as much as 20%, by allowing tighter, and double parking, as well as optimally distributing vehicles within the available parking real estate. While the low speeds of AVP also mean a much lower risk of damage to people, cars and infrastructure.

For automakers, AVP is seen as the catalyst for wider autonomous driving features, such as enabling EVs to be autonomously charged or vehicles to be washed and serviced, then automatically returned to a parking spot that is convenient for the driver to find.

AVP is likely to be the first SAE Level 4 automation product made publicly available to drivers, due to the lower cost of implementation and lower risk profile of low speed driving in a constrained environment, however, it requires consistent and reliable global navigation within the entire area of operation. This localisation can be achieved by using advanced robotics techniques, sensors that are already available on most connected vehicles and landmarks that are already present in all car parks. 

The Parkopedia AVP Project 

In 2020, Parkopedia demonstrated automated valet parking with HD Maps of indoor car parks that are suitable for both navigation and localisation, using vision-based localisation techniques based on artificial landmarks (fiducial markers). 

For the project, Parkopedia led a consortium that included The University of Surrey and The Connected Places Catapult (CPC), which was funded by the Centre for Connected and Autonomous Vehicles (CCAV) and InnovateUK, and focused on the development of solutions for SAE Level 4 automation and above. 

One of the key objectives was to develop a vision-based indoor positioning system that will enable decimeter level accuracy based on on-vehicle cameras, wheel odometry, IMU and Indoor Maps, while demonstrating the solution using Parkopedia’s existing self-driving vehicle, fitted with representative automotive-grade hardware and cameras.

Parkopedia’s proprietary state-of-the-art Simultaneous Localization and Mapping (SLAM) system integrates Light Detection and Ranging (LiDAR), inertial measurement unit (IMU), Global Navigation Satellite Systems (GNSS) and high-resolution imagery collected from a compact 3D mobile mapping rig to generate highly accurate colourised point clouds that represent the parking facility.

The project found that the installation of fiducial markers (similar to QR codes) reflected in the map, alongside crucial computer vision algorithms, to calculate the position and orientation of the vehicle, enabled the localisation accuracy needed to allow the automated vehicle to navigate the car parks safely. However, the requirement for the design, installation and maintenance of hundreds or thousands of markers in a car park presents a considerable economic challenge that threatens the commercial viability of AVP. As such Parkopedia is now replicating this demo, using common objects that occur in all applicable car parks (natural landmarks), and adapting our existing technology for production use cases via our Indoor Mapping product. 

Following the successful delivery of the AVP project, Parkopedia has proved that vehicles can precisely locate themselves using artificial landmarks inside a centimetre-accurate map of a parking facility without access to Global Navigation Satellite System (GNSS). By replicating the AVP demo using natural landmarks (e.g. columns, road markings, signs, etc.) Parkopedia is showcasing that the same result can now be achieved without requiring any modification to the car parking infrastructure itself, or indeed the vehicle.

A natural landmark solution uses features that already exist inside car parks and sensors that are already present on vehicles, meaning minimal investment and maintenance costs for automakers and parking facilities operators. The process is the same as for artificial landmark localisation except that the natural landmarks inside the car park can be captured at the same time as the structure of the car park and be embedded inside the map right from the start. Then objects from the different natural landmark categories are detected using the cameras already mounted on the vehicle – delivering the result required for a truly scalable AVP solution.

Natural landmark categories as established by Parkopedia:

• Columns (2 different types)
• Railings
• Signs
• Road markings
• Doors

Parkopedia’s solution involves training a Deep Neural Network (DNN) to recognise and identify objects in the above categories via the images provided from the on-board cameras. The range and bearing to each landmark can then be calculated using associated algorithms. 

All the natural landmarks identified at the mapping and modelling stage are incorporated into the map so that they can be matched with the observed landmarks. Together with odometry from the car itself, the map will contain the necessary information needed to allow the vehicle to be accurately located within the map of the car park. Parkopedia’s technology can detect landmarks in the images and estimate the vehicle pose at more than 20 frames per second.

This technology will allow drivers to plan journeys to not just their chosen parking facility, but to a specific parking space or location inside that facility. Drivers will also be able to navigate seamlessly between the public road network and parking facility’s internal roads, with the map supporting indoor navigation, automated parking and summoning.

Indoor Maps

An Indoor Navigation service will guide drivers to the most likely available spot that minimises the overall journey time, by optimising a multi-modal route consisting of driving and walking directions, or even enable navigation to ‘hidden’ EV charging stations and find-my-car applications through to Automated Valet Parking use cases. AVP is the logical extension of this and builds upon ADAS and park assist technology, already present in vehicles today. 

When used in conjunction with Parkopedia’s dynamic, predictive availability data, drivers will know if there is a space or EV charger available at their destination at their estimated time of arrival, if not, it can offer alternative suggestions, helping drivers to complete frictionless journeys, and helping manage facility demand and occupancy. 

Indoor Navigation opens up many use cases that are not possible at present, such as:

• Routing to specific parking spots in a facility, e.g. those with an EV charger
• Utilising multi-storey and underground car parks for MaaS, logistics, E-commerce applications
• Driver convenience systems, e.g. Find-my-car applications 

Charging Electric Vehicles 

Despite rapidly growing sales and legislation to encourage drivers to swap to EVs, the highly fragmented public charging infrastructure is still a major barrier to ownership. As many as one-third of prospective buyers will not be able to charge at home, and will be completely reliant on the public charging network so improvements are required immediately. The decrease in on-street parking will also see these EV chargers moved into indoor facilities, however, without indoor navigation capability, drivers will not be able to easily find them. Parkopedia’s indoor mapping product is unique in that it is currently the only commercial available indoor navigation service, and when used in conjunction with its leading parking and EV charger POI data, resolves the most common pain point around public charging.

New use cases for indoor facilities 

Localisation of vehicles within indoor parking facilities allows the facilities to be used for more than just parking. Industries such as car sharing and repurposing of sections of the car park for multi-usage, e.g. green last mile delivery networks, ghost kitchens, and E-commerce applications, such as straight-to-trunk delivery, will thrive with the mass introduction of indoor localisation. 

Usually centrally located, and/or near to other modes of transport such as train and bus stations, parking facilities are an ideal, easy to locate hub for mobility and logistics services. When combined with accurate and reliable parking data, indoor localisation provides a car sharing solution that is finally convenient enough to encourage drivers out of their private vehicles, supporting decarbonisation initiatives. 

Driver convenience features

Whether due to a lack of time or mobility, drivers want to park as close as possible to their final destination, reducing the ‘on-foot’ part of their journey. Indoor mapping services allow for this feature to move indoors, improving journey efficiency as well as drivers safety and security, by being able to park as close as possible to an end destination.

An Indoor Navigation service will guide drivers to the most likely available parking spot that minimises the overall journey time, enables navigation to ‘hidden’ underground EV charging stations and also find-my-car applications. 

Parkopedia’s Indoor Mapping Product is a unique, cost effective, scalable indoor mapping solution for automakers looking to future-proof their navigation and deliver driver convenience services expected today. 

Having successfully delivered multiple indoor mapping proof-of-concept projects for global automakers, covering both industry and private use cases, Parkopedia expects this technology to be rolled out to consumer vehicles within the next few years.

Author:

Dr. Brian Holt leads Parkopedia’s team developing parking solutions for autonomous cars, having previously worked at Samsung Electronics managing on-device AI, vision-based localisation and AR/VR projects.

Published in Telematics Wire