Imagine self-driving cars in a city like Mumbai. They would have to be intelligent enough to navigate through the eternally busy streets. And, to navigate right, these vehicles would need data —lots of it.
This data usually comes in the form of detailed maps that can be interpreted by sensors and onboard computers. These maps are the Rosetta Stone for autonomous vehicles and are one of the most important aspects of the whole equation.
Mapping is key to autonomy
Self-driving vehicles are technology-enabled smart machines. They struggle to understand the nuances of driving, unlike humans. Understanding local rules, location-wise norms, signages in different languages (especially in India) and local symbols that may vary from place to place may be difficult for artificial intelligence (AI) systems to keep up with. A human, even if new to a place, will have the intelligence to build an understanding of local traffic laws or the ability to deconstruct a sign that says: “On odd dates, park on the left. On even dates, park on the right.” But this could be difficult for the current technology to comprehend and act on in real time.
For self-driving cars to navigate without hindrances, it is important for technology developers to map the world’s roads as a blueprint. The idea is to build high-definition maps with all the information in advance. This way, self-driving vehicles can know the common signs and symbols, thus reducing the reliance on computer vision systems to detect the sign or symbol, understand it and act accordingly on the go. Such high-definition data and information will boost accuracy and reduce dependency on real-time processes for better navigation. It will not only help build trust for self-driving vehicles in public, but also aid in securing regulatory approval to bring this technology on the road soon.
Enabling mapping – process, opportunities, challenges
For an autonomous vehicle, our traditional maps are insufficient. For a self-driving car, a map has to give a lot more detail – for example, the height of a speed breaker, distance of the pavement from the driving lane, width of the road, exact location of a traffic light, etc. This needs a completely different way of mapping cities, and it requires a lot of data collection to make it into a meaningful piece of information for the autonomous system.
For example, if we were to create a map for a self-driving car to be used in Mumbai, we would build a detailed picture of what is around using sensors and cameras. As the car drives through the city, laser pulses would create a three-dimensional picture of Mumbai. Through this, the car would determine the distance between locations and between two speed breakers, and dimensions of other road features based on the amount of time required for the light beam to bounce back to the sensors. This data is turned into useful information by the mapping and tracking team to categorise vital features on the road, such as intersections and traffic signals, tolls, school signs and many more.
This high level of detail assists the car to exactly know its position and navigate with higher accuracy. As it autonomously navigates the streets, smart software can help it to match the built maps with real-time data captured by the car. This also reduces dependency on technologies such as GPS or conventional maps to navigate. This further allows the car to concentrate on moving pedestrians, animals, other vehicles, construction zones, etc, and avoid accidents or risky situations.
However, our roads are ever-changing and thus self-driving cars need the ability to recognise real-time conditions and adjust accordingly. For example, in Mumbai, a self-driving car needs the ability to recognise signs of road or metro construction (work in progress, uniform of construction workers, written messages on a board) and understand that a road might be closed or diverted, so that an alternative path is identified.
To keep these maps updated, self-driving cars can send automatic reports of their journey to the mapping teams and feed in new elements or changes on the roads. These can be quickly updated into the system maps and the information can be shared with the autonomous fleet which is on the road. Similarly, a connected car (a vehicle equipped with internet access) can gather data on its journey and share the same with other connected cars and update them in real-time. This way, the fleet of self-driving cars can learn of new information and update the same in their systems to navigate better. Some ride sharing companies are using this method to update navigation maps for their fleet and improve the transit for travellers.
MobilEye, an Israel-based company, is the world’s largest supplier of advanced driver assistance systems (ADAS). These systems use an array of sensors such as cameras, radar or lidar and an advanced computer chip to power the safety features that, for example, beep when a driver is about to back into a wall or automatically brake to avoid a crash. Of the roughly 100 million driver assistance systems on the road today, about 80% of them are built by MobilEye, which sells ADAS chips to more than 30 carmakers.
The company has been reported to test an early version of its software, which operates a fleet of robo taxis in Tokyo, Paris, Shanghai, and Detroit. Key to this effort is MobilEye’s plan to map out traffic lights, crosswalks, and other key features of streets around the world, which the company believes will help it meet safe driving standards before other self-driving car developers.
Companies like Tesla, however, primarily rely on cameras, believing computer vision will improve enough within a few years so that AI can navigate roads by just using visual data from onboard cameras. This technology is nascent and requires further development.
Technology and approaches for mapping
In March 2022, Nvidia launched a new mapping platform with a vision to provide the autonomous vehicle (AV) industry with ground truth mapping coverage of over 300,000 miles of roadway in North America, Europe, and Asia by 2024. The platform, Drive Map, is built to enable high levels of autonomous driving. Drive Map enhances the company’s existing solutions for the industry.
Nvidia has also announced its next generation of Drive Hyperion, a sensor and compute self-driving toolkit used by automotive giants such as Mercedes, Volvo, JiDu, BYD and Lucid Motors to provide a variety of smart driving and advanced driving assistance features. Along with these, AV companies like TuSimple, WeRide, Zoox and DeepRoute.ai also use Nvidia’s Drive Hyperion.
Drive Map provides centimetre-level accuracy by combining DeepMap’s (a high-definition mapping start-up acquired by Nvidia in 2021) accurate survey mapping with anonymous mapping data crowdsourced from all the vehicles that use Nvidia’s Hyperion architecture. The mapping tool features three localisation layers — camera, lidar and radar — to provide the accuracy needed for autonomy.
All the data extracted from Nvidia customers is constantly uploaded to the cloud as vehicles drive. It is then gathered and stacked onto Nvidia’s Omniverse, the company’s open platform built for virtual collaboration and real-time physically accurate simulation to update the map so vehicles can achieve proper localisation. In the process, Nvidia can more quickly scale its mapping footprint. Additionally, Omniverse uses automated content generation tools to build a detailed map, which is then converted into a drivable simulation environment that can be used with Nvidia Drive Sim, an end-to-end simulation platform for autonomous vehicles.
Another good example is Mapbox, a company founded in 2010. It provides a platform and software development kit that uses AI and augmented reality (AR) to overlay information on its maps. Mapbox then builds tools that allow developers to use its maps in their own services in a wide range of industries, including media, logistics, agriculture, government, real estate, and drones. It’s being used for Snap Maps on Snapchat, the Weather Channel App, the Washington Post election results, and visualising data in Tableau.
In addition to powering these third-party services, the maps collect and anonymise the data and feed it back to Mapbox’s main platform. The company has more than 400 million map endpoints in use, gathering a constant stream of data to enrich those maps.
These companies proposing high-definition maps offer more ready-to-use solutions, but customers cannot self-edit or update the data as the maps are owned by the mapping companies. Furthermore, it is also a time-consuming process to update maps as customers depend on and wait for vendors to cover newer regions. For lack of a viable readymade option, many companies prefer to build their own maps. Ascending the process to generate large volumes of tremendously precise annotated map data creates the same challenges as scaling data pipelines for any other AI application.
Extending mapping capabilities to the skies
Most of the mapping is terrestrial but, with new technologies such as drones mapping, the techniques are being extended to the skies. These drones are capable of capturing ground data (geospatial data) to build maps of the ground below. However, a sky full of unmanned objects could pose a challenge. So, the need is to bring order and regulation up above.
Leading this evolution, AirMap, based in Santa Monica, California, has created an unmanned traffic management (UTM) platform for drones to help them navigate the complex regulatory and geographical challenges they encounter when flying. AirMap works by combining mapping technology with sophisticated databases it has created of public regulations around drone use. These regulations could be especially complex and extremely specific. For example, limits on how close a drone can fly to a school means that a drone operator needs to know where every school in a region is located.
Need for human experts
Technologies like self-driving cars, drones, coupled with AI systems, are carrying out the mammoth task of collecting data and feeding intelligent systems to build maps for autonomous mobility. But they still need human assistance to learn and deploy functions efficiently. Humans are needed also to monitor and track processes such that the data is expertly processed, and edge cases are identified and solved quicker.
In the case of self-driving cars, the autonomous system lacks the human ability to understand certain nuances. For example, a self-driving car may not have the ability to judge a road blockage correctly. A temporary blockage due to a public procession or other reasons could be fed into a system as a permanent one. When a human is monitoring the data collected by cameras on a self-driving car, the human can differentiate the blockage correctly and improve the systems mapping process. This way, companies can utilize human intelligence to avoid errors and unnecessary elements when mapping for autonomous vehicles.
For example, iMerit collaborates to deploy AI and machine learning in the computer vision, natural language processing and content services ecosystems by enriching, annotating and labelling data. The iMerit workforce extracts intelligence from remote sensing datasets such as satellite, aerial, and drone imagery to power the development of AI technology in mapping.
Mapping technologies and companies working towards building the most efficient tools are still exploring the right ways to build accurate maps of the world. However, the task is more challenging than one can imagine. With the advent of emerging technologies, we are entering an exciting time of experimentation with some of the smartest innovations ever to empower autonomous mobility. The day is not far off when self-driving cars will be the norm, safely navigating megacities like Mumbai.
Authors:
Siddhartha Bal
Director of Autonomous Mobility
iMerit
Siddhartha Bal is the Director of Autonomous Mobility at iMerit, the industry leader in data solutions for machine learning and artificial intelligence. His team has annotated over 100 million images for computer vision projects and is considered expert in developing unique solutions for solving some of the most complex use cases in the industry. He works with clients to create an end-to-end solution, ensuring projects run with operational efficiency and produce the high-quality data required.
Published in Telematics Wire
