Tesla gives peek at supercomputer it says will replace “meat” drivers

Tesla has given a sneak peek at the supercomputer behind its vision-only autopilot system, claiming it is number five in the world in terms of computing power and should allow human “meat computers” to safely handover driving duties to their silicon counterparts for good.

The computer, a precursor to Tesla’s mysterious Project Dojo supercomputer program, is allowing the electric carmaker to solve the problem of designing a reliable self-driving system that uses only cameras, dispensing with radar and lidar.

Most autopilot systems use a mixture of cameras, radar and lidar to build as holistic and reliable a picture of the car’s surroundings as possible.

Tesla itself has traditionally used radar and cameras (not lidar). But last month Tesla confirmed it would begin overhauling the autopilot system in its Model 3 and Model Y cars immediately, ditching radar in favour of an entirely camera-based system.

Speaking at the 2021 Conference on Computer Vision and Pattern Recognition this week, Tesla’s senior director of artificial intelligence, Andrej Karpathy, said cameras were now surging ahead of radar, meaning the latter were beginning to act as a drag.

“Vision is getting to the point where the sensor is like 100x better than say radar,” he said. “If you have a sensor that is dominating the other sensor, and is so much better, then the other sensor is actually holding you back and is actually starting to contribute noise. So we are really doubling down on the vision only approach.

“The reason people are squeamish about using vision, and the challenge that almost always is pointed out, is that people are not certain if neural networks are able to do range finding or depth estimation of these objects.”

He said humans were able to use a purely visual system do depth estimation. The question was whether computers could use cameras to do the same. He said the answer was an “unequivocal yes”.

He said the key to developing a purely vision-based autopilot rested on advanced capabiltiies in three areas: depth, velocity and accelleration. To achieve this, he said a “large, clean and diverse data set” was needed.

This data, he said, could be collected by the large number Tesla’s on the road, and was then sent back to what he called an “insane supercomputer that we are building and using now”.

“For us, computer vision is the bread and butter of what we do, and enables the autopilot. And for that to work really well, you need a massive data set. We get that from the fleet. But you also need to train massive neural nets, and experiment a lot,” he said.

“This is a massive super computer. I actually believe in terms of FLOPS [floating point operations] this is roughly number 5 super computer in the world. So it’s actually a fairly significant computer.”

For readers with tech nous, Karpathy ran through some of the specifications, which included:

• 720 nodes of 8x A100 80 GB (5760 GPUs total)
• 1.8 EFLOPS (720 nodes * 312 TFLOPS-FP16-A100 * 8 GPU/nodes)
• 10 PB of “hot tier” NVME storage @ 1.6 TBps
• 640 Tbps of total switching capacity

Karpathy didn’t reveal much about the next phase in Tesla’s computing developemnt, Project Dojo, except to say: “Next up, we’re currently working on Project Dojo, which will take this to next level, but I’m not ready to reveal more details of that just yet.”

While Tesla’s ambition to roll-out full self-drive has had some setbacks – particularly in public opinion, which has tended to see accidents as evidence that computers can’t reliably drive cars – the Californian car maker continues to believe the future automobility is in self-driving cars. He referred to human brains as “meat computers”, and said they were far inferior to silicon computers.

“People are not very good at driving, they get into a lot of trouble and accidents, they don’t want to drive, and also in terms of economics, we are involving people in transportation, and of course we would like to automate transportation and reap the benefits of that,” he said.

“It should be possible to replace a meat computer with a silicon computer.”