Very interesting video presentation by nVidia as to what inputs a self driving car actually is getting from cameras, Lidar, radar and GPS etc.
Last edited by a moderator:
You can install our site as a web app on your iOS device by utilizing the Add to Home Screen feature in Safari. Please see this thread for more details on this.
Note: This feature may not be available in some browsers.
Also interesting is the implied dig at Tesla by showing how inaccurate GPS is, since Elon has said that Tesla uses GPS. Actually, I've always been wondering about that. GPS is indeed inaccurate. I can see how Tesla can create a high resolution map by averaging million of trip miles, but how do you use such a map given that an individual car's GPS is inaccurate?
Fair enough. I wonder whether a well-functioning autonomous can - eventually - perform better than the best human driver does under the truly wretched conditions one can find in a normal Alaskan January: whip-blowing snow with visibility from 2 to 20 feet; determination of road surface based on combinations of feeling outer edge of pavement/gravel interface; the tracks - incuse and obtuse - of prior vehicles; extrapolation of road ahead based upon remembrance of the most recent oncoming vehicle's headlights; recollection that after that latest dip the road curves to the right for 30 yards, then bumps to the left....
....at the very least, those systems won't get as fatigued as any human does after some number of hours of such driving.
The thing I wish that video showed was what the computer really sees, which is a series of matrices which include pixel colors, intensities, etc. for the entire image. When learning about image recognition, that was a big "aha" moment for me. Show a picture of a dog, then flip to what it looks like as a numeric matrix of RGB values. Then it's not so easy to decipher.
Based off of the Mobileye talk, using the landmarks that will still be recognizable even when snowing(signs, poles, etc) the car should know exactly where on the road it is, even if it can't see the road at all. Based on that information the car should be able to keep you exactly in your lane +- 3.5cm.
A true visual neural net does not have a pixel per pixel visual field layer (at least beyond the very first photon camera sensor input layer). The input is immediately convoluted into detectors for edges, contrast, lines, and primitive shapes, and then gets more and more abstract as the layers go up. So there is no great way of seeing what the computer sees, other than at the final semantic layer, which is described, but not shown in the latest Mobileye talk (this is the front of a car, back of a car, sign type xyz, person, curb, roadway, etc.)
The human visual cortex is the same way. You might think there is a low level brain layer that is a pixel for pixel representation of what the eye picks up, but there isn't really, other than the rods and cones themselves that are light activated.
Btw, don't forget that Mobileye technologies rely on grayscale cameras only, no color. I don't know if they ever plan on using color information...