Yes, the whole virtualization of the scene and all objects moving and not moving will have to mapped in real time, and then programmed to move the car strategically in real time. Tesla is in the beginnings of doing that, but actually being able to navigate basic streets safely sounds so easy...