There is a clear ground truth and therefore a rather clear definition of “error”. For example, if there is a car in front of the AV and the Perception System fails to recognize it, it is a clear fact that the Perception System failed. Two humans will provide the same answer when they will be asked whether there is a car in front of the AV or not. Therefore, we can easily measure the performance of the Perception System by calculating how often it makes an error.

Driving Policy depends on “what would happen next” reasoning, which is not factual. Two humans might provide two different answers when asked whether the AV should yield to a car at an intersection or take the right of way. One will say that he thinks it is safer to yield and the other will say he thinks it will be safer to take the right of way so as not to block traffic. So, there is no clear definition of “error”, but rather it is open to interpretation or judgement of the situation. Furthermore, this judgement often in retrospect, after an accident occurs, and with the advantage of knowing the future (that is, with the knowledge of “what happened next” and without the need to guess “what would happen next”).

For the sensing, there is a clear monotonic relation between “safer AV” and “better AV”. That is, the fewer Perception mistakes the system will make, the “safer” and “better” the AV will be.

For Driving Policy, being “safer” does not always mean being “better”. To make this point clear, consider for example a residential road, with parked vehicles on both sides. Since a pedestrian might run into the road, the “safest” AV system would drive very, very slowly, making sure that even if a kid with an electric bike driving extremely fast suddenly entered the road, the AV would manage to brake in time. However, such extremely slow driving will block traffic. In fact, no human drives so slowly in such a case. The reason is that as a society, we balance safety and efficiency, by determining what the “reasonable risk” we are willing to take is.