Glance at the next lane in traffic and it is always moving faster. This is a known illusion. You spend more time being overtaken than overtaking, so that is what the memory keeps, and it holds up remarkably well as a description of watching other people's careers.
Their lane also stops. You are simply not looking when it does. The only speed you have real information about is your own, and it happens to be the only one you can do anything about.