Driving is a right, not a privilege and I'm not gonna leave it to the government to decide what rights I have and I don't. We are taxed in too many innumerable ways for the damn roads to buy into the government's little myth. You may just as well apply your licensing rationale to the Second Amendment, if that was the case, for no other reason than "A "right" does not need to be enumerated in the Constitution to be a "right", as you wrote.
There are two definitions of a "Right" - the liberal one, in which a commodity is provided to everyone at the expense of others, and the conservative one, which is usually a cost-free declaration that you may pursue a particular course of action without interference and at your own expense.
The typical conservative/libertarian view is that govt can only prohibit and punish actions that harm others -a concept known as "negative law" . The opposite end of the spectrum is "Positive law", in which the govt laws mandate certain behavior - things you <must> do to produce desired "beneficial" societal outcomes- Robert's decision handed that power to the liberals in spades.
In the conservative view You don't have the right to use your liberty in such a way that it endangers the "Life, Liberty, and Pursuit of Happiness" of others. You have a right to own a gun. You don't have a right to use one recklessly, and we (in theory) deprive people of that right when they show they will abuse it (say a felon) , or if we feel that they are (more) likely to abuse it (insane).
Libertarians tend to prefer laws that punish only real, identifiable harm. Hence they fell drug use should be legal - even if it increases the possibility that an addicted drug user will commit crimes to feed their habit. They feel we must wait till there is an identifiable victim before the govt can step in. However, that means someone may have died during a drug related robbery attempt, so most conservatives favor a "middle ground" further over on the spectrum toward "positive law" - where the probability of bad results is in included in the calculations of if an activity is considered illegal. Its an attempt to regulate behavior deemed likely to result in harm to others - "legal endangerment" is the embodiment of this concept . Consequently, your freedom of speech doesn't include a right to yell "fire" in a crowded theater, your 2nd amendment right doesn't allow you to fire your gun in the air in a crowded city ( or at all in some places) , and your right to "free movement" doesn't allow a blind person to get behind the wheel of a car.
The potential for abuse is there because you are relying on someone's vision and determination of what is "likely" to cause harm- just as positive law relies on someone's view of "what is best for society" A driver's license is a way to determine who is "likely" to cause harm - the blind, known drunks with DUIs, etc. Likewise, the requirement for liability insurance is to protect others in the case that a person who WAS issued a license has a bad day - a second order protection of the rights of others.
Just because something is a "Right", doesn't mean it can't be restricted to protect the rights of others (especially at the State or local level) , and that is all the licensing system is supposed to be doing. Of course, as Radioman pointed out, at other times we (being our govt and system for determining such things) decided the potential harm did not warrant a licensing system - partly because Horse drawn speeds were more limited, roads were poor and traffic was less.
This difference is really the main dispute between conservatism and libertarianism - and why I typically identify as Conservative and not libertarian. I do believe that the govt should engage in restricting behavior likely to cause harm, and accept the tradeoff that in certain circumstances a right may be restricted unfairly and for no reason.