So a couple issues here.
#1 - I wouldn't invite him back. Homes games are a big social event for me and my players. If someone does this at a home game, the chances of him doing it again at the next home game is zero, because I'm not allowing them back.
#2 - Collecting keys is embarrassing. A home game isn't a high school or college party. There needs to be responsibility.
#3 - I would have not allowed him to leave in his own car, and if things got physical, so be it. Either some other players who know him have to drive his car home with him in the passenger seat, or the police need to be called. I have a feeling that most local PD's would find a way to get this guy home while someone drove his care home, and not jam him up. Letting him leave in his own car leads to...
#4 - This is a major legal issue, that if he kills or injures someone in a crash, and is drunk, the family of injured or killed could have a field day with you as a host. The lawsuits would come in fast and furious, and whether you could be held partial liable or not, you would have to go through the massive headaches of lawyers and tons of money to defend yourself.
I don't want to be that guy, but #4 is severe, and him driving home should have never happened, even if Jesus Christ wanted to follow him home.