Silver errs?

It's dangerous to suggest Nate Silver erred, as he rarely does, but surely in this discussion the relevant population is not all tax payers but the ones Obama mentioned by name?