This is the sort of thing that ultimately drove me to Protestantism, which I say with complete affection for my many Catholic friends and family members.
I understand the doctrine that the pope only claims infallibility when he speaks ex cathedra, and he doesn't claim to be doing so here. But you can't invest that type of authority in an individual on certain matters, and allow him to be as sloppy as he is here on others: