Why do people that the number of members is an indication of the validity of their religion?

Jesus Christ Himself said that the road to heaven is narrow. Wouldn't the largest religion therefore not be correct? I'm Catholic, the Vatican is guilty of pointing to numbers as well, so I'm not picking on anyone. Just an observation. Or could it be.....I'll use my own religion as the example. Say Catholicism is correct, there are a great number of people that identify as Catholic yet practicing is a different story, and those that pray the rosary everyday are even fewer. Maybe that's where the narrow comes in? What are your thoughts?