Why do people think that people who dont have a religious affiliation are "bad" human beings.?

Some honestly can't imagine having any morals without having a book to tell them what they are. I know, it sounds ridiculous, but they really can't imagine it.