Anyone conservative, whether secular or otherwise, is bound to oppress anything new and different. I don't think religions in themselves oppress asexuals, but religions very often, and very happily, oppress new and alien ideas. Not all, of course, which is why I'd say the biggest problem comes from right-wing Conservatives, and generally Conservative people.
Traditionally, people marry, have sex, and produce offspring. So asexuality is alien and therefore not traditional. Conservatives often don't much like it when people leave tradition behind, I have noticed.