I'm 48. I did not start feeling the effects of allergies until about ten years ago.
Some of your symptoms might be from the Benadryl rather than your allergies. If you don't want to see a doctor, then give some of the newer OTC antihistamines a try. The loratadine works all right for me, and it's available for dirt cheap at Costco.
Here's a list of what's now available OTC:
http://reference.medscape.com/drugs/antihistamines-2nd-generation