I have this same conversation with my sister and I always try to explain that it doesn't "bug" me or surprise me when this subject is brought up in casual conversation, but it really just kind of rubs me the wrong way. I think
Pansy explains my point nicely.
