Stop looking at me!

Do you ever get the feeling like the person you're talking to is giving you a once over? You try to make eye contact with them but you can see them looking out of the corner of their eye at your hand, your arm, your breasts (happens all the time and I'm not even big in the chest area), your clothes... This drives me insane and makes me very self-conscious. Why is it so hard to look people in the eye when you talk? And do you have to make it so obvious that you're looking at them? Sure I look at people when I talk to them but I do it in a way so that they will not think I'm staring at them. Look....at the door...at...out the window...what...at my shoe...she's...door again...wearing!
This way I don't make them feel uncomfortable, you know when someone is staring at your clothes to the point where you think you might have spilled something on yourself, or your skirt is tucked into your pantyhose, or your fly is open or something...