Not everyone follows the rules I am about to discuss. I do, because personally they make sense to me. I am a conservative and a traditionalist. That said, traditions have a reason. I fully believe in breaking rules of style, but to break the rules effectively you have to first know the rules. Wrist watches are already quite traditional, and those who wear them tend to be more old school. So among watch wearers, particularly those who are aficionados, your adherence or non-adherence to these rules will be noticed. Let's go over the basics.
Firstly, this one is very simple. Match your watch band to your shoes (and belt if it is visible!). Wearing a black watch band with brown shoes creates a visual dissonance. Maybe that is what you are going for, but it certainly is a very casual approach. I wouldn't do this when going to a meeting of conservative businessmen, certainly not somewhere like New York. But a meeting in San Francisco of tech workers? They probably don't know the rules, don't care, and wouldn't notice.
As a corollary to this, try and match your watch dial to your watch band. They don't have to be the same color, but they should complement each other. You don't need to match your watch dial to your shoes or belt, but matching the dial to some aspect of your attire will lead to a much more finished look. A red dial with a green shirt probably won't look very good, unless it is Christmas time.
Next, match the metal in your watch to the metal in your belt buckle and/or other jewelry. Wearing a gold watch with a bright silver belt buckle will look like you are trying too hard. It will appear ostentatious and again, create undesired dissonance. A lot of watches have both gold and silver/stainless steel so you won't have to think about this.
A stainless band and stainless watch is easy to match to most of your wardrobe, but keep in mind that generally metal bands are considered casual. James Bond's look of wearing a stainless Omega dive watch with a tuxedo is a huge no-no. In general, a big faced or thick watch is not dressy. If you are dressing formally, your watch should slip under your shirt cuff with ease, not get snagged on it.
Dive watches are casual, not dress. You could wear a dive/sport watch with ALMOST anything, just not more formal attire. It might work with a brown sports jacket when dressing business casual, but keep in mind it will dress down the whole look. Maybe this is a good way to make a more formal look casual, but be careful.
Similarly, you wouldn't wear a dress watch with sports attire. Firstly, leather bands and sweat do not mix well. Your band may be "waterproof", but it WILL collect sweat and begin to smell, sometimes quite badly. Not to mention it just doesn't fit.
What makes a dress watch? Well, opinions vary on the exact details, but in general a dress watch is thin, on a leather band, and has few (if any) complications such as the date. However, in modern society complications on a dress watch have become more popular. Again, keep in mind the company you keep and the image you wish to portray. If you are a youthful businessman in a meeting with older, conservative colleagues, you probably want to adhere closer to the rules.
Personally, I have a variety of watches (and different bands for each watch) that allows me to always have something to match to my clothes. You could get away with just two watches if you purchase extra bands. One thin, relatively simple dress watch, with at least 1 black and 1 brown band will do. For everyday/casual use, one watch with metal bracelet, most people don't wear much gold so you could get away with all stainless. But if you have some gold belt buckles or shoes with gold hardware then you may want to get a watch with both stainless and gold. You might want to also have a couple different bands for your casual watch, but with a metal bracelet it isn't really necessary.
To break the rules effectively, you have to understand why you are breaking the rule. As in my previous example, maybe you are dressed business casual but want to tone the look down. Wearing a stainless dive or chronograph watch will do the trick. Perhaps you are dressed relatively casually, with a button down and jeans, but want to elevate your look? Well then wearing a dress watch on a leather band (matched to your shoes) will give your look that extra "oomph".
Do you have any more rules or guidelines you follow? Do you even give a shit about any of this, or is it just snobbery to you?