Does adding an address to a single Yat add that address to all of my Yats?

No, adding addresses is done on a per Yat basis; this is to ensure Yat Creators have full agency over which addresses are associated with which Yats.