Nope, nope, nope.
And I think most SWs wouldn't want to have clients in the same building (I'm not one so not speaking for them).
1) The number of times I hear from SWs that client 'B' started to get infatuated and started to say things, then find out personal information, then started to hang around near wherever they provide services from, and then moved on to stalking. This is a common story.
2) Linked to the above, infatuated clients trying to impede other clients going in to have sex with 'their girl'.
3) Separation between personal and professional lives. If you live in the same building it is highly likely that you either already know their apartment number or will find it out, and then number 1 becomes a potential risk. Also that would make it much easier to find out their government name by asking neighbors etc, which opens the door to extortion, stalking or exposing them to people in their lives that do not know what they do for a living and don't want them to.
And that's just the potential problems for the SW that spring immediately to mind.
But ok, you might not give a toss about their concerns, which wouldn't be a great attitude, but it is a pretty common one.
So here's a concern for you the client.
How often does one of the blokes here say SW 'X' was a bit crazy, or gave off a disturbing vibe, or similar.
Assuming that a small percentage of those stories are actually true, and they might be, why would you want to risk your paid for activities being exposed to people coming out of your apartment by someone who thinks that they can get something out of you.
Being exposed as a person that pays for sex could have very real personal and professional repercussions for you.
For the record, I think this is a highly unlikely scenario to ever actually take place as every WG I've encountered is way more mentally stable than people I encounter on the streets, but was placed here for illustrative purposes of thinking of your own personal security, even if you aren't thinking about theirs.
I have a really simple rule.
Don't eat where you shit.
Maybe just smile and say hello to a person that you know is or think might be a SW who lives in the same building as you, then go punt elsewhere with someone who you are unlikely to encounter in the wild on the regular.
Much better solution for everyone involved.