Wall sweep in group?

[CADMama]

Is there any way to restrict a wall sweep in a group?
In the attached example I do NOT want the brick sweep to go beyond the windows but it does - on one side only.
Any ideas?
CADMama