is there any way to avoid this?

http://www.jeromeabel.net/files/code/pd/pd-works/dynamic-patching/reviews/DynamicPatching_review-3.txt

@seed I use dynamic patching a lot...... but.......
I build all the patches once at the start of the show (ram use is laughably small).
I only connect/disconnect (re-route) [inlet] and [outlet] whilst outputs are muted
I soft switch inside patches with line~ where necessary
I always keep windows that I don't need to see minimised
I have everything osc controlled so that there is no hid activity and minimal screen drawing.......
David.