GRID unit frames don't always appear instantly

[Axyrian]

So I'm invasion hopping like everyone else seems to be doing so I've been in and out of parties for the past 8 hours or so. Sometimes when I join a new party, the unit frames will be there straight away while sometimes it can take up to 5 minutes. This is the case about 80% of the time. Is this a visibility problem?
I got this UI maybe a week ago and I haven't updated anything as of yet as I've had issues before with updating messing up my whole UI.

Any help is appreciated

Edit: This is also while in combat.