Use a different design of gamepass door. The one you are using is a teleport gamepass door, pretty uncommon but good to stop people following people in.
It teleports people to the nearest "possible" y coordinate to -2 studs or +2 studs in the direction by default.
I would personally suggest either using a normal game pass door or finding a gamepass teleporter as those do the same thing and won't have the problem you describe.
LordHenryVonHenry |