You're gonna have a hard time keeping "trash fish" out of your pond if the water source that feeds it is coming out of another pond as you described... Particularly if there is high water flows during rain events.

You may just have to learn to work around the added fish species. It is not impossible to overcome the addition of certain "trash fish" in a LMB pond. You just have to keep that in mind. Have you done a fish inventory of your pond to see what species you have and what their abundance and size structures are? That maybe a good starting point.

Once you know what fish you are gonna be dealing with, you can work from there. You can still have a great LMB pond with the fish you talk about being present, you just have to manage it differently...

I am sure the experts will chime in soon enough! Another option maybe to find the owner of the rogue pond and see if they would be interesting in starting over with their pond and killing off all the "trash fish". Then you can both start new and you will not have to worry about "trash fish" from his pond washing into yours...