You don't say which way they go alright through the turnout, but I suspect that going from the frog ( heart-shaped piece in the center) to the points (the moving part) they go OK. Check for irregularities on track. Couple 2 cars ( no locomotive) and push the rear one by hand slowly. Check the front wheels on the front one to see where it goes out of the track. Most likely it catches the points and jumps the rails. Post your findings and me or somebody else will get a solution.

Most likely the track is out of gauge as some parts of the turnout, and/or the point rails do not make a smooth fit with the stock rails. This is easily checked by visual observation and with an NMRA track gauge. Often out of gauge rail can be rectified with some gentle "bending" or filing of the points. Once the entire turnout is in gauge, check the wheel sets on your rolling stock and engines, make any adjustments needed, and then test the rolling stock on the turnouts.

If you are using EZ-Track turnouts check the point rail at the pivot point. There may be mis-alignment there.