I have sailed my 320 with a smaller fin to sail in shallow water. If you are sailing in light winds (subplanning), I dont find that it makes any performance difference. the board is noticeably easier to tack.
There will be a difference in planing conditions particularly sailing upwind. You certainly won't do any harm so just do it and see how you like it. there should be plenty of second hand small tutle fins on the market if you don't already have one.