Too big and board will want to twist up to windward.Hence wide boards holding down bigger fin.May also cause tail to lift and control problems.
Too small creates later planing (more side lip initially) and poorer upwind performance (esp at lower speeds)
On occasions there are positives to using "wrong" fin. Setting absolute Vmax (IMO) you will put up with negatives to get better offwind speed and enhanced control a smaller fin (can give)
On the hand if you want to be real quick off block you could use "too" large a fin. (Helps initial planing) and work around problems you then encounter.
If you encounter control problems and or overpowered change fin first.(to a smaller one)Its sometimes enough without changing sail.
Its trial and error.
Good place to start though is
Multiply sail size(in metres) by 5 and then add 2 (or 3 ) (Eg for a 6 metre .5x6 =30 then add 2 giving 32cm)
Obviously there are large /thick/thin 32cm fins but on average its a good guide and works over a surprising range.I like to use same range of fins to overide design/thickness differences) (Eg My Fanatic 36 in performance is almost exactly same as my Northshore 32)
Try and build up a quiver of second hand fins.(Ebay is great for it) Then when you know exactly which you want buy them new.(Resell old ones back on Ebay)
It really is worth messing about with fins and changing them.(Its as important as sail size)