What Bill and Ellen suggest will work nicely.
Only thing I would sugest isthat you get a "pin drive punch" just slightly smaller than the outside diameter of the brass barrel nut.
With the pin drive punch, the punch will stay nearly centered in the hole and do a much nicer job of "driving" the old nut out and the new nut in.
Just be sure when you install the new barrel nut that you get it started nice and straight. If you get it started in the hole at an angle or cockeyed, it will damage the hole.
A good trick to use here is the apply some soap in the hole and a little soap on the outside of the barrel nut as a "lubricant" to allow the brass barrel nut to slide in more easily.
Be sure to align the threads to the top of the fin and the fin screw hole in the board's deck before you drive the barrel nut in.
Best way to do this is to mark the screwdriver slot before you drive the old nut out.
Hope this helps,