Sounds like you need another larger fin for your handmade 80 ltr FSW.
You are going from a board that's really exceptional upwind (the Hypersonic) to
a soft railed board with a tiny fin that's not ever "good upwind" even on a good day.
Ellen is correct, freestylers rarely need to sail upwind, and when they do, they stand the board up on the upwind rail and go upwind on the shape in the bottom of the board.
Most wave boards and all smaller freestyle boards use quite small fins as that's the size
rigs (< 5.5 m2) they mostly use.
Light wind freestyle, with a 5.8 m2-6.5 m2 rig is going to take a larger board and a wider chord (front to back length) fin.
The reason so many freestylers use really small (short here) fins is that they are sailing in almost no water, and they need the board to be loose for their moves.
When doing spocks and other tail first moves, a larger fin really causes problems.
So, stand up straighter, keep your weight more over the boards centerline, and make your moves, then sail back upwind if necessary.
The pro guys in Bonaire all sail in knee deep water. It helps to be able to step off and then back on if you miss on a move. The people watching from shore hardly realize you've made a mistake.
A 5.8 m2 on an 80 liter board is a pretty unbalanced/mismatched combination, so don't beat yourself up too much.
Your 5.8 would be good on a 95-105 liter freestyle board.
Hope this helps,