It's pretty impossible to give suggestions without a clip.I'm still having a hard time setting up a good Death Metal tone on the FM3 with my Vader Cab. I'm using a Ritter gba280 watt power amp and using a Caparison Horus with a Black Winter pickup. I am creating a preset at loud stage volume but no matter what I do I can't get rid of this sharp, harsh high end. I am currently using the Angle Severe 2. I keep separate presets for recordings and I am very happy with those, but once I create a preset for my cab I can't get anything to sound good.
It should sound good with not much more than having cab sims disabled (or just no cab block in the preset) and using an amp model you like. Not every amp model is going to play nicely for your preference so just roll through them until you find something you like. Try adding a parametric EQ before the Out block to try to find where the offending frequencies are happening.