“The Jaunt” Quest bugged out

So I started the quest up until the part we’re supposed to get our Horse. Instead I went away and started exploring to try and get better charisma-equipment to woo the stable master into giving me pebbles back.

At one point out near the cabbage thief area I got an alert it was too late to join them. I had no clue the quest was actually timed, nothing up to that point had been.

So I go back to the castle and try to talk to the two old guys I was supposed to Jaunt with and he begrudgingly explains what happened and all the fun things I missed.

I leave to ride back to the blacksmith, generally upset I missed a seemingly fun quest, the game freezes while I’m on my Horse. I reset to BEFORE talking to the old man to continue, but now he refuses to talk to me at all, and tells me to fuck off. I reload multiple times and no luck.

The save file before starting the quest takes me back 2.5 early game hours of exploring and skill crafting, so that’s also not an option… anyone run into this on PS5?