It’s not a "joke"

I went out to dinner with my dad (14M) and after we ate dinner left we were talking about being full from the meal and he said I’m fat to my baby brother I know this may not seem significant but I’m actively tryna lose weight I’m hitting the gym daily and this hurt so bad and I’ve been made fun of a fare amount about this before but never from my family it was supposed a joke but to me I felt betrayed and distraught.