Has Phil ever almost quit the show?

I’m a bigger survivor fan and it’s common knowledge that Jeff really wanted to quit hosting around season 17. They eventually let him become an executive producer and gave him a short-lived talk show to appease him. Looking at the Wikipedia, Phil still isn’t listed as an executive producer. I know he created Tough as Nails, but are there any rumors if he ever considered not coming back? I know Phil was a finalist when choosing the host of Survivor back in the day so it’s interesting to consider how different things could be if the shows they host had switched