Is cl40 4800 mhz ram that bad?

I bought ram for my new pc that I built a few months ago, back then i relatively naive to all the technical stuff for pc building so I didn’t care too much about ram specs until I found out that it mattered quite a bit a few months later, is it worth it to upgrade to cl30 6000 mhz or should I stick with my current ram?