Inside the seaside town where thousands face paying hundreds more in income tax