…unless you are not subject to mandatory insurance in the new EU country of residence (which is and needs to be determined according to that country’s social security laws after you‘ve been living there for a couple of weeks/months).
In the case of Portugal, for instance, if you’re not actually employed there, chances are rather high you won‘t be subject to the insurance requirement and will thus become eligible for a full withdrawal of Swiss pension benefits (as opposed to France).