The thing you will want to look at are the fees charged for early withdrawal due to emigration. For example, the Sparkasse Schwyz charges CHF 25 if you withdraw within 1 year of opening the vested benefits account, and nothing after that. The Schwyzer Kantonalbank charges CHF 800, and Liberty charges between CHF 475 and 950. You can find all fees on the pruduct info pages in this comparison:
https://www.moneyland.ch/en/vested-benefits-accounts-comparison
In any case, Schwyz has the lowest capital withdrawal taxes. If you withdraw after deregistering in Switzerland, capital withdrawal taxes are charged in the vested benefits foundation’s canton of domicile. This is particularly important if you will not be eligible to claim a capital withdrawal tax refund (I’m not sure whether non domiciled UK residents benefit from the double-taxation treaty).
If you withdraw while still resident in Switzerland, then you pay tax where you live. If you are willing and able to move to another part of Switzerland for a while before you withdraw, then Appenzell Innerrhoden is where you will want to be.