Yes what you said is true, except for that important assumption you are making about immigration. I'm sure with enough terrorist attacks even that can change. That's not quite right. They would not be punishing the UK, merely allowing them to return to a somewhat pre-EU state; if that's a threat then it's very much based on the reality that there was a rationale for the existence of the EU. I don't see why reminding other members of the situation without the union would be "keeping them together with fear" any more than it would be keeping them together by reminding them of the benefits.