Unfortunately what was supposed to be a two minute job ended up taking over 2.5 hours which we had not scheduled maintenance for in our production environment. Our window was 30 minutes. This was disappointing, but the loadbalancer team did rally around the problem when it became apparent it was not as trivial as first imagined. However, the bug that was encountered came from a basic upgrade of the appliance which failed due to SSH Keys being deployed differently in the Operating System re-platform to a later version of CentOS. This should have been caught in testing as is the most basic upgrade test imaginable I would have thought. This too was very disappointing.