Holding down the swing button will make you swing faster.
You won't stick to a wall you swing into if you turn away from the wall.
The faster you swing the less control you have.