The grey push button on my Baxi Bermuda 401 VP will not remain on?

I have a Baxi Bermuda 401 VP and the grey gas push button will not remain on. After keeping this pressed and then using the ignition button to light the pilot the grey push button pops out after about 1 minute.
I can't get it to stay pressed, any suggestion please?