Add it once at the bottom and hope it survives, or apply it inside every layer where the score is computed.