AWS::ApiGateway::GatewayResponse Invalid API identifier specified when deploying authorizer

Trying to add custom responses to gateway authorizer. The authorizer is deploying fine but the custom response is not attaching to the correct API. I keep getting an error and hours of googling has turned up nothing.

Error:

 An error occurred: GatewayResponseInvalidApiKey - Resource handler returned message: "Invalid API identifier specified 892584095726:zu8pz12we8  (Service: ApiGateway, Status Code: 404, Request ID: 826a0602-99be-40a4-9eeb-9ec4b75cdb91, Extended Request ID: null)" (RequestToken: 14cd04c7-2443-1705-5b06-bcf03ca675e5, HandlerErrorCode: NotFound).

Those ID above I changed for the example above. Notice that cloud formation (or serverless) is stuffing a random 892584095726 and a colon : before the api key.

I’ve double checked in the aws console and the zu8pz12we8 is correct for the api key (again not the real number I changed it for this example)

Code:

GatewayResponseInvalidApiKey:
      Type: 'AWS::ApiGateway::GatewayResponse'
      Properties:
        RestApiId:
          Ref: HttpApi
        ResponseType: ACCESS_DENIED
        ResponseTemplates:
          #https://stackoverflow.com/a/49806967/10212963
          application/json: "{ \"message\": \"$context.authorizer.customErrorMessage\" }"
        StatusCode: 403

I’ve also tried referencing the CF output directly as follows with the same result:

RestApiId: "${cf:${self:service}-${self:custom.stage}.HttpApiId}"