Amazon API Gateway supports certificate-based mutual Transport Layer Security (TLS) authentication . Mutual TLS authentication requires two-way authentication between the client and the server. When using Custom Domain Names for API Gateway and enabling mutual TLS on custom domain name, There is a restriction as mentioned in AWS Documentation
To enable mutual TLS, your domain name must use a publicly trusted server certificate issued by AWS Certificate Manager.
To resolve this error, Create a publicly trusted server certificate issued by AWS Certificate Manager.
Follow this Blog for instructions on securing your API with mutual TLS.