Troubleshooting 404 Not Found Error – Tyk

A 404 error, also known as a "Not Found" error, is an HTTP status code that indicates that the requested resource cannot be found on the server. In the context of API gateways, 404 errors typically occur when the gateway cannot locate the specified API or API endpoint.

While Tyk is a robust and reliable platform, you may inevitably encounter some 404 issues from time to time. This guide will help you troubleshoot 404 issues quickly and efficiently.

Note: Refer to self managed setup for hybrid cloud deployments
Since major parts of the troubleshooting take place on the gateway, a self managed 
setup, in this case, could also mean that you have deployed the control plane 
(comprised of the dashboard, MDCB and management gateway) in Tyk cloud but are using
your own gateways deployed in an environment of your choice (on-prem, Alibaba, Azure,
AWS, Cloudflare, Digital Ocean, GCP etc.)

Self Managed Setup Checklist

Verify Gateway Endpoint
Check Gateway-Dashboard/RPC Connection
Review Gateway Configuration
Inspect Loaded APIs
Enable 404 Logging
Analyse Analytics Records
Audit API Definition
Validate Upstream
Seek Support

Cloud Setup Checklist

Verify Ingress Endpoint
Check for Cloud Outage
Lookup 404s in the Log Browser
Review Plan and Quota
Audit API Definition and Verify Segment Tags
Contact Support

Self Managed Setup

Tyk's Self-Managed setup involves deploying and managing the gateway along with other Tyk components on your infrastructure e.g. Bare Metal, AWS AMI, Docker, Kubernetes, etc. If you're facing 404-related issues in a Self-Managed setup, you can follow these troubleshooting steps:

Self Managed Troubleshooting Steps

1. Verify Gateway Endpoint

Verify you are indeed calling the right endpoint on the gateway. Check the host and the port values match your gateway’s hostname/listen_address and listen_portfields

2. Check Gateway-Dashboard/RPC Connection

Probe your gateway and dashboard/rpc connection. Explore whether the connection is intact by testing the status via our liveliness health check endpoint. You can also dissect the gateway and dashboard interconnected config fields e.g. db_app_conf_options and tyk_api_config. Skip this for RPC connections.

3. Review Gateway Configuration

Review your gateway configuration and validate your environment variables are not overriding the config file. If your gateway is sharded, ensure the right segment tags are added.

4. Inspect Loaded APIs

Inspect the list of APIs loaded on the gateway and confirm the desired API definition is registered on the gateway. Use the gateway REST API (tyk/apis) to reveal the API definitions loaded on the gateway. If in RPC mode, peruse the gateway logs for slugbinding warnings, listen_path collision errors, or duplicate API debug messages on gateway startup. You can emit logs in debug mode for more verbose logging.

time="Dec 02 10:38:30" level=info msg="Detected 41 APIs" prefix=main
time="Dec 02 10:38:30" level=info msg="Loading API configurations." prefix=main
time="Dec 02 10:38:30" level=info msg="Tracking hostname" api_name="REST Http Bin Api" domain="(no host)" prefix=main
time="Dec 02 10:38:30" level=info msg="Tracking hostname" api_name="Sample Basic Authentication" domain="(no host)" prefix=main
... # Loading API definitions
time="Dec 02 10:38:30" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
time="Dec 02 10:38:30" level=debug msg="Loaded API Endpoints" prefix=main
... # Duplicate errors
time="Dec 02 10:38:31" level=error msg="Listen path collision, changed to /duplicate_api_definition/-f7fa46a3c20b412c54e0a63d6c4fef12" api_id=f7fa46a3c20b412c54e0a63d6c4fef12 api_name=TestAPI org_id=653646c1d13bcc0001a557ff
... # Initialising API definitions
time="Dec 02 10:38:31" level=info msg="API Loaded" api_id=2027fb466dcc47854de54889da98a874 api_name="REST Http Bin Api" org_id=653646c1d13bcc0001a557ff prefix=gateway server_name=-- user_id=-- user_ip=--
time="Dec 02 10:38:31" level=debug msg="Initializing API" api_id=2027fb466dcc47854de54889da98a874 api_name="REST Http Bin Api" org_id=653646c1d13bcc0001a557ff
time="Dec 02 10:38:31" level=debug msg="Setting Listen Path: /http_bin_api/" api_id=2027fb466dcc47854de54889da98a874 api_name="REST Http Bin Api" org_id=653646c1d13bcc0001a557ff

Refer to our guide on making API routes unique if duplicate listen paths are discovered.

Additionally, v5.3.1 and v5.0.12 introduced a security fix where APIs won't register on the gateway if the custom plugin bundle associated with it fails verification

Fixed a bug where Tyk failed to properly reject custom plugin bundles with 
signature verification failures, allowing APIs to load without necessary plugins, 
potentially exposing upstream services. With the fix, if the plugin bundle fails to 
load (for example, due to failed signature verification) the API will not be loaded 
and an error will be logged in the Gateway

5. Enable 404 Logging

Enable track_404_logs on the gateway and examine the gateway logs for the proxied resource path. Isolate and troubleshoot, if the resource path contains a regex, middleware, or plugin.

time="Dec 28 15:10:44" level=error msg="Not Found" origin="xxx.xx.xx.x:xxx" request=
"POST /b8b49debcf89425d4bc4f450541854e3/tyk/oauth/authorize-client/ HTTP/1.1"

6. Analyse Analytics Records

Examine the latency field of the resulting analytics record. If using the dashboard, look up the logs in the Log Browser and analyse the response details. If latency/RequestTime is greater than zero, Tyk successfully proxied the request upstream. Which means the 404 response is coming from your backend.

7. Audit API definition

Audit your API definition configuration:

Inspect the port, protocol, domain, version, and internal state of the API definition.
Investigate the proxy settings fields. Confirm the listen_path is correct. Ensure preserve_host_headeris not needed for outbound requests. Check if strip_listen_path is desired. Also, test if disable_strip_slash does not interrupt the final resource path.

8. Validate Upstream

Validate your upstream is working as expected. Inject a dummy upstream with lean configurations into a test API definition to verify there are no issues. Or bypass the Tyk API gateway and hit your backend directly. Check that the issue may not be coming from a proxy between Tyk and your upstream e.g. load balancer or firewall

9. Seek support

Reach out to Tyk support and share the:

response result,
the request details,
any accompanying gateway logs in debug mode
and your API definition.

Cloud Setup

Tyk Cloud setup involves deploying the entire API management stack (gateway, dashboard, developer portal, and MDCB) in a managed cloud environment. For cloud-based issues, follow these steps:

Cloud Setup Troubleshooting Steps

1. Verify Ingress Endpoint

Verify you are indeed calling the edge gateway/ingress and not the management (control plane/dashboard) endpoint. Check the host and the port values.

2. Check for cloud outage

Confirm there isn't a Tyk cloud outage. Simply calling the liveliness health check endpoint should reveal the result. Do not use the gateway's root endpoint e.g. https://<component-name>-gw.aws-<region>.cloud-ara.tyk.io/ to check for liveness probes as it would return 404 unless overridden.

3. Lookup 404s in the Log Browser

Lookup the analytics record from the dashboard's Log Browser view for any records with status code:404. Refer to the analyse analytics records section of the self-managed troubleshooting guide if you spot an instance of a 404 record.

4. Review plan and quota

Review your plan and verify you are not exceeding your quota. Also, inspect that detailed logging isn’t enabled on all the APIs.

Note 
It only takes one API with detailed to swamp a gateway and cause it to go Out of 
Memory (OOM) if that API logs a great deal of information. This would then lead to
nginx giving a 404 because the gateway would be unreachable for a short time

5. Audit API definition and verify segment tags