Router Customization and Debugging (Traefik)¶

DDEV uses the well-known open-source Traefik for its router container.

DDEV’s router plays an important role in its container architecture, receiving HTTP and HTTPS traffic for requests like *.ddev.site and delivering them to the appropriate project’s web container.

Traefik Configuration¶

Before continuing, it’s important to note that very few users ever experiment with custom Traefik configuration. This is an advanced topic, and the vast majority of users never need to know anything about it, as DDEV generates all the necessary configuration. In general you don’t need to understand Traefik configuration.

However, you can fully customize the router’s Traefik configuration. (DDEV uses the Traefik v3 rule syntax.)

All Traefik configuration uses the file provider, not the Docker provider. Even though the Traefik daemon itself is running inside the ddev-router container, it uses mounted files for configuration, rather than listening to the Docker socket.

Tip

Like other DDEV configuration, any file with #ddev-generated will be overwritten unless you choose to “take over” it yourself. You can do this by removing the #ddev-generated line. DDEV will stop making changes to that file, and you’ll be responsible for updating it.

Traefik Static Configuration¶

Static configuration is automatically generated in the $HOME/.ddev/traefik directory (see global configuration directory). “Static” configuration means Traefik configuration which is only read when the router is started.

.static_config.yaml (a hidden file) is the configuration that gets used. It is not to be edited; it is generated from DDEV’s base configuration while merging any files named static_config.*.yaml. It is read on router startup, and does not change until the router starts again (normally after ddev poweroff).
Additional static configuration may be added by adding static_config.*.yaml files, which will be merged into the generated .static_config.yaml. For example, a static_config.loglevel.yaml might override logging configuration, a static_config.plugin.yaml might contain external Traefik plugins, or a static_config.dnschallenge.yaml might provide configuration for additional certificatesResolvers. Merging is done with an override strategy, meaning that the final file in alphanumeric sort to touch a particular element of the YAML structure wins. Some examples of static_config.*.yaml files are:
- static_config.loglevel.yaml:
```
# Enable extensive error and access logging
log:
  level: DEBUG
accessLog:
  filters:
    statusCodes: {}
```
- static_config.cloudflare.yaml:
```
certificatesResolvers:
  acme-dnsChallenge:
    acme:
      email: admin@example.com
      dnsChallenge:
        provider: cloudflare
```
- static_config.fail2ban.yaml
```
experimental:
  plugins:
    fail2ban:
      moduleName: "github.com/tomMoulard/fail2ban"
      version: "v0.8.1"
```
certs/default_cert.* files are the default DDEV-generated certificates, normally created by mkcert.
config/default_config.yaml contains global dynamic configuration, including pointers to the default certificates. It is possible to add other Traefik configuration in the config directory, which will apply to all projects. For example, a config/router_middlewares.yaml file might provide middleware implementations that would apply to all projects.

Project Traefik Configuration¶

Project-specific configuration is automatically generated in the project’s .ddev/traefik/config directory. For example, a project named example will have a .ddev/traefik/config/example.yaml which describes the routers, middlewares, and services generated by default for that project. These are based on the base hostname, additional_hostnames, and additional_fqdns defined for the project. They also include support for add-ons and services that use HTTP_EXPOSE, HTTPS_EXPOSE, and VIRTUAL_HOST configurations (see Conventions for Defining Additional Services for more details).

The certs directory contains the <projectname>.crt and <projectname>.key certificate generated for the project.
The config/<projectname>.yaml file contains the configuration for the project, including information about routers, services, and certificates.
- If you alter a config/<projectname>.yaml file, you need to remove the #ddev-generated from the top of the file, so it won’t be overwritten on the next ddev start.
- It is possible to add additional files in config that you own and manage that add additional facilities. For example, a config/example_middlewares.yaml would be read by the router and used. It’s important that the filename be keyed by the project name, and usually its contents need to be unique to the project. For example, if it defines middleware implementations, they should usually be limited to the project. For example, an extra file might be:
```
http:
  middlewares:
    example-redirectHttps:
      redirectScheme:
        scheme: https
        permanent: true
```

Router `docker-compose` Customization¶

The default Docker Compose configuration for the router container is found in $HOME/.ddev/.router-compose.yaml (see global configuration directory). It is quite unusual to override this configuration, but it can be overridden in the same way project configuration can be overridden (project .ddev/docker-compose.*.yaml). These ultimately get merged into $HOME/.ddev/.router-compose-full.yaml

You can create a $HOME/.ddev/router-compose.*.yaml. For example, as $HOME/.ddev/router-compose.cloudflare.yaml might contain environment variables like this:

services:
  ddev-router:
    environment:
      - CLOUDFLARE_EMAIL=you@example.com
      - CLOUDFLARE_API_KEY=some-key

Troubleshooting Traefik Routing¶

Traefik provides a dynamic description of its configuration you can visit at http://localhost:10999. When things seem to be going wrong, run ddev poweroff and then start your project again by running ddev start. Examine the router’s logs to see what the Traefik daemon is doing (or failing at) by running docker logs ddev-router or docker logs -f ddev-router. The Traefik logs are set to a minimal set by default, but you can enable much more extensive logging and access logs with a static_config.loglevel.yaml as described above.

“Warning: There are router configuration problems”¶

If you see a warning on ddev start about router configuration problems, it is most likely a result of custom configuration problems, which could be an invalid docker-compose.*.yaml or leftover project .ddev/traefik/config/*.yaml files.

“Cannot start the provider *file.Provider error=”error adding file watcher: no space left on device”¶

This warning on ddev start is sometimes encountered on Linux systems, the solution is below.

Linux file watcher errors¶

If you encounter Traefik file watcher errors on Linux systems, these are common solutions:

Error adding file watcher: no space left on device¶

This error appears during ddev start or in the router logs, docker logs ddev-router, as:

ERR Cannot start the provider *file.Provider error=”error adding file watcher: no space left on device”

This indicates that the system has reached the limit for inotify watches on your Linux computer. First, check the current value:

sysctl fs.inotify.max_user_watches

Then create a configuration file to increase the limit (for example, if the current value is 65536, you can increase it to 524288):

echo 'fs.inotify.max_user_watches=524288' | sudo tee -a /etc/sysctl.d/60-inotify.conf

Apply the changes without restarting the system:

sudo sysctl -p /etc/sysctl.d/60-inotify.conf

Error creating file watcher: too many open files¶

This error appears during ddev start or in the router logs, docker logs ddev-router, as:

ERR Cannot start the provider *file.Provider error=”error creating file watcher: too many open files”

This indicates that too many inotify instances are open. First, check the current value:

sysctl fs.inotify.max_user_instances

Then create a configuration file to increase the limit (for example, if the current value is 128, you can increase it to 8192):

echo 'fs.inotify.max_user_instances=8192' | sudo tee -a /etc/sysctl.d/60-inotify.conf