The Problem You Don’t Know You Have
You’ve got a nice reverse proxy setup. NGINX, HAProxy, Caddy — doesn’t matter. You grabbed an SSL cert from Let’s Encrypt, threw it in, and figured you were done. Things mostly work. Some connections fail mysteriously. Mobile phones hit it weirdly. Your mom’s browser times out. You blame the internet.
Here’s the thing: you’re probably only serving the leaf certificate, not the full chain. And browsers are way pickier about this than you think.
Why This Happens
When you get a cert from Let’s Encrypt (or any CA), you actually get three pieces:
- Your leaf cert — the one with your domain name
- The intermediate cert — signed by Let’s Encrypt’s intermediate
- The root cert — signed by a trusted root authority
Your reverse proxy needs to send items 1 and 2 to the client. The client already has the root in its trust store, so it doesn’t need that.
If you only send the leaf cert, the client has to figure out the chain on its own. Some clients do this fine (modern desktops). Others bail immediately (some mobile phones, older browsers, API clients). That’s why your production app works fine for you but your mom can’t visit.
How It Looks in NGINX
Here’s the rookie config:
server { listen 443 ssl; server_name example.com;
ssl_certificate /etc/letsencrypt/live/example.com/cert.pem; ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
# ... rest of config}That cert.pem is just the leaf. Your browser has to hunt down the intermediate, and if it can’t reach it (offline, bad network, whatever), the connection dies.
The fix is stupid simple:
server { listen 443 ssl; server_name example.com;
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
# ... rest of config}Use fullchain.pem, not cert.pem. Done. That file already has the leaf + intermediate bundled.
Checking If You’ve Got It Wrong
Want to verify your reverse proxy is sending the chain? Use openssl:
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null | grep -A 20 "Certificate chain"You should see something like:
Certificate chain 0 s:CN = example.com i:C = US, O = Let's Encrypt, CN = R3 1 s:C = US, O = Let's Encrypt, CN = R3 i:C = US, O = Internet Security Research Group, CN = ISRG Root X1If you only see one cert in that output, you’re missing the intermediate. Fix it.
Other Reverse Proxies
HAProxy:
ssl-default-bind-options ssl-min-ver TLSv1.2cert /etc/ssl/certs/fullchain.pemCaddy: Caddy handles this automatically. If you’re using Let’s Encrypt, it builds the chain for you. No config needed.
Traefik:
tls: certificates: - certFile: /certs/cert.pem keyFile: /certs/key.pem stores: default: defaultCertificate: certFile: /certs/fullchain.pem keyFile: /certs/key.pemBuilding a Chain Manually
If you got certs from a non-Let’s Encrypt CA and they gave you separate files, you can build the chain yourself:
cat cert.pem intermediate.pem > fullchain.pemOrder matters: leaf cert first, then intermediates. The root cert is optional (clients have it already). If you have multiple intermediates, add them in order from leaf to root.
Verify the chain is correct:
openssl verify -CAfile /etc/ssl/certs/ca-certificates.crt fullchain.pemYou should see fullchain.pem: OK. If you see “unable to get local issuer certificate”, something’s in the wrong order or missing.
Cert Expiry and Renewal
While you’re here: check when your cert expires:
echo | openssl s_client -connect example.com:443 2>/dev/null | openssl x509 -noout -datesOutput:
notBefore=Jan 1 00:00:00 2025 GMTnotAfter=Apr 1 00:00:00 2025 GMTLet’s Encrypt certs expire every 90 days. If you’re using Certbot with auto-renewal, the certbot.timer systemd unit handles it. But it only replaces the files — it doesn’t reload NGINX. Add a deploy hook:
#!/bin/bashsystemctl reload nginxchmod +x /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.shNow every renewal automatically reloads NGINX with the new chain. No manual intervention needed.
Why This Still Trips People Up
Let’s Encrypt used to be unclear about which file was which. The names cert.pem vs fullchain.pem aren’t exactly screaming “include both these files.” And most online tutorials show the wrong approach because they were written in 2016 and nobody updated them.
Modern ACME clients (Certbot, etc.) do the right thing by default now. But if you’re renewing old configs or using an older client, you might still have cert.pem lying around.
Bottom line: always check your reverse proxy config. Use fullchain.pem. Test with openssl s_client. Set up a renewal deploy hook. Your users — and your mom — will thank you.
Testing Behind a Load Balancer or CDN
Here’s a gotcha that bites people with Cloudflare or any other CDN in front of their origin: openssl s_client against your public domain hits the CDN’s cert, not yours. You can have a broken chain on your origin server and never know it — until Cloudflare falls back to full-strict mode or you bypass the proxy.
To check your origin directly, hit the origin IP on port 443 while faking the SNI and Host header:
echo | openssl s_client \ -connect 203.0.113.42:443 \ -servername example.com \ 2>/dev/null | grep -A 10 "Certificate chain"Replace 203.0.113.42 with your actual server IP. If the chain comes back with only one cert, your origin is broken — even if the public site looks fine through the CDN.
Same idea applies if you’re behind an internal load balancer. Your upstream health checks might be HTTP, so nothing flags the TLS issue. Then one day you rotate to full mTLS between the LB and origin and suddenly half your upstreams fail cert validation because they were never serving the chain correctly.
Another quick check from the origin server itself — no network needed:
openssl x509 -noout -subject -issuer -in /etc/letsencrypt/live/example.com/fullchain.pemopenssl x509 -noout -subject -issuer -in <(openssl x509 -in /etc/letsencrypt/live/example.com/fullchain.pem -skip 1)The first command shows the leaf cert’s subject and issuer. The second command (using process substitution) skips the first cert and shows the intermediate. If the intermediate’s subject matches the leaf’s issuer, your chain is properly ordered. If you get “no certificate or crl found” on the second command, you only have the leaf — go back and fix the config.