You Asked Which Gateway to Self-Host. Half the Popular Answers Are Dead.
Every few months someone in a self-hosting forum asks “what’s a good self-hosted secure email gateway,” and every few months the same list of names gets copy-pasted back at them. The problem is that a chunk of that list stopped being maintained a while ago, and nobody updates the forum thread to say so. You go install one, hit a 404 on the download page, and lose an evening figuring out the project quietly died sometime in the last two years.
So let’s be precise about what we’re even talking about, because this term gets mushed together with a completely different category of software.
A secure email gateway (SEG) sits in front of wherever your mailboxes actually live — Microsoft 365, Google Workspace, on-prem Exchange, or a self-hosted mail server you’re already running like mailcow — and does the filtering, quarantine, and policy enforcement before mail ever reaches the backend. It doesn’t store your mail. It’s the bouncer, not the bar.
A mail server hosts the actual mailboxes: IMAP, SMTP submission, webmail, the works. That’s a different job, and if you’re comparing mailcow vs Mailu vs Stalwart, you want the sibling article, not this one.
This article is only about the bouncer. Here’s the honest state of that world in 2026: one boringly reliable option that’s still the default answer, one ambitious newcomer-that-isn’t-really-new wearing a young container wrapper, one DIY route for people who like owning their config, and a graveyard section for names you should stop recommending.
Why Even Run a Gateway?
Quick gut check before we get into products, because “add another box in front of my email” isn’t free complexity.
- Central filtering in front of a backend you don’t want to mess with. M365 and Workspace both have spam filtering, but it’s a black box you can’t tune, and a gateway lets you add your own rules, quarantine, and greylisting in front of it without touching the tenant config.
- One quarantine to check, not five mailboxes’ worth of junk folders. Admins get a single pane for what got blocked.
- Outbound control. If a compromised workstation or a hacked mailbox starts blasting spam, a gateway can throttle or kill outbound before your whole domain gets blacklisted. This is the one people underrate until it happens to them.
- Keeping spam/malware scanning off the mail server itself. ClamAV and spam scoring are CPU and RAM hungry. Doing that work on a dedicated box in front of mailcow (or Exchange) means your mail server stays lean and fast.
- Encryption and compliance policy in one choke point, instead of trying to enforce it per-mailbox.
None of this is mandatory. If you’re running a two-person mailcow instance for your household, you probably don’t need a gateway — Rspamd inside mailcow already does real filtering. This gets useful once you’ve got a backend you don’t fully control (M365/Workspace) or enough mailboxes that a single quarantine view actually saves time.
Proxmox Mail Gateway — The Pick
If you already run Proxmox VE, this name will feel familiar, and that’s not a coincidence — it’s the same company. Proxmox Mail Gateway (PMG) is a Debian-based appliance whose entire job is sitting in front of your real mail server and relaying clean mail through. It does not host mailboxes. That’s not a missing feature, it’s the design: you point PMG at your backend (Exchange, M365, Google Workspace, or your own Postfix/mailcow box) as a relay destination, and PMG becomes the front door everything has to pass through first.
Under the hood it’s the stuff you’d expect from a mature mail-security stack: Postfix handling SMTP, SpamAssassin scoring junk, ClamAV scanning attachments. On top of that it layers greylisting, postscreen, RBL checks, and receiver verification — the last of which is worth calling out specifically, because rejecting mail for addresses that don’t exist on your backend at the SMTP level (instead of accepting it and bouncing later) can cut the junk traffic your server actually has to process by something like 90%. Add per-user quarantine digests, a rule/object system for building your own policies, cluster support so you can join multiple PMG nodes under one GUI, TOTP login, and encrypted backups, and you’ve got a full appliance, not a script.
It’s also refreshingly light. Minimum spec is 2 GB RAM, 4 GB recommended, around 8 GB of disk. A 2-core VM with 2 GB of RAM will comfortably chew through 100,000+ messages a day. You can run it as a bare-metal ISO install, or as a VM/LXC-adjacent guest under Proxmox VE, VMware, Hyper-V, KVM, or even VirtualBox for testing. Current major version is the 9.x line.
Licensing follows the same playbook as Proxmox VE itself: it’s AGPLv3, and every feature is available for free, full stop. There’s an optional paid subscription (starting around €110 per host per year) that gets you the enterprise repo — which is just the more heavily tested, slower-moving package repository — plus vendor support. You do not need the subscription to run this in production. You just accept slightly less battle-tested package updates on the free “no-subscription” repo, the same tradeoff Proxmox VE admins have been making for years.
Conceptually, getting mail flowing looks like this: install the appliance, then tell it where your real mailboxes live.
Relay/Transport concept for PMG: 1. Add your domain(s) under Configuration > Domains 2. Set a Transport rule pointing the domain at your backend's hostname/IP (Exchange, M365 connector, or your mailcow box) 3. Point your domain's MX record at the PMG instance instead of your backend directly 4. PMG receives, filters, quarantines-or-passes, then relays clean mail to the transport targetThe exact fields differ by backend (M365 wants a dedicated connector, Exchange wants an SMTP transport, mailcow just wants a relay host), but the shape is always: PMG in front, your mail server behind it, MX pointed at PMG.
Who it’s for: basically everyone. If you need a gateway and don’t have a strong reason to pick something else, start here. The catch: it’s a relay, not storage — you still need a real mail server behind it, and if that’s part of your confusion, go read the mailcow/Mailu/Stalwart article first. Verdict: the default correct answer. Boring, well-funded, huge install base, does exactly what it says.
Hermes SEG — The Ambitious Newcomer That Isn’t Actually New
Here’s where you need to read carefully, because there’s a nuance that gets lost every time this comes up.
Hermes SEG, from deeztek, has existed for years as a bare-metal Ubuntu installer (docs at docs.deeztek.com, product site hermesseg.io). It is not a new project. What’s new is that it was recently repackaged as a “Docker Edition” — and that containerized packaging, the thing you’d actually deploy today, has a genuinely thin public track record: the GitHub repo sits around 13 commits and roughly 100 stars, with one giant squashed initial commit standing in for whatever development history actually happened. So the honest framing is: mature product concept, mature underlying components (Postfix, Dovecot, SpamAssassin have decades between them), young and lightly-vetted container packaging. The README also reads more like a marketing page than a docs page, which isn’t disqualifying, but it’s worth knowing going in.
What makes Hermes SEG unusual is scope. It’s not “just” a gateway — it can be a gateway, a full mail server, or both at once, running some domains as pure filter-and-relay and others as full mailbox hosting, all from a single deployment.
The Docker Edition compose stack is not small: 19 containers. Postfix plus OpenDKIM/OpenDMARC/OpenARC for mail authentication, Amavisd-new tying together SpamAssassin and ClamAV for filtering, Dovecot for actual mailboxes, Nextcloud bolted on for webmail/files/CalDAV/CardDAV, OpenLDAP plus Authelia for SSO with MFA (TOTP, WebAuthn, Duo), CipherMail handling S/MIME, PGP, and encrypted-PDF delivery, Dovecot mail-crypt for at-rest encryption, MariaDB as the backing store, Unbound for DNS, a Lucee/CFML application server, and Nginx in front of all of it. The Pro tier adds “Link Guard,” time-of-click link rewriting — the self-hosted answer to what Proofpoint and Mimecast charge enterprise money for.
That’s a lot of moving parts, and the hardware requirement reflects it: minimum 4 vCPU / 8 GB RAM, 16 GB+ recommended for anything production-facing, roughly 120 GB just for the config/OS disk, with separate storage tiers for data, archive, vmail, and Nextcloud on top of that. This is comfortably more than double the heaviest option in the mail-server comparison article — running Hermes is closer to standing up a small appliance farm than dropping in a relay box.
Licensing is a two-tier split: Community Edition is AGPLv3 and gets you the full stack. Pro is commercial, licensed per-server rather than per-mailbox (a genuinely nice model if you’ve got a lot of users), and adds the advanced features including Link Guard.
Who it’s for: a small org that wants filtering, encryption, compliance features, and mailbox hosting in one deployment, and has real hardware to throw at it. The catch: heavy, and the Docker packaging specifically is young — kick the tires hard in a lab before you bet a company’s mail flow on it. Verdict: genuinely interesting, worth watching mature over the next year or two, but not the safe default the way PMG is.
DIY: Rspamd + Postfix — The Hacker Option
If appliances make you itchy, you can build the gateway yourself: Postfix configured to accept mail for your domains and relay it to your real backend, with Rspamd hooked in as a milter doing the actual spam scoring, DKIM verification, and policy work. Optionally throw ClamAV in front for attachment scanning.
Rspamd’s a good foundation for this — it’s actively developed, fast (it’s not writing Perl regexes against every message like the older tools), and ships a web UI for watching scores and managing what got quarantined. The appeal here is total control and minimal moving parts: two services, not nineteen containers.
A minimal shape, as a compose file plus the Postfix side you’d configure to talk to it:
services: postfix: image: your-postfix-relay-image ports: - "25:25" volumes: - ./postfix-config:/etc/postfix depends_on: - rspamd
rspamd: image: rspamd/rspamd:latest volumes: - ./rspamd-config:/etc/rspamd/local.d - rspamd-data:/var/lib/rspamd ports: - "11334:11334" # web UI
volumes: rspamd-data:On the Postfix side, you’d point mail at the backend and wire in the milter:
relay_domains = yourdomain.comtransport_maps = hash:/etc/postfix/transportsmtpd_milters = inet:rspamd:11332non_smtpd_milters = $smtpd_miltersmilter_default_action = acceptAnd /etc/postfix/transport just tells Postfix where “clean” mail actually goes:
yourdomain.com smtp:[mailcow.internal]:25Pros: you own every line of config, no appliance overhead, no vendor to wait on for a patch, and it’s free. Cons: you own every line of config — there’s no support line, no quarantine UI handed to you out of the box (Rspamd’s UI covers scoring, not a polished end-user quarantine digest), and every edge case is yours to debug at 2 AM. This is the option for people who already enjoy owning their Caddy config and their Ansible playbooks, not people who want to install something once and forget about it.
Here Lie the Dead
Two names still get recommended in old forum threads. Don’t use either for anything new.
MailCleaner Community Edition — the GitHub repo was archived (read-only) in April 2025, development is limited to occasional bug/migration fixes at most, and the virtual appliance itself was discontinued on March 31, 2025. There’s no current ISO to grab. A commercial version of MailCleaner still exists, but the free, self-hosted Community Edition is done. It was MailScanner-based under the hood, for what that’s worth to the archaeology.
Scrollout F1 — pitched as a beginner-friendly anti-spam and DLP gateway, and it’s still technically downloadable, but it’s been dormant for years; a lot of its documentation reads like it’s stuck around 2013. Nobody’s actively maintaining it.
Both still show up when people search “self-hosted email gateway,” because search results don’t expire. Don’t build anything new on either.
Decision Matrix
- Pick Proxmox Mail Gateway if: you want the boring, proven, well-supported default. This is almost everyone.
- Pick Hermes SEG if: you want gateway + mailbox hosting + encryption + compliance in one all-in-one stack, you’ve got real hardware to give it, and you’re willing to vet the Docker Edition properly before trusting it with anything important.
- Pick DIY Rspamd + Postfix if: you already enjoy owning your infrastructure config and want the absolute minimum moving parts with maximum control.
- Just want to host mailboxes, not filter in front of them? You’re in the wrong article — see the mailcow vs Mailu vs Stalwart comparison instead.
Wrapping Up
For most people reading this, the answer is Proxmox Mail Gateway. It’s light, it’s free with every feature unlocked, it’s been doing this job for a long time, and “boring” is a compliment when it comes to anything sitting in front of your mail flow. Hermes SEG is the one to watch — genuinely ambitious scope, but the Docker Edition you’d actually deploy is young enough that it deserves a real shakedown in a lab before you trust it with production mail. DIY Rspamd is for the crowd that already runs their own reverse proxy config by hand and likes it that way.
Whatever you do, don’t be the person three years from now asking why your MailCleaner install stopped getting updates. Check whether a project’s still breathing before you build your mail security around it.
