Zimbra Security Tip – Regulate Outbound Traffic With Stateful Firewalls

Zimbra Security Tip – Regulate Outbound Traffic With Stateful Firewalls

Zimbra Tips

Historically, Zimbra firewalling has focused on regulating only Inbound traffic, but we can better protect our Zimbra systems if we also regulate Outbound traffic by using a Stateful Firewall.

What Is a Stateful Firewall?
Stateful firewalls, which now account for the majority of SMB and enterprise grade firewalls, can track sessions by examining traffic at Layers 3 and 4 of the 7-layer OSI model.  In other words, they are able to match up the inbound and outbound traffic of (and different ports used by) a “session”, e.g. when a user connects to and logs in to Zimbra (inbound), and then Zimbra paints the contents of their email via the Zimbra web interface (outbound). Different firewall companies call this “deep packet inspection” or other similar terms.

Why Are Stateful Firewalls Useful?
Stateful firewalls significantly simplify configuring inbound and outbound rules. Once an inbound rule is configured (say, to allow http and https connections to Zimbra’s Proxy service), there is no need to configure outbound firewall rules for when Zimbra’s Proxy service sends traffic to a successfully authenticated web client user. With a stateless firewall however, one would have to configure both inbound and outbound rules manually to support our web client login example — and due to ephemeral ports, you might need to configure tens of thousands of allowed ports to allow two-way traffic with a stateless firewall. With stateful firewalls tracking connections, there is no need to configure any ports for return traffic, nor indeed to configure any outbound rules to cover outbound traffic where the connection was initiated from outside of Zimbra (and therefore managed by the inbound rule).

Most importantly, stateful firewalls allow us to block almost all common data exfiltrations from a Zimbra server which has been breached, for example via the recent CVE-2022-27925 attack (now patched).

This is because, in the recent CVE-2022-27925 attack, data exfiltration took place over http and https ports to web servers controlled by the attacker.  IOW, customers were protected if they were using a stateful firewall and had configured Outbound rules to allow Zimbra to initiate http/s connections only to known good servers.  The exploit that allowed a bad attacker to deposit a rogue jsp file in a web accessible directory still existed, but after the attacker next deposited software to exfiltrate data, no data exfiltration could take place over http/s.

Put differently, Zimbra itself doesn’t need to initiate a lot of outbound connections, and, attackers rely on a compromised Zimbra server to initiate outbound connections to talk to command and control servers and exfiltrate data. Consequently, limiting Zimbra’s ability to initiate outbound connections only to known good/required destinations improves security.  It won’t stop an attacker who has figured out a new way to drop a rogue jsp file into the jetty public directory, but it will prevent that rogue jsp file from successfully executing a wget command to download malware onto the file system — malware that will then try to initiate an outbound connection to a command and control server, and/or exfiltrate data via an outbound ssh connection.

Stateful Firewalling on Amazon Web Services
AWS’s Security Groups feature is in fact a stateful firewall.  By default, Outbound Rules are set to Allow ALL, but once you create a new outbound rule, the default rule is automagically removed.

You can then add all of the needed rules to allow Zimbra to function.

What Outbound Rules Are Needed?
Probably fewer than you might be thinking!  But here are the outbound ports/protocols/destinations needed by Zimbra based on our tests to date.  Since AWS Security Groups accept only IP addresses as inputs, and because some of the services use content distribution networks, we suggest you do DNS lookups from your Zimbra server(s) to determine the IP addresses to use.

Note that many of these services’ IP addresses will change.  For example, Zimbra’s package repositories leverage Cloudfare, so their IPs are changing frequently.  You can update the rule(s) when it’s time to install a new Zimbra Patch, or alternatively, you can mirror Zimbra’s repositories to a server you control and safeguard, and whose IP address will never change.

Remember… in a Stateful Firewall, outbound firewall rules are needed only for outbound connections initiated by Zimbra — not for inbound requests to which Zimbra is responding with outbound traffic. Here’s the list of connections initiated by Zimbra which need supporting Outbound firewall rules.

  • Repositories.  We use Ubuntu, so we need to add the IP addresses for Ubuntu’s and Zimbra’s repositories.  Note that Ubuntu accesses its repos via http, and Zimbra accesses its repositories via https. You’ll need to get the IP addresses for:
    • archive.ubuntu.com, and allow outbound connections to this IP for http
    • security.ubuntu.com via http (may be some IP overlap with archive.ubuntu.com)
    • repo.zimbra.com via https
    • Ubuntu’s Key Servers (162.213.33.8 and 162.213.33.9) via http
  • DNS Forwarders.  Whether you use dnsmasq or Zimbra’s Unbound DNS cache, you’ll need to open UDP (and also TCP if you use TCP for DNS lookups) to the IP addresses of your forwarders on port 53.
  • Zimbra’s License Server.  When performing a fresh Zimbra installation or version upgrade, you’ll need to get the current IP addresses for license.zimbra.com and allow http and https connections.
  • Time Syncing.  We prefer chrony over NTP, but regardless, we recommend NOT having Zimbra sync against some of the default “pool….” settings, as many pool members can be minutes apart.  Pick a single reputable time service server (AWS provides free time synchronization services at a single IP address) and allow outbound UDP connections to that time server’s IP address on port 123.
  • Downloading Zimbra.  Zimbra’s installers are downloaded from files.zimbra.com, so after you get the IP addresses for that fqdn you’ll need to allow outbound https to those IPs.
  • SMTP. This one is easy…  You should allow outbound TCP connections on port 25 to 0.0.0.0/0 if Zimbra delivers your mail outbound for you.  If Zimbra delivers email via a relay host, you should limit outbound TCP port 25 traffic to just the IP(s) of your relay host(s).
  • Inter- and Intra-Server Traffic. Hopefully your Zimbra servers are in an entirely separate, say, /24 network, in which case you can just allow All Traffic between hosts on the /24 network.  If your Zimbra servers are on the same network as your end users, you’ll need to reference Zimbra’s Ports Wiki and add all the inter- and intra-server ports and protocols individually, for all of the IP addresses of your Zimbra servers.  This is one reason why we strongly recommend deploying Zimbra on its own, separate network.
  • Zimbra BSP Partners. ZURT requires outbound TCP port 8443 access to usage.zimbra.com.
  • Zimbra Support’s FTP Site.  Zimbra Support for some cases will ask you to upload content to ftp.zimbra.com. If you perform this uploading directly from your Zimbra servers, you’ll need to allow passive FTP ports to reach the IP address of ftp.zimbra.com.
  • External Network Storage.  If you are using S3 for storage in AWS, there is a json-formatted file available from AWS that you can search to get the IP addresses for S3 buckets, filtered by Region.  S3 bucket lookups are via DNS (why you need to name newly created S3 buckets with a DNS-compliant name), and the IPs in use change frequently as S3 storage replication moves new storage frames into the pool and life cycles older storage frames out of the pool.  Depending on the external network storage you are using, you’ll need to open different ports and protocols.  If you get it wrong, the Zimbra web interface will report “NO_SUCH_BLOB” when a user clicks on an email to have the entirety of the email’s contents appear in the reading pane.  This does not cause corruption (except if an HSM job is in progress).
  • External Email.  If you allow your users to add External Email accounts (i.e., have Zimbra fetch via IMAP their gmail account),  IMAPS uses TCP/993; you may want to limit this to the IP addresses of the external mail servers from which users have configured to fetch their email.
  • External LDAP/Active Directory/GAL Sync/Exchange.  If you have configured external Domain Authentication, or are running a Split Domain environment with Exchange, you’ll need to add the IP and TCP port(s) used to access the domain controllers (or external LDAP directory servers) as well as the Exchange server used by Zimbra’s Free/Busy Connector.  If you have configured Zimbra’s GAL Sync accounts to obtain data from external GAL directories, you’ll need to add the ports and protocols for those external directory servers as well.
  • External Syslogging.  If you have a centralized syslog server, and have configured the Zimbra servers to “push” copies of their log files out to your syslog server, you’ll need to open UDP (perhaps TCP as well) to the IP address of your syslog server.
  • SpamAssassin and ClamAV Rule Updates.  As Zimbra updates versions of these products, be sure to allow outbound connections to enable this functionality. SpamAssassin updates use updates.spamassassin.org. ClamAV needs https access to database.clamav.net.
  • Lets Encrypt.  As there are several ways to deploy Lets Encrypt within Zimbra, I will leave it to the reader to document the systems, ports and protocols to which Zimbra needs to connect to obtain and renew Lets Encrypt certificates.

 

Conclusions
Except for the case where your Zimbra server(s) are on the same LAN segment as your end users, configuring Outbound firewall rules for Zimbra in a Stateful firewall is a straightforward exercise, and will provide an extra layer of data exfiltration protection, as well as breach protection from malware that needs to initiate outbound connections to command and control servers.  With Amazon Web Services, the easy-to-configure Security Group function provides a powerful stateful firewall service for no additional cost.

If you’d like help with your Zimbra’s security situation, note that we have a generalized fixed-price security hardening service for smaller environments, but would be happy to discuss a bespoke engagement as well.  Please feel free to start the conversation by filling out this enquiry form:

Hope that helps,
L. Mark Stone
Mission Critical Email LLC
26 October 2022

P.S. Thank you to Matthew Francis at In-Tuition for his suggestions on improving this blog post.

The information provided in this blog is intended for informational and educational purposes only. The views expressed herein are those of Mr. Stone personally. The contents of this site are not intended as advice for any purpose and are subject to change without notice. Mission Critical Email makes no warranties of any kind regarding the accuracy or completeness of any information on this site, and we make no representations regarding whether such information is up-to-date or applicable to any particular situation. All copyrights are reserved by Mr. Stone. Any portion of the material on this site may be used for personal or educational purposes provided appropriate attribution is given to Mr. Stone and this blog.