Zimbra 8.8.12 quietly replaced Java 8 with Java 11, and combined with a change in Java Garbage Collectors, we are seeing significant mailbox server performance improvements just from doing an upgrade to 8.8.12.

Key Takeaway
If your mailbox servers are swapping but you believe you should already have sufficient RAM; if users are complaining that Zimbra seems to “pause” periodically, and; if you’ve already allocated enough RAM for the MariaDB InnoDB buffer pool, then the Java improvements introduced in 8.8.12 alone make upgrading Zimbra attractive. 
Here’s why:

“Garbage Collection” in layman’s terms is code within Java that cleans up the memory space allocated to Java. The amount of memory Zimbra will allocate to Java on startup of mailboxd (called the Java heap) is controlled by the localconfig variable “mailboxd_java_heap_size”, expressed in megabytes.

The Zimbra mailbox service is continuously creating and no longer using objects in the heap space; the job of the Garbage Collector is to clean up the heap space so that software developers can focus on software features and not worry (too much) about managing memory.

Not surprisingly, there are several different Java Garbage Collectors from which to choose, just like choosing between all-season, snow or summer-only high-performance tires for your car.  Depending on the nature of the Java application, some Garbage Collectors might be better suited than others, and heavy duty Java applications (of which Zimbra is certainly one) will benefit greatly when the optimal Garbage Collector best suited to the application is selected.

What’s Changed?
Java 11 (the next Java Long Term Release) introduced a new Garbage Collector called G1 (“Garbage First”), and in Zimbra 8.8.12, this is the default Garbage Collector. Earlier 8.8 versions of Zimbra used the CMS (“Concurrent Mark Sweep”) Garbage Collector.  A lot of Googling will turn up articles and blog posts showing that for workloads like Zimbra, just changing from the CMS to G1 Garbage Collector can yield performance improvements of 15% or more.

You can see what version of Java you have by running “java -version” as the Zimbra user.  Here’s the output from a Zimbra 8.7.11 system:

zimbra@webmail:~$ java -version
openjdk version "1.8.0_172-zimbra"
OpenJDK Runtime Environment (build 1.8.0_172-zimbra-b01)
OpenJDK 64-Bit Server VM (build 25.172-b01, mixed mode)
zimbra@webmail:~$

And here’s the output from a Zimbra 8.8.12 system:

zimbra@webmail:~$ java -version
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)
zimbra@webmail:~$

In the Zimbra systems I have upgraded to 8.8.12 over the past few weeks, the upgrade was accompanied with the mailbox servers showing less of a preponderance to use the swap file (and the busier the server, the less the swap file was used), as well as lower Load (as reported by top), and in a few cases, users noticed the server seemed “snappier” to respond.

The garbage collector logs also showed that garbage collection activities were taking a lot less time to complete.

One particularly hammered mailbox server, after the upgrade to 8.8.12, went from top reporting loads of more than 4 routinely, with more than 8GB of swap file usage, to loads of around 2 and less than 500MB of swapfile usage.

Why Is G1 Better Than CMS for Garbage Collection?
G1 carves the heap into different regions, and applies garbage collection activities to each region differently — and in a more efficient manner.  You can read more about it in Plain-English at this blog post.

But the better performance and memory utilization reported above may not be due entirely to the change in Java versions nor the change in Garbage Collectors.  Zimbra and ZeXtras have made performance improvements a development priority.  Zimbra 8.8.12 has an updated version of ZeXtras (“NG” or Next Generation tools) built in, and a review of Zimbra Release Notes for the latest 8.8.11 and 8.8.12 Patches shows a myriad of bug fixes.

What’s Next?
Our view is that 8.8.12, a Standard (i.e. Stable) Release, is effectively a Release Candidate for Zimbra’s next Long Term Release, 8.8.15, which is due out on July 1, 2019We recommend upgrading to 8.8.12 at your earliest convenience; if any of your customizations, integrations etc. have issues with 8.8.12, then now would be the time to open a Support Case with Zimbra so that your issue has a chance to be addressed and incorporated in 8.8.15.

Hope that helps,
L. Mark Stone
Mission Critical Email
26 April 2019

The information provided in this blog is intended for informational and educational purposes only. The views expressed herein are those of Mr. Stone personally. The contents of this site are not intended as advice for any purpose and are subject to change without notice. Mission Critical Email makes no warranties of any kind regarding the accuracy or completeness of any information on this site, and we make no representations regarding whether such information is up-to-date or applicable to any particular situation. All copyrights are reserved by Mr. Stone. Any portion of the material on this site may be used for personal or educational purposes provided appropriate attribution is given to Mr. Stone and this blog.