It is already included in the community release and will soon be packaged and available to all grommunio customers: Thanks to a change in the way grommunio handles email mailbox databases, teams will benefit from significantly improved mailbox performance.
The problem is complex, but widespread: In the past, it could happen that users experienced longer waiting times when many clients accessed individual mailboxes at the same time, for example when several users carried out longer search queries on a mailbox at the same time, such as when preparing for a meeting. The reason for this is the desire for data consistency: only one client at a time may access the data in a mailbox and make changes, after which it is the next client’s turn – otherwise there is a risk of inconsistencies. This “serial” access (i.e. one after the other) could take a long time, especially with larger data sets (and numerous clients wanting to access at the same time). The problem is not limited to grommunio, but applies in principle to all web and mail services: If you want consistent data, you have to process accesses one after the other; if you want high performance, you have to parallelize accesses. But there are smart solutions that can help with this dilemma. grommunio has implemented one of these and thus achieved a major increase in performance.
Write and read-only: serial or parallel?
The solution to the problem: A distinction is made between write and read-only access. Read-only access (e.g. by a mail client updating a shared folder) can be permitted by the server in parallel by several clients at the same time, and caching is also possible. A “single source of truth” is only required for writing, i.e. exclusive access by one client – otherwise inconsistent data may occur or one client may overwrite the changes made by another.
Implementing this technically requires some understanding of how grommunio (or any other mail and database server) works. In grommunio there is a central MySQL database for the metadata and caches for the user’s metadata, i.e. a large table with all data, including, for example, the table of contents for information about users and the assignment of where exactly a user’s mailbox can be found.
There is also an SQLite database for each user, which contains the specific user data, including all emails. For the individual user, it is also the central component of the mailbox, but it can also be a bottleneck if numerous requests come in at the same time. Nevertheless, the system is flexible and has proven itself, it can be scaled as required and is an important part of grommunio’s high mailbox performance.
However, if a client performs a search in a larger mailbox, for example, this is blocked for other requests until the search request has been fully processed. On the server side, this is implemented via a Mutex, other clients have to wait and, in the worst case, run into a timeout – which means an error message for the user.
Increased mailbox performance thanks to a management server that makes situational decisions
Because the Mutex doesn’t really make sense for search queries or similar read-only accesses, as the database is not changed, the developers of grommunio have implemented a solution that removes this bottleneck. The developers have adapted the management server (exmdb) for this purpose. It is complex and has already coordinated access with its more than 120 functions. To incorporate parallelization for read-only access, the developers had to adapt over 18,000 lines of code. Where previously one Mutex per mailbox prevented conflicts but also occasionally caused congestion, the new system manages without a “large” mutex, instead inspecting each request and deciding on a situational basis whether it can be safely parallelized.
“This approach proved to be significantly faster, more flexible and is noticeable for every team that works with shared mailboxes, especially in day-to-day work. A typical example is the joint preparation of a team meeting, where all participants want to quickly update each other in the hours before the meeting. But that’s just one example, users tell us enthusiastically about other situations that have now been significantly accelerated by the change. “, says grommunio CTO Michael Kromer.
Learn about other grommunio features.