Soliton MailZen Cloud - Cloud Management Portal and Push Server issue because of certificate revocation!
Incident Report for Soliton Systems Europe - Status
Resolved
Dear Customers and Partners,

The monitoring from our side is complete and we have resumed normal operations!

We apologise for any inconvenience this may cause and thank you for your patience and understanding. As always, if you have any questions or concerns, please submit them at: https://solitonsystems.atlassian.net/servicedesk or send us an email at: help@solitonsystems.com.

Thank you for your time and please stay healthy.

Sincerely,

The Soliton Systems IT Service Management Team
_____________________________________________
SOLITON SYSTEMS EUROPE N.V.
Barbara Strozzilaan 364 | 1083 HN Amsterdam | Netherlands | P: +31 (0) 20 896 5841NL | www.solitonsystems.com
Posted Jun 30, 2024 - 14:18 CEST
Monitoring
Dear Customers and Partners,

The new certificate has been installed and we have verified its functionality. We have also received positive feedback from customers and partners that they are now able to log in using Firefox and Safari as well. We will continue to monitor the situation for a short period of time and will soon resume normal operations.

The fingerprint (SHA256) of the new certificate is: c151a7dfbda254369d2d9b604340a5deb71d22c81c383cd36b146af689429363

We apologise for any inconvenience this may cause and thank you for your patience and understanding. As always, if you have any questions or concerns, please submit them at: https://solitonsystems.atlassian.net/servicedesk or send us an email at: help@solitonsystems.com.

Thank you for your time and please stay healthy.

Sincerely,

The Soliton Systems IT Service Management Team
_____________________________________________
SOLITON SYSTEMS EUROPE N.V.
Barbara Strozzilaan 364 | 1083 HN Amsterdam | Netherlands | P: +31 (0) 20 896 5841NL | www.solitonsystems.com
Posted Jun 30, 2024 - 10:45 CEST
Update
Dear Customers and Partners,

We have waited now for more than 24h to propagate our new CAA, and we will now reissue our certificate.

While we have investigated the root cause and talked to customers and partners, we found out that the different internet browsers we are using show different behavior when checking revoked certificates. Google Chrome, Microsoft EDGE, and Opera are very "relaxed" and do not even show a warning. They call it "soft-fail." Firefox shows a "SEC_ERROR_REVOKED_CERTIFICATE," and Safari shows a strong "Your connection isn't private." The last two do not even allow you an option to show the website with the revoked certificate, while the "relaxed" ones do not even giving you a warning. We leave the browser choice up to you, but at least you should think about the "relaxed" ones ;-)

Good reading: https://www.gradenegger.eu/en/google-chrome-does-not-check-revocation-status-of-certificates/

We apologise for any inconvenience this may cause and thank you for your patience and understanding. As always, if you have any questions or concerns, please submit them at: https://solitonsystems.atlassian.net/servicedesk or send us an email at: help@solitonsystems.com.

Thank you for your time and please stay healthy.

Sincerely,

The Soliton Systems IT Service Management Team
_____________________________________________
SOLITON SYSTEMS EUROPE N.V.
Barbara Strozzilaan 364 | 1083 HN Amsterdam | Netherlands | P: +31 (0) 20 896 5841NL | www.solitonsystems.com
Posted Jun 30, 2024 - 08:50 CEST
Identified
Dear Customers and Partners,

We have investigated further and a missing DNS CAA record seems to be the root cause of this issue. On short notice, our certificate provider has informed us that they will revoke our wildcard certificate, and we need to rekey the certificate. We have updated our DNS Zone and are now awaiting the availability of the new record to be active. Sadly, this can take up to 24 hours and cannot be enforced by us. We are relying here on our service providers. As soon as the CAA is in place, we will reissue our certificate and reconfigure our MailZen infrastructure with the new certificates.

We would like to stress that this issue is neither our fault nor a security issue with the certificate we are using. Our certificate provider has revoked the certificate because they stated a weak process from their side.

Our team is currently investigating this issue further, and we will provide regular updates. The next update will be provided at the latest by 15:00 CEST.

We apologize for any inconvenience this may cause and thank you for your patience and understanding. As always, if you have any questions or concerns, please submit them at: https://solitonsystems.atlassian.net/servicedesk or send us an email at: help@solitonsystems.com.

Thank you for your time and please stay healthy.

Sincerely,

The Soliton Systems IT Service Management Team
_____________________________________________
SOLITON SYSTEMS EUROPE N.V.
Barbara Strozzilaan 364 | 1083 HN Amsterdam | Netherlands | P: +31 (0) 20 896 5841NL | www.solitonsystems.com
Posted Jun 29, 2024 - 08:21 CEST
Investigating
Dear Customers and Partners,

We are currently experiencing issues with our MailZen Cloud service. Partners and customers attempting to access the Solion MailZen Cloud Portal getting the browser information, that our certificate was revoked and with that the access to the MailZen Management portal is not possible. However, devices are functioning properly. Push notifications might also be affected, because we are securing the TLS connection between the Exchange servers and the push server with the same certificate. Exchange Subscriptions cannot be renewed but will remain active until the renewal time has reached.

While this situation impacts both our Cloud and on-premise customers (with regard to push notifications only), we want to emphasize that GoDaddy informed us on short notice that they are revoking our certificate due to an issuance mistake from their side.

Our team is currently investigating this issue and we will provide regular updates. The next update will be provided at 08:00am CEST.

We apologize for any inconvenience this may cause and thank you for your patience and understanding. As always, if you have any questions or concerns, please submit them at: https://solitonsystems.atlassian.net/servicedesk or send us an email at: help@solitonsystems.com.

Thank you for your time and please stay healthy.

Sincerely,

The Soliton Systems IT Service Management Team
_____________________________________________
SOLITON SYSTEMS EUROPE N.V.
Barbara Strozzilaan 364 | 1083 HN Amsterdam | Netherlands | P: +31 (0) 20 896 5841NL | www.solitonsystems.com
Posted Jun 29, 2024 - 00:38 CEST
This incident affected: Soliton MailZen Cloud Service - Europe (MailZen - Management Portal, MailZen - Push Server).