The MySQL 5.6 end of life date is February 5, 2021. Many Connectwise Automate partners are on MySQL 5.6, so here at Automation Theory, we wanted to clarify what that means for the day-to-day operations for Automate, and how to take action.
End of Life: Just like Windows
End of life in the Oracle world means very much the same thing as in the Microsoft world: no more security/stability patches are coming, and you can’t get help for issues via official support channels. Also, just like Windows, it means that keeping the old version in production starts to become a security liability. Vulnerabilities will be discovered, documented, and end up in all the common exploit scanners.
Why it matters to MSPs
First and foremost, MySQL patching is simply a matter of security. A Connectwise Automate instance is probably one of the most valuable targets for cybercriminals on an MSP’s network. Databases are also more challenging to protect; while the Automate application might have 2FA and lockout protection, no such measures exist by default at the database layer. It’s also worth noting that even read-only access could prove to be a nightmare; the credential data and computer information in Automate are likely enough for bad actors to launch successful attacks on the client computers.
Also, a factor for MSPs is auditing and compliance. Even if all the attack vectors were mitigated by a secure configuration, the fact remains that unsupported software is still in production. Any auditor (or vulnerability scanner) worth their salt should flag the software as being less secure. Most MSPs working in regulated industries get client requests for security reports of some type, and it would be difficult to explain away the fact that the RMM platform is running on an unsupported database.
Another important aspect would be the stability that comes with patching. At the publication time of this post the current version of MySQL is 5.7.32, and in the release notes bugs fixed include problems with stalled queries, concurrent updates, and more (bugs #30594501 and #31205266 respectively). Just like updates for Windows, updates for MySQL bring a plethora of security and stability fixes — and in a majority of the cases, the risks of not updating far outweigh any risks of installing the update.
Finally, vendor support is also an important consideration. The primary driver behind the lack of Server 2019 support is that Oracle only Supports MySQL 8.0 on that OS (which is incompatible with Automate out of the box). Connectwise has not yet announced any plans to stop formal support for MySQL 5.6, but the potential to use it as a scapegoat for certain support issues is there (here at Automation Theory we’ve had partners report this already!).
What should MSPs facing MySQL 5.6 end of life do?
We universally advise partners to upgrade to the latest version of MySQL 5.7. It contains performance improvements, additional functionality, and it’s supported until 2023. Most partners have never upgraded a MySQL database before, so our common suggestions are below.
First, if you haven’t upgraded MySQL before, we don’t advise using your production Automate server as a test environment. As IT people we often take the bull by the horns and perform lots of tasks that are potentially risky. It’s certainly possible to go that route with upgrading Automate’s database, but we’ve had several partners contact us after an upgrade has failed and the database won’t start (or the application isn’t working). These situations are never pleasant to clean up. Databases are also a bit more involved to backup (since disk-only snapshots don’t capture the data in RAM), so until you’ve verified that you can restore from your backups and get a working MySQL server we’d advise against a DIY crash course in MySQL upgrades.
Secondly, we advise against following the Connectwise documentation for doing MySQL patching. The method they suggest is known as a “dump and reload,” and that is normally only done in the case of data corruption. The process can take several hours depending on database size, and it is the most convoluted method possible for upgrading MySQL. It requires moving the data twice — and this is beneficial for anyone charging hourly to perform the upgrade (as many other consulting firms do).
For any partner wanting to take on the upgrade themselves, we suggest doing plenty of research — to the point where you understand the big picture of what’s happening. You’ll find many different guides for MySQL upgrades, but you’ll want to be prepared in the event you encounter issues not documented in the guide. It is “normal” to see databases that won’t start due to invalid configurations, permission issues, and software dependency issues when upgrading an average Automate stack from MySQL 5.6 to MySQL 5.7. We advise anyone wanting to go this route to be prepared for such events.
Is there an easier way to upgrade MySQL for Automate?
Yes! Here at Automation Theory, we offer MySQL patching as a flat-rate service. We perform an in-place upgrade, and the process takes about an hour total (including updating the MySQL Connector software as well). We’re able to work with our clients to schedule maintenance during non-business hours and to make sure that any special requirements are satisfied. We’re also certified MySQL DBA’s; we’ll make sure that the upgrade goes smoothly and that the application comes back online correctly.
What if I can’t upgrade?
There might be extenuating circumstances preventing you from upgrading before the MySQL 5.6 end of life date. Our advice in this situation is to harden the infrastructure as much as possible, primarily focusing on limiting MySQL network access to a whitelist of the devices that need it for daily operations. Ideally for non-split servers, the MySQL configuration can be set to only accept connections on the local machine, but this can be difficult depending on what other integrations are in use (many dashboards connect directly to the database).
We hope this has been helpful to you. Please don’t hesitate to contact us if you have any upgrade needs (or want to use the opportunity to migrate your Automate stack).
