Author: Darryl Leaning

It’s really not uncommon to ‘do everything right’, yet end up with a unifi CloudKey that is messed up after clicking update in the web ui/app.

Various things can cause this, from out of sequence updates, to the base software getting downgraded when the firmware is updated etc. If you are here, you already know what I mean.

Symptoms can include, but are not limited to:

Connection refused when trying to open the cloudkey on its management port (usually 8443)
Being able to get to the cloudkey options, but trying to manage the network results in a blank page
Being able to get to the cloudkey options, but logging into the cloudkey never gets past the login screen
The CloudKey favicon showing in browser, but just a blank page displayed

The CK light might be flashing or solid on.

This guide is targeted at the CloudKey Gen1, it ‘may’ help with other versions. You will need to be able to connect to the CK and SSH onto it. If you can’t, see the emergency recovery note at the bottom of this post.

So, for these steps you will need the credentials for the key and for it to not be totally corrupted (i.e. you can SSH to it). these are the same you normally login with (admin/[password]). This guide is for windows, but should be translatable if you are on another platform. DISCLAIMER: This guide is for reference. If you connect to any device via SSH, then the responsibility is on you to ensure you don’t make things worse.

Dont ever try to restart these by pulling the power/PoE connection. That will likely make things a lot worse if you can at the moment still SSH into the key. A lot worse means emergency recovery and backup restores.

Step 1: try to get the backups off the Cloudkey using WinSCP. Login to the CK IP, accept any certificates and browse to data/autobackup. Copy everything off the key just in case. If you have no backups here, put that in your notes to turn on when you have a working CK. You’ll need these if you have to do the emergency recovery later.

Step 2: Login to the key via SSH, I like to use SolarPutty, but anything that works will do. Try issuing a reboot command. Wait at least 15 minutes to

Step 3: Get yourself some information about the software on the key (this may or may not match what you see in network.unifi.ui.com before if went offline, and may be a lesser number). The command is: dpkg -l | grep unifi

Step 4: Grab the latest firmware and software URLS from https://www.ui.com/download/unifi/unifi-cloud-key. You’ll be able to see them once you accept the EULA. These are for using over SSH on the CK.

Step 5: (This may or may not be required). Run the 2 shell commands from https://community.ui.com/releases/UniFi-Cloud-Key-Firmware-1-1-6/525efb47-ce23-434c-ae73-49f4d44251bc#comment/42e00042-fc58-455b-9a53-fc09a70ffd1a
referenced here:
unifi_version=$(grep -io “We do not support upgrading from 5.” /usr/lib/unifi/logs/server.log | grep -io “5.” | cut -d’.’ -f1-3 | sort -V | tail -n 1)
if [[ -n “${unifi_version}” ]]; then rm unifi_sysvinit_all.deb 2> /dev/null; wget https://dl.ui.com/unifi/${unifi_version}/unifi_sysvinit_all.deb && dpkg -i unifi_sysvinit_all.deb && rm unifi_sysvinit_all.deb; fi

Step 6: Ensure ubnt tools are up to date with: apt-get update && apt-get install –only-upgrade ubnt-tools (This is refernced here https://automatica.com.au/2018/06/solution-to-failed-firmware-updates-in-a-unifi-cloud-key/ )
I saw that the version here was lower than I expected, but ran the command, let it complete and moved onto next steps.

Step 7: Firmware update. Using the URL and latest firmware you found above, issue the firmware update command: ubnt-systool fwupdate https://dl.ui.com/unifi/cloudkey/firmware/…… (an example would be ubnt-systool fwupdate https://dl.ui.com/unifi/cloudkey/firmware/UCK/UCK.mtk7623.v1.1.13.818cc5f.200430.0950.bin ). Wait for it to complete. Give it plenty of time after completing, i.e. go make a drink.

Step 8: Software update preparation. Using the software URL you found above, issue the commands:
cd /tmp (to move to temp dir.)
ls – l (To check if there are any previous software update files there (none in my case). If you do have them, issue ” rm unifi_sysvinit_all.deb ” to remove them.

Step 9: Software update download. Issue the command wget https://dl.ui.com/unifi/x.xx.xx_version/unifi_sysvinit_all.deb but substitute the URL you found for the latest software earlier (an example would be wget https://dl.ui.com/unifi/6.1.71/unifi_sysvinit_all.deb ). Wait for it to complete.
(referenced here https://help.ui.com/hc/en-us/articles/216655518-UniFi-How-to-manually-change-the-controller-version-on-Cloud-Key-via-SSH )

Step 10: Install the software with: sudo dpkg -i unifi_sysvinit_all.deb

Hopefully after this completes, you’ll have the latest firmware and software. Your current config will be able to load and you’ll then be able to get back onto the management of the CloudKey/network.

If you cannot SSH, or this doesn’t work, then recovery may be the next steps. See here for that: https://help.ui.com/hc/en-us/articles/220334168-UniFi-Cloud-Key-Emergency-Recovery-UI#2
You’ll need a backup from somewhere for this.

Picture this:

Everything on your network is running fine, multiple sites, multiple VPNs, multiple domains. The site may or may not be virtualized.

Something dramatic happens (lets say a power cut to the head office). All the equipment comes back up in a strange order (servers before SAN, Exchange before AD etc).

You get the servers back online but strange things are happening. There are lots of errors in the event logs, relating to Active Directory and DNS. Failure to find domain, replication, zone transfer failures etc. All the fun stuff.

After a little investigation (and waiting for very unhappy exchange servers that cannot find AD to respond), you may find some strange behaviour.

Servers might have dropped onto the public network and are unable to replicate or communicate properly with the domain. I say properly, because some things will work and some things wont.

Exchange might have most of its services still starting, and it will be a total pain, and take a very long time to open ncpa.cpl (network control panel). If you manage to change the DNS servers on exchange to a non problematic DC, exchange will come up – or so it seems. Mail should flow and members of the domain that exchange is on will be able to login and get mail. Members of other (parent/child) domain who have mailboxes on the server may not be able to login to webmail though, and will get authentication errors on mobile devices.

Rebooting the servers or flushing DNS doesn’t resolve these odd issues. Closer investigation reveals that the servers (possibly AD) are on the public network, and other servers (possibly Exchange) are on the domain network with “(unauthenticated)” showing on the network connections.

Before you try removing and re-adding the vNic (this will take ages/forever for the server to recognise, don’t bother!) try this simple fix:

Open device manager (devmgmt.msc) and disable the network connection on the AD servers first. Re-enable it and it should drop back on to the domain profile. If not change the primary DNS of the server to a known good DC, ipconfig /flushdns and try again. Once you get all the DC’s in the domain (or at least the ones exchange is looking for for DNS), try the same with Exchange.

It should also then drop back onto the domain profile without “(unauthenticated)” after the NIC name. Once it has, restart the Exchange Active Directory Topology Service (check all the other exchange services restart OK), and after a little while mail should flow. Child/Parent domain users should be able to access their mailboxes too.