First, the problem. As I have indicated in my previous article (Private Cloud Architecture) I have 4 Cisco Small Business SG 200-08 Switches that help make the network backbone of my Private Cloud. One issue I have had from day 1 of this setup is the inability to use NFS or RPC based technology. After getting frustrated at both lack of available verbosity surrounding why NFS was timing out, as well as my lack of knowledge surrounding NFS and RPC technology in general (I know how to do a simple NFS mount server and client, but nothing below that in the technology stack), I shelved the initiative to use NFS as well as the shared storage problem between Nova Compute Nodes. Fast Forward almost a year, I was finally at the point in my infrastructure that I was going to need to try it again. I tried many things and was given the suggestion, verify what you know in a known environment and slowly migrate it into the problem environment.
My OpenStack Nodes are all running CentOS 7. So I quickly took a CentOS 7 Vagrant Box file (generated via Packer and my own manifest files) and made a multi-box VagrantFile with shared networking between both boxes. One box in the file would be my NFS Server and the other my NFS Client. I started them up, installed NFS software and was able to mount the share, no problems. So far so good, my knowledge of simple NFS setup still held true. So now I needed to attempt to connect to my NFS share being hosted from my FreeNAS box.
I made the mount attempt and it didn’t hang as it had done before, instead it simply said connection refused. What, why? After some poking around in the FreeNAS settings I decided to attempt enabling the option “Allow non-root mount”. BINGO, mount successful from the Vagrant NFS client box. Let’s attempt to mount from the OpenStack Nova nodes.
Not only did it not work, it worked in the same way as before. The command would hang for several minutes and then return with a connection timed out message. The frustration building in me was palpable. I attempted an rpcinfo to the NFS server, and it hanged, but after several minutes it would finally return something. After much more testing, and frustration building, I finally came across this blog that has become my saving grace.
This explained my problem down to the exact firmware version, model number, everything. So my path forward should be simple, upgrade the firmware and everything should work. Upgrading the firmware would turn out to be an equally frustrating experience.
These particular Cisco switches have two ways to update the firmware. Either via TFTP or HTTP. I first attempted to do the HTTP upload method. However every time I clicked on Apply to upload and start the upgrade, it would pop up a new window with only an OK button, click it and nothing would happen. I then attempted to upload via TFTP, which requires an FTP server to host the files first. Pointing it at my FreeNAS FTP didn’t work, and after some additional problem browsing, I came across a suggestion to use TFTP32 portable version on my Windows machine. I set the program to host my downloads folder where the firmware resided. Entered the IP of my laptop, and the file name, and finally an indication from the Web UI that something was working.
After the update was applied I needed to reboot the switch to move to the new firmware. In the Web UI I navigated to reboot and told it to reboot. Again a pop up prompt with OK and Cancel appeared, I click OK and nothing. Based on my searches the prompt was supposed have more information in it. At this point I have determined that the UI in its current form is too old to handle current browsers for specific types of prompts. I can edit most settings that only calls for a confirmation prompt like setting VLANs to a port, but a prompt that wants to have more information passed along fails to work. Additionally there is no other way to manage the switch outside of the Web UI. In order to reset the switch I was forced to physically hit the reset button. Not the worst of things, but I was also taking the chance that the reset button tap would either reboot without the firmware upgrade, or I brick the device. There is no indication that I could find that the reboot the UI does was the same as the reboot of the reset button. I took a chance and hit the button. Not only did the device come back not bricked, but the firmware did indeed upgrade.
As for the original problem, I can confirm that after rebooting the switch and the new firmware was applied that the NFS problem has been fixed and can mount NFS mounts with no issue or delay.
I must say, I am not impressed with the Cisco interface to a managed switch. It is one thing to have to upgrade the firmware in general. In my book there is no difference between Cisco and any other company when a bug needs to be fixed. But the UI is atrocious and fails to satisfy the most basic or requirements for this type of switch, especially since it is the only way to interact with the settings. While I’m sure it may be handled better in the more expensive models with CLI enabled, I have no reason to buy those models with my current experience. So if Cisco is reading this, my suggestion for handling interactions with your switch is either, 1) always allow CLI on every product, especially in the event your Web UI may become too old for modern browsers (give it time, it will), 2) do not use pop ups for additional prompts, or if you do, keep them inside the same window/interface, 3) talk to Ubiquity, they allow CLI in the Web UI directly, as well as pop ups without the new browser window.