People say, “S**t happens.” I say, “Firmware updates happen.” Right?
You’re at a client’s home sitting in front of some device that is cheerfully bearing the ominous words: “Firmware Update Available.” Do you proceed? Risk it? Wait? Maybe try it out somewhere safer, like back at your shop? Seriously, what harm could be done? I mean, this update is official! The manufacturer is telling you to do it. But what about all those firmware updates that brick things? Ooooooh. What should you do?
A recent firmware update I enabled on a 4K set fixed a 4:2:2 chroma bug interface with the upstream switcher, but broke the black level linearity. I wish I had known of the liability ahead of time. You need a plan to handle this type of situation, or else you risk awkward moments of panic and angry calls from clients victimized by bad firmware updates. I’ve witnessed a lot of different ways integrators handle updates, and here are some of the better ideas. These are by no means step-by-step instructions, but hopefully they will point you in the right direction.
1. You need a point person who’s responsible for tracking firmware updates for all the gear you install, who briefs your field crews when it’s safe to run updates, and who takes questions from the field when an update catches you by surprise.
2. Your firmware person needs to spend a good amount of time on the forums. AVS Forum for sure, but many products have their own forums where users and company representatives interact. We’ve reached a point where enthusiasts are often the most effective beta testers. Obviously, you shouldn’t believe everything you read, but there are some tech wizards hanging out on forums who have time to flyspeck products in ways you don’t. Find ’em and follow ’em. If an update is toxic, chances are they’ll find it first. They won’t have solutions, but at least you’ll know to push the pause button until stuff gets sorted out.
3. Be upfront with your clients that firmware updates are a fact of life. Use it to your advantage to promote your business over your competition. Build firmware update services into your contract. Call them out when you’re talking to clients. They want you, as their integrator, to handle all the details of getting them the best picture and sound with total reliability. They don’t just want you to install gear and walk away. Give them the peace of mind that if they sign on with you, they don’t have to worry about their system being offline because a firmware update knocked them out and you’re nowhere to be found.
4. Sometimes a manufacturer compels you to update, even when everything is working, and you don’t need any of the new features. The device won’t connect to a required service without the update. If, however, the device continues to function, and all you have is the obnoxious bug reminding you to update, then you may be able to get rid of it by blocking the update server in the router.
5. Your guys in the field should be able to quickly assess if an update has caused a critical failure or degradation in performance. For AV systems, I recommend carrying basic video test patterns (PLUGE, steps/ramps, color bars, luma-chroma resolution) and audio test files (narrowband/wideband channel noise, tone sweeps) on a USB key chain. Plug them into anything with a media player, and you’ve got a test pattern/noise generator. If a video device updates, for example, run the basic patterns through it to make sure your choices for settings are still valid. You don’t have to do a full calibration every time. Just look for obvious gaffes.
6. If you think you’ve found a problem, please report it. Have your firmware guru on the phone to your product rep right away. Report it to the manufacturer on their forum (anonymously, if you must). Report it on AVS Forum. Report it to someone in the press like me. (Actually, report it to my technical editor, Chase, because he is our firmware guy!) The faster these issues are known, and the more people report them, the faster manufacturers will jump on them. The squeaky wheel definitely gets the oil. Get squeaking!
The new normal is that, at any given time, some aspects of a system will be operating at less than maximum performance levels. There may be degradation in the audio chain. Or the video chain. Or the network. Or the content provider(s). No matter how hard you work or how good you are, systems are never going to be 100 percent in all respects at the same time. Everyone, including clients, must accept it. We let the camel’s nose in the tent when we allowed manufacturers to continue working on their products after they’re in our homes. We can’t very well turn around and complain when the camel spits at us occasionally!
