May 22, 2018 by Siobhan Climer
Network downtime got you down? You aren’t alone.
According to the Ponemon Institute’s annual report, the average cost of unplanned outages is $8,850 per minute. PER MINUTE. True, the variance is significant, ranging from $137 to $17,244. But the takeaway is that no matter the size of your business, production outages affects the bottom line. What do you do when these failures arise? How can you prevent them – and the associated enormous loss in productivity and revenue – in the future? The good news is that network downtime is preventable.
What Is Network Downtime?
As a quick refresher, a network is the multitude of devices that connect with one another in an environment (either a wide-area network (WAN) or a local-area network (LAN)). This network includes the desktop computers, laptops, tablets, smartphones, televisions, and even smart appliances and electronics that communicate with each other.
Downtime is when a part of that network fails. It can be limited to a few systems, or it can encompass the entire environment. Either way, downtime refers to the time that the network (or any system) is unable to perform its job.
Internet Connectivity Increases Risk For Downtime
Everyone’s reliance on the internet for regular business transactions – from small to large size organizations – means the risk for business continuity failure is huge. And the risk floods all industries. Manufacturing. Finance. Healthcare. Education.
Contact centers, which rely on Voice over Internet Protocol (VoIP) and chat systems are a prime example of how internet connectivity transcends industry. In finance, every millisecond matters when making automated trades. Network and internet failures can halt production on the manufacturing floor, costing hundreds of thousands of dollars in lost revenue. These operational costs can be avoided, but first let’s understand how they happen.
The Causes Of Network Failure
Human error accounts for up to 80% of network failures, according to a recent Gartner study. The reason? The roles most associated with preventing downtime, such as industrial maintenance technicians, are harder to fill. Many of these skilled employees are retiring and highly trained replacements are hard to find. Couple that with the increased complexity of the environments, and the high percentage of human and process errors makes sense. Connected operations that highlight network infrastructure can help prevent downtime, but only when the right people with the right expertise are leveraged appropriately.
Equipment failures account for about 40% of unplanned outages, depending on what systems fall into this category. Poorly ventilated server rooms, UPS system failures, water or heat system breaks, and generator errors combine to form some of the most expensive costs in outages.
Cyber security threats, third-party power outages, and natural disasters all play a smaller role in causing network failures; however, these factors have increasingly larger costs associated with them, especially with the rise of DDoS and ransomware attacks that can bring a thriving business to the brink in one click.
How To Prevent Downtime
Prevention is the best protection against the causes of network downtime. Here’s how to plan ahead.
- Create a Disaster Recovery (DR) Plan
A DR plan is preparation for power outages, cyber attacks, and system failures. The risk is real, and a DR plan ensures your company and IT team can respond immediately. With the increased number of devices operating in WAN environments, a DR plan becomes paramount to protecting business continuity.
- Involve IT in Business Decisions
Relish the opportunity to engage with your own team. IT team members don’t just keep computers and phones working; they address big business challenges. Enabling IT to implement technology solutions and improve efficiency can only improve operations.
- Engage in Proactive Monitoring
The biggest step you can take to avoid network downtime is to monitor the health of your network. By setting up alerts and response protocols, you can be ready to get your network up and running in a matter of minutes, instead of hours.
- Invest in Managed Services
While IT is engaged in delivering on strategic business goals, a managed service provider offers support by monitoring systems. If the network goes down, the managed services team will be able to respond immediately, ensuring your team can be back at work without delay. By providing routine maintenance and monitoring, the IT consulting team takes on the tasks that would otherwise overload and create a backlog for the IT team. When your IT team is acting strategically to increase efficiency and support business goals they won’t be worried about downtime.
Calculate The Cost Of Downtime For Your Organization
Take the guesswork out of cost analysis. We put together a quick calculator tool so you can determine the cost per hour of downtime to your business.
Downtime Cost = Productivity Cost + Revenue Loss
[uCalc id=116180 name=Downtime Cost Calculator]
It’s important to remember that this is only an estimate. You may have additional costs, such as recovery, overtime, repair, and supply chain delays. In addition, there are qualitative costs, such as reputation and trust that may affect your future business growth.
How Do You Recover From Downtime? The Downtime Post Mortem
Downtime happens. Even if it’s only for a few minutes, it’s important to conduct a downtime post mortem so your team can avoid future outages. A managed service provider can help you perform this essential task and create a plan going forward to eliminate network downtime.
Step 1: Collect information
Think like an investigator. The electronic information gathered from your system logs, employees, and internet services will likely contain valuable insight, as well as timestamps.
Step 2: Identify the cause
Earlier in this article, we identified the main causes of network downtime: human error, equipment failure, cyberattacks, natural disasters, and third-party outages. The response – and implementation of your DR plan – will likely depend on the root cause.
Step 3: Identify the impact
It may seem easy, but determining which systems were affected will help your business rebound. Who was affected? It is important to assess the impact immediately as there may be steps your employees need to take to ensure their individual responsibilities are met.
Step 4: Evaluate the response
To prevent future failures, you’ll need to identify how the event unfolded and how employees responded. As part of your DR plan, make sure staff document the steps they take during downtime. During an outage, panic often ensues. It can be easy to try lots of “fixes” and have no data on what actually worked. Remove the guesswork by making documentation a part of your response protocols.
Step 5: A.R.I.T.
Assess. Respond. Implement. Test. These are the important follow-up steps to take once the initial evaluation has been completed.
Assess
Did existing safeguards fail or were they ignored? Determining whether current safeguards are enough (or if they even exist) is a vital first step. You may need to implement new safeguards, but it is more likely that current protocols were improperly followed. It can be helpful to implement additional monitoring – even a daily on-site walk-through – to ensure there aren’t any blinking lights. In addition, remote monitoring can help free up your IT staff for other strategic business needs.
Respond
If it was human error that caused the outage, it is important to identify how you will respond. Do staff need additional training? Would a peer-approval system work, where colleagues verify command codes before entering? Might a new change management system be needed?
It can be tempting to want to do everything after an outage; be thoughtful and consider the causes and logical steps to take. Trying to throw on too many band-aids might actually make things worse. Target the root cause and respond with a single strategy.
Implement
Whatever strategic response you determine is needed, implement it. Make sure all major stakeholders are engaged and involved in the process. Be sure to document the improvements. Create a SOP or another protocol administration document so your staff know what to do going forward.
Test
Test. Test. Test. You might wish to wash your hands of the problem once your systems are up and running, but it is vital that you set up maintenance windows to test your changes. By creating the same situation in a testing environment, you should be able to determine whether your alert system is working and the appropriate staff are notified.
Don’t Let Downtime Get You Down
At Mindsight, we can help you assess your current network infrastructure and determine the solutions your individual business needs going forward. Need a roadmap for the future? Network downtime and internet outages shouldn’t put your business at risk. Get in touch with us today to create a path forward that ensures your business’ success.
Like what you read?
About Mindsight
Mindsight, a Chicago IT services provider, is an extension of your team. Our culture is built on transparency and trust, and our team is made up of extraordinary people – the kinds of people you would hire. We have one of the largest expert-level engineering teams delivering the full spectrum of IT services and solutions, from cloud to infrastructure, collaboration to contact center. Our highly-certified engineers and process-oriented excellence have certainly been key to our success. But what really sets us apart is our straightforward and honest approach to every conversation, whether it is for an emerging business or global enterprise. Our customers rely on our thought leadership, responsiveness, and dedication to solving their toughest technology challenges.
Contact us at GoMindsight.com.
About The Author
Siobhan Climer, Science and Technology Writer for Mindsight, writes about technology trends in education, healthcare, and business. She previously taught STEM programs in elementary classrooms and museums, and writes extensively about cybersecurity, disaster recovery, cloud services, backups, data storage, network infrastructure, and the contact center. When she’s not writing tech, she’s writing fantasy, gardening, and exploring the world with her twin two-year old daughters. Find her on twitter @techtalksio.
How Emerging And Midmarket Companies Leverage Managed Services