‘IT Stories from the Road’ is a series of first-person stories told by IT professionals. If you’d like to share a story, email us at [email protected]!
---
The Cat’s Out of The Bag
As part of my previous role as Client Strategist at a large financial services firm in Hartford, our IT leadership frequently brought issues and escalations to our engineering team that were of a large concern, but perhaps not a specific outage needing immediate triage.
During one project I was briefed by a senior leader visiting our large West Coast office on Market Street in San Francisco. This office was equal in terms of importance, and stress, to our New York City office.
Our CEO, CFO, and business SVP, stopped into the office to get a business briefing, and one of the topics they inquired about was the state of IT. As you might imagine, they got an earful.
The business users, who were about 3,000 miles away from our East Coast data center, described scenarios of painfully slow response times, applications that would unexpectedly crash, slow logins, and other sporadic symptoms that were all impacting their productivity, user satisfaction, and impacting the experience of our external customers.
So, when the senior executives asked them about IT, they didn’t hold back.
In prior years, SWAT teams had been put together to analyze the company’s network, trace application activity, upgrade network bandwidth, and troubleshoot with trial-and-error changes without any real benefit. In general, our teams always agreed that it must be due to network latency, which was not easily solved – You can’t do much to change the speed of light.
Upon their return to Hartford, their first call was to our CTO where they described in excruciating detail the pain that these folks were experiencing, and then laid down the challenge to fix it, and fix it fast.
Challenge Accepted
After some detailed meetings with the business users to make sure we had the right problem and impact statements, we dispatched one of our senior engineers in the area to the site to collect more information.
Fortunately, we were just enabling our Nexthink infrastructure and were quickly able to roll out the collector and begin gathering Nexthink’s unique experience data.
Rather than needing to coordinate with unproven service areas like our Network, Server, Application, and AD teams, or having to blame an unproven cause like network latency, we were able to quickly pinpoint the main culprits:
Memory utilization was off the charts;
Improperly configured Active Directory Sites and Services triggered in slow logins;
Slow spindle hard drives versus solid state drives; and
Users that were remote controlling into office PC’s, with round trip network traffic crossing the country four times and routing thru Hartford!
Without this data, we would have been on another needle-in-a-haystack hunt just like other SWAT teams in the past.
Saving The Day
We quickly piloted memory upgrades, reconfigured AD for proper network routing, upgraded laptops with solid state drives, and gave work at home staff laptops rather than having them remote control their in-office desktops.
With these changes piloted, we received immediate validation from the business, coupled with live data from Nexthink that performance had vastly improved. We were then able to invest fully at not only this site, but 80+ sites around the country.
The right data was what allowed us to successfully justify a significant spend in RAM hardware and installation labor, along with thousands of early-refresh laptops, and return customer satisfaction and productivity back to the business.
Data turned into actionable information, available on-demand and across the entire enterprise, is the key to knowing your business and keeping it running efficiently and cost effectively. And you don’t always need to blame the network!
