Engineering and IT: Bridging the Gap

Let’s face it, IT departments typically aren’t the most loved members of their organizations. While they perform a vital role, other departments consistently view IT as a roadblock or bottleneck. While this is an unfair characterization of IT departments, the perception persists. Of course, your level of interaction, and thus frustration, with your IT group depends on how much you leverage information technology in doing your job. So it should be no surprise that the group of people that have the most friction with the IT department are the engineers.
An engineering manager I recently worked with came to our company after several years of trying to improve the IT environment for his team. He was frustrated by his organizational constraints and was taking action on his own to find a better way. The underlying frustration for this manager was that his IT group required a solution that worked for the entire company. While what they provided worked for a vast majority of their design engineers, it was falling painfully short for this manager’s simulation group and he needed a solution. His group needed better, more available HPC resources, not something IT can easily, quickly or cheaply add!
Most certainly, the relationship between IT managers and engineers is a match destined for strife. It isn’t hard to see why. Engineers create, design and test. If you give an engineer better resources (or more resources) to accomplish these tasks, you can bet those tools will be utilized to their utmost to create the best end product. IT professionals are cut from the same cloth in that they try to deliver the best products for their users with the resources they have. But the resource that constrains their ability to provide is a major pain point and source of argument for countless relationships: The Budget. Their toolkit is a checkbook and the available balance does not expand easily. Within the constraints of their budget, IT professionals have to create the best solution for everyone in the organization. Additionally, IT folk must consider things that aren’t primary concerns of the engineer…infrastructure security, widespread usability, and product compatibility, to name a few.
What compounds this problem between IT and engineering is the fact that the tools engineers use to solve problems, both hardware and software, are constantly evolving and engineers are typically on the leading edge of advances in technology. While Moore’s law may indeed be dead (or in the process of dying), advances in artificial intelligence, deep learning, and the rise of the Internet of Things are sure to keep engineers continually pushing the technology envelope and driving their need for IT resources ever forward.
When you look at the IT and engineering relationship, what you find is two groups of very capable, hard-working professionals whose paths are inextricably divergent. IT and engineering operate under different paradigms with different objectives that fail to co-exist happily, ensuring there will always be friction.
So here we are, two groups of individuals that seem destined for existence of surface platitudes with an underlying tension that can’t be avoided. But what if it could?
What if the system was redesigned so that the tension was removed and each group was empowered within understandable constraints?
Enter the cloud!
With the arrival of cloud computing, IT resources have been democratized. What was once available only by engaging IT professionals within an organization and going through a complicated approval and integration process is now readily accessible with an internet connection. It is no longer necessary for the IT group to take a limited budget and make difficult decisions about which hardware and software are best for the entire group. Each group can now pick and choose the hardware and software solutions that best fit their needs.
So how does this affect the relationship between IT and engineering?
Before the cloud, IT managers were seen by engineers as the roadblock to getting things done. As engineers move to the cloud, IT managers are no longer controlling their access to resources. IT only needs to approve the integration of that cloud resource. This mainly involves ensuring security is sound, but as the cloud matures, any reputable cloud provider is built on a solid foundation of security and can pass most audits with relative ease.
But what about that source of friction in the previous relationship, the budget IT was working within to provide for all? The budget is still there, but the responsibility for managing it has shifted. Instead of the IT specialist painstakingly analyzing significant capital investments, cloud providers enable the costs to be shifted to an operating expense controlled by the engineers.
A shift in the budget requires engineering managers to change the way they approach a problem. Computational engineering has been trained to be an expanding gas, filling every inch of the space given. With the cloud, and theoretically infinite resources, the engineer must now consider the cost of the project when designing their workloads.
Looking back at the engineering manager I mentioned at the beginning of this post, we were able to present a solution for HPC resources that his IT department was more than happy to accommodate. The solution allowed their engineering and IT to work together for a solution that satisfied a small subset of users within the organization while shifting budgeting control for this type of work to the engineers themselves. IT was relieved to be able to alleviate a point of tension while the engineering group found an affordable alternative to in-house hardware.
This new relationship the cloud affords will take some getting used to. IT managers must learn to be comfortable with the security and integrity of the cloud. Engineers must learn to budget for resources when designing their projects. But as each department learns to operate within the cloud environment, the only thing holding back innovation is the operational budget. If operational funding can’t be made available for an engineer, it is likely due to higher priority needs within their department or organization, and this is much easier for the engineer to understand and deal with.
This shift is already occurring at a drastically increased rate. Worldwide spending on public cloud services is growing at a near 20% compound annual growth rate. The question is no longer are you going to use the cloud, but how. As the movement continues, innovative growth no longer gets bogged down in a complex IT process. It can now soar in the cloud…budget permitting.