You Haven't Automated Anything Until Someone Else Can Operate It

Automation has a way of looking finished too early.

The script works. The proof of concept passes. The diagram is cleaner than the old process. The repeated task is gone, at least in the lab.

Then the people who will inherit the system look at it and say, in effect: we understand the old way well enough to own it. We are not sure we understand this new abstraction well enough to bet production on it.

That can be frustrating if you are the architect. It can also be rational.

The Production Context

The environment was a large enterprise remote application and desktop platform supporting a mission-critical dealer-facing equipment diagnostics service. The service carried significant monthly business value, so availability mattered in the practical, pager-driven sense of the word.

The application delivery architecture used Citrix published applications, redundant ADC and load-balancing infrastructure, global server load balancing, redundant web tiers, and a pool of roughly 30 interchangeable application delivery servers, or VDAs. The platform used a PVS-style single-image management model across QA, staging, and production environments.

Application updates were frequent. Weekly was normal. Urgent changes could happen faster.

The application team would update a maintenance VM that represented the master application image. The platform team would then seal or finalize the image and promote it through the delivery process. Once promoted, the new image backed a pool of healthy application servers that could receive user sessions through the load-balanced Citrix path.

There was also an external synthetic testing service. It exercised the published application through roughly the same externally exposed path real users took: DNS and GSLB, ADC and load balancing, Citrix access, then the application delivery pool.

The synthetic testing tool was capable, but rigid. Because a user or test session could land on any healthy application server, the testing process depended on configuration data that understood the relevant server identities. If that data was stale or copied incorrectly, monitoring could report a failure even when the platform architecture itself was behaving correctly.

The weak point was not the redundant platform. It was a small recurring configuration-maintenance task around the monitoring system’s expectations.

The Manual Model

After each image deployment, application engineers would determine the current application or server information and propagate a configuration or hostname list across the application delivery pool.

The process worked. It was visible. It was easy to explain:

Update the image.
Deploy the image.
Copy the expected configuration to the application servers.
Let synthetic testing confirm the published application path.

It was also repetitive. With roughly 30 VDAs in the pool, the process created opportunities for missed copies, stale data, and false-positive monitoring noise.

The Proposed Improvement

I designed and proof-of-concepted a small configuration improvement.

Instead of copying the same server-list or configuration data across the VDA pool after each image deployment, the managed image would contain a permanent symbolic-link pattern. The application-visible configuration location would point to centrally maintained data on a shared SMB location.

The link setup would be done once in the image. After that, every server created from the image would reference the appropriate shared configuration source automatically.

Conceptually:

Update the image.
Deploy the image.
Application servers resolve configuration through the pre-established link.
Let synthetic testing confirm the published application path.

The design was technically straightforward. It was tested in a non-production environment, demonstrated successfully, and documented. It reduced recurring work. It reduced the chance that one server in the pool would be missed.

From my side of the table, it looked like an obvious improvement.

Why It Was Not Adopted

The likely issue was not that the mechanism failed.

The issue was that it introduced an abstraction the application team did not feel confident owning. The old process was repetitive and imperfect, but it was easy to see: copy this configuration to these servers. If something looked wrong, they knew where to look.

The proposed model moved the work into symbolic links, shared configuration, image lifecycle behavior, SMB availability, indirect file resolution, Citrix session placement, server identity assumptions, and synthetic testing behavior.

To me, that compressed complexity. To the operators, it may have relocated complexity into a less visible place.

Complexity is not measured only by the person who designs the mechanism. It is measured by the people who must diagnose it when the mechanism is no longer new, the architect is unavailable, and production is noisy.

Technical Simplicity Is Not Operational Simplicity

The solution reduced manual effort, but it increased the amount of abstraction the owning team had to understand.

That is the tradeoff automation reviews often undercount. Traditional automation ROI looks at labor saved, consistency improved, errors removed, and speed gained. Architecture has to ask a few more questions.

What new dependency did we introduce? Who understands the failure modes? What happens if the SMB path is unavailable? How does a broken symbolic link present to the application? How does the team prove whether a failure lives in Citrix, the image, the link, the share, or the synthetic testing configuration?

A simpler process is not always the process with the fewest manual steps. Sometimes the simpler process is the one the owning team can explain under pressure.

Adoption Is Part of Architecture

There is a tempting mental model for architecture work:

Current state -> Design -> PoC -> Testing -> Production

That model is missing the people who inherit the result.

The better model is closer to:

Current state -> Design -> PoC -> Testing -> Team understanding -> Operational confidence -> Adoption -> Production

The point is not to create more approval gates. The point is that a technically successful design that nobody adopts has not delivered its intended value.

Documentation helps, but documentation is not the same as knowledge transfer. A runbook can tell someone what to do. Education gives them enough confidence to decide what to do when the runbook is incomplete.

For this design, the missing deliverable probably was not another diagram. It was shared ownership.

Automating the Workaround

There was another lesson hiding in the problem.

The recurring task existed partly because the synthetic testing service had rigid expectations inside a load-balanced application delivery architecture. The infrastructure was doing the right thing by allowing sessions to land on any healthy VDA. The monitoring system introduced a configuration burden because it needed a particular view of server identity.

That does not mean the workaround was illegitimate. Monitoring matters. False positives matter. Sometimes you really do need to automate the supporting process around another system’s limitation.

But the distinction should be explicit.

Before automating a repetitive task, ask whether you are automating a genuine system requirement or automating around the rigidity of another tool. The answer may not change the immediate work, but it changes how you describe the architecture and where you look for a better long-term fix.

What I Would Do Differently

Looking back, I would bring the application engineers further into the mechanism before asking them to accept it.

I would design more of it with them, not only for them. I would let them implement part of the proof of concept. I would deliberately break the link in a test environment and have them diagnose it. I would demonstrate SMB share loss, stale configuration, rollback, and the difference between an application failure, a Citrix placement issue, and a synthetic testing failure.

Most importantly, I would make sure they could prove to themselves that they could support it.

The goal should have been for the team to stop thinking of it as the architect’s solution and start thinking of it as their solution. Until that happened, the automation was technically real but architecturally incomplete.

The Closing Opinion

An architecture that cannot survive the departure of its architect is not finished.

Automation is not complete when the PoC passes or the manual step disappears from the diagram. It is complete when the people responsible for the system understand it, trust it, can diagnose it, and can operate it without the person who designed it standing beside them.