Repeatable Experimentation in Industry

This article is written in response to David J. Bland’s “5 Steps To Make Experimentation a Repeatable Process,” posted on July 2, 2019. https://medium.com/precoil/5-steps-to-make-experimentation-a-repeatable-process-8a664690ea96

The formation of hypotheses is a practice that trained and self-taught researchers alike need to master in order to attain actionable insights and results no matter the industry they’re focused in; however, these hypotheses can take many forms, especially when comparing a hypothesis in biology that may tackle the simultaneous functions of sodium-potassium pump efficiency and nutrition to a hypothesis in data science that aims to increase analytical efficiency of machine learning models. Though each hypothesis is likely going to be multi-part, one serves as a vessel for increasing concrete understanding of an established process, and the other tackles an evaluative process and determines whether the existing method should be improved upon or canned entirely.

Despite the inherent differences in the goals of the experiments, the 5 recursive steps outlined by Bland illustrate the constant need for reevaluation upon the collection of new information, as any scientist would promote.

Why do so many organizations stop investigating once a proto-solution is found?

Much of the time, the reasons for halted discovery and experimentation fall into two categories: budgetary and operational restrictions.

Budgetary restrictions can halt experimentation by deeming any one of the 5 steps as a drain on resources that are deemed better used elsewhere in the company. For example, if a data analyst is tasked with managing both the statistical evaluation of efficiency data for machine learning models and analyzing discrepant data on behalf of a client, management will likely always defer to the latter task due to its immediate monetary impact on the company’s success and because continual project development in itself will always present evolving costs based on project stage and slated improvements.

Operational restrictions affect the five step process most often when the resources needed to conduct the desired experimentation fall into different departmental silos, and require extensive interdepartmental coordination to properly deploy. Not only must the experiment be scheduled between multiple teams, it will likely also take a backseat to day to day work depending on the size of the firm, reducing the efficiency and efficacy of the experimentation due to losses in momentum.

When planning for experimentation, it is best to determine all stakeholders, timing plans, and budgetary requirements ahead of time, as to create an effective and cooperative schedule that will not only ensure the execution of the experimentation, but also prevent it from losing momentum throughout the process.