Access Official Technical PDF

Share Summary:LinkedInEmail

Human responsibility in automation is difficult to assess when people and intelligent systems share information processing and control. An operator may be formally accountable for a decision while relying on information filtered by software, having limited authority to intervene, or exerting little influence over the final outcome. The Responsibility Quantification (ResQu) model offers a way to examine this gap by estimating a person’s comparative causal contribution to system outputs.

Developed as a theoretical engineering model, ResQu applies information theory to the interactions among a human, an automated module and the operating environment. Its potential relevance extends beyond the high-stakes autonomous weapon systems used to illustrate the model: the underlying question applies wherever automated functions shape decisions and actions, including industrial operations and control systems.

Why assigned duties do not tell the whole story

In human-automation interaction, role responsibility describes the duties assigned to a person. Causal responsibility concerns how much that person’s actions actually contribute to an outcome. The two can diverge. An operator might be expected to supervise a process but lack the information, authority or time needed to change its course.

Advanced automation can widen this gap. A system may collect and analyze information, recommend an action, or implement a decision. When processes are difficult to interpret, or outcomes depend on uncertain conditions, an operator may not be able to predict or control the system as their formal role implies. Responsibility is also affected by the automation’s design and reliability, human capabilities, and features of the environment.

For industrial decision-makers, this distinction matters when assessing operator oversight, allocating functions between people and machines, and evaluating what a control-room role means in practice. Simply assigning a person responsibility for monitoring a system does not establish how much influence that person has over its results.

How the ResQu model estimates human contribution

ResQu represents a combined human-automation system as a flow of information through four functions: information acquisition, information analysis, action selection and action implementation. A function may be performed by the human, the automated module, or through interaction between them. The model describes these relationships using variables for environmental information, human and automation assessments, selected actions and the system’s implemented output.

It then uses Shannon entropy, a measure of uncertainty, to estimate the human’s unique comparative contribution to the output. In simplified terms, the model asks how much uncertainty about the system’s output remains when the automation’s information and processing are taken into account, relative to the output’s total uncertainty. A higher value indicates a greater unique human contribution to the output distribution; a lower value indicates that the automation variables account for more of it.

This is not a direct measure of blame, legal liability, moral responsibility or operator competence. It is a quantitative perspective on causal contribution within a specified system model. The measure is useful only when its assumptions and the system boundaries are made explicit.

Applying the framework to system design

To use ResQu for a real system, practitioners must define the information flow and estimate the probability distributions of the relevant variables. Those estimates may come from known system properties or empirical observations over time. For an industrial application, the analysis would need to reflect the actual arrangement: what sensors and software observe, what information reaches an operator, how recommendations are presented, who selects an action, and what determines the implemented response.

The model can support comparisons between design or deployment alternatives. For example, a team could examine how changing the allocation of an information-analysis or action-selection function affects the estimated human contribution to system outputs. The point is not to assume that more human involvement automatically means more causal influence. A person may be “in the loop” yet have little ability to evaluate the information or alter the outcome.

ResQu can therefore complement design reviews and policy discussions about meaningful human control. It provides a structured way to ask whether the operator has a consequential role, rather than relying only on labels such as supervisory control or human approval. It does not, by itself, establish whether a particular level of control is appropriate.

Interpret results with care

The initial model assumes that the combined human-automation system is stationary and ergodic: broadly, that the probability relationships being analyzed remain stable and can be estimated from observations. Real operations may not meet these assumptions, and the available data may be insufficient to estimate the required multivariate probabilities accurately.

Other simplifications may also matter. In the illustrative application, human and automation characteristics and their information sources are represented using specific assumptions. A practical analysis should test how its results change when inputs or assumptions change. Sensitivity analysis can reveal whether the estimated level of human contribution is robust or depends heavily on uncertain probabilities.

Where reliable numerical estimates are not possible, building the information-flow model can still help teams identify which functions, information routes and decision points shape human influence. The resulting measure should be treated cautiously when the system changes over time or cannot be observed sufficiently.

Practical implications for automation teams

ResQu offers engineers, integrators and decision-makers a method for examining a consequential design question: who, or what, actually determines the system output? Its information-theory approach can help compare configurations and clarify the difference between an operator’s assigned duties and their causal influence. It should be used as an additional analytical tool, not as a substitute for engineering judgment or broader responsibility assessments.

In practice, teams should map the human and automated functions, document the data and assumptions behind probability estimates, and test how sensitive results are to those assumptions. These steps can make discussions about oversight and function allocation more concrete—and help ensure that claims of human control are examined against the system’s actual information flow and capacity for intervention.

Access Official Technical PDF

About The Author