Science in policy, an engineer’s take

I went to a talk on Thursday evening hosted by CSaP. It was Mark Henderson talking about his new book, The Geek Manifesto. The talk was interesting and Mark had lots of good points to make about science in government policy (specifically, the lack of science in policy). His basic points boiled down to:

  1. Science should be better at promoting itself in government policy.
  2. Policy creation should be more scientific in its process.

At the end of the talk I tried to ask a question picking up on something that was raised during the discussion. Mark made the point that science should just be one of the factors that is invoked during policy making, the others being such things as voter wishes, human rights, ethics etc.

In the long and rambling question, I tried and failed to articulate the point that science should be the overarching framework for policy, and that these other factors are just parameters within that framework. Science should not be subservient to other points, but these other points need to be made to fit within the scientific policy framework. The question was accompanied by lots of shaking heads and a response that reiterated Mark’s original point. This got me thinking about how better explain what I am talking about.

The basic point is that to talk about science as being one of several factors in decision making is rather missing the point about science. It is the only reliable way we have of building knowledge about the world. For this reason, it makes perfect sense that any mechanism that attempts to define the world needs to do some from an exclusively scientific perspective. Any attempts to do otherwise are misguided at best and dishonest at worst. Science makes no claims about the nature of the knowledge it discovers, and it makes no claim about the tractability of the discovery process.

The core of it though, and I think the point where I got lost initially, is that policy making isn’t really the acquisition of knowledge (and hence science) at all; at its heart, it is engineering. What we have is a massively multivariate optimisation problem with some poorly defined cost function. It is this cost function that needs to be debated, and it encapsulates all those issues that were argued need to be considered in parallel to science.

The dimensions of the optimisation problem correspond to policy parameters – laws, taxation, incentives etc. The cost function then reflects how those laws are translated into real world consequences, giving a metric of “goodness”. This means that ethics, maintenance of democracy, not locking everyone up etc are necessarily encapsulated in the cost function. I would argue this cost function is something akin to a utilitarian type total happiness metric.

Of course, such a cost function is something that is inordinately difficult to both define and measure in the most general sense, but I expect the problem can be considered as a whole set of subdomains with reasonable separation – e.g. the economy, social welfare, foreign policy etc

Science then does have something to say about measuring the cost function. A whole discipline will arise around honestly quantifying the impact of a given policy change. Done properly, slowly but surely a body of knowledge will develop around how certain outputs can be achieved through policy change, as well as a body of skills and knowledge about how to measure and trial policy changes.

Any attempt to say, trample on human rights, is prevented by a huge negative impact to the cost function.

Of course, the real problem, and the essence I think of the political problem that underpins a lack of scientific scrutiny in policy making, is that there is a strong political will to not ever define the cost function carefully. Never properly defining what you’re trying to achieve is one sure way of stopping people telling you you haven’t achieved it.

This brings me to my final point – wouldn’t it be wonderful if politics became no longer about defining policy, but about defining that cost function. Civil servants could then go away and optimise policy to improve the cost function, drawing in new research and optimisation techniques as they become available.

I discussed this at length with my house mate and he made the point that the only cost function that matters is one’s own personal cost function. Whether politicians and civil servants can escape this trap would dictate whether such a utopia is possible.


About Henry Gomersall

I'm a engineer wanting to make things actually useful. I'm someone that wants to drive technology and ideas to be helpful for everyone. I'm someone that realises the disconnect between technology and utility and I hope to correct that...
This entry was posted in Engineering, Life. Bookmark the permalink.

5 Responses to Science in policy, an engineer’s take

  1. James Semple says:

    The trouble lies in the indeterminacy of the metric. Financial Derivative models were endorsed by Nobel prize winners and look what happened to them.

    Society is too unstable to model with any precision: policy vectors need constantly to be tested against social perceptions and shifted quickly and easily. A complex model may be more rigid and less responsive than a politician’s intuition.

    MSc Decision Science

    • I never suggested we try to model it. The evidence is derived from the real world. Perhaps models have a place, but the core test is whether it agrees with reality. There was plenty of evidence to suggest that financial models were broken.

      The metric (presumably what I referred to as a cost function) is definitively not indeterminate. If it’s indeterminate, it’s a bad metric. Is there any way in which stumbling around in the dark is better than honestly trying to achieve a predetermined outcome?

  2. James Semple says:

    You cannot escape from the model concept. The real world is impossibly complex and variable and, as such, beyond direct measurement. That is what statistics is for: you take samples of the significant parameters, build mathematical models with them and then test how well the model complies with reality. A continuous sampling-modelling-testing cycle will eventually produce a useful level of correlation, but you are always dealing with models, not reality. And it would cost a fortune to set up and maintain.

    Your housemate is no guide, here. True, everyone judges policy in personal terms, but everyone is different and their collected reactions have to be treated as just another statistical parameter to feed into the model.

    Determinacy is interesting. Even if you could define a generally acceptable social metric (the term cost is too locked into finance to be used without misunderstanding) there is no guarantee that you could determine it. The travelling salesman problem is easy to define; but beyond current solution in general terms.

    It seems you are a software engineer. You must know about many indeterminate algorithms. Didn’t Turing prove halting could be indeterminate ?

    • I don’t know what point you’re making. I’m not saying it’s a simple or even a tractable problem, just that there are currently no honest attempts to solve it. Politicians pulling policies out of their arses based on “intuition” (whatever that is) is not a good strategy, which is shown time and again to be the case. What I’m saying is, let’s be systematic about it and actually begin to accumulate knowledge in the area.

      Are you suggesting that humans have some magical insight into the problem that can’t be encapsulated within a rigorous framework? Perhaps that would be an interesting place to begin the research…

      I’m not sure what the halting problem has to do with anything.

  3. Guest says:

    This is exactly right!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s