When we look back at past decisions, there were times we did not end with the outcomes we expected. But that does not mean the decision itself was wrong - it is possible that our process was sound, yet the final outcome was affected by random factors and ended not as we expected.
This week, I explore the contrast of process and outcome in the context of public opinion and foreign policy. I use data from a survey that my undergrads completed, and replicate a recent study that asks: do members of the public care whether governments ignore the norm of transparency and implement covert operations in foreign policy? Or is the only thing that matters is the outcome?
The Assignment
The survey focused on evaluating public opinion regarding the use of covert actions in international affairs, compared to using non-covert policy options. I replicate several elements from three experiments conducted by Myrick (2020) . The first scenario describes a dictator in the Middle-East who engages in constant repression of the population, and rebel forces attempt to overthrow the government. The US government contemplates sending a small military force to help the rebels.
There main factor in this scenario is the transparency norm: The intervention is either known to the public in advance, or it is kept secret from the public. A third version of this factor states that despite no value for keeping the action secret (i.e. experts suggest it provides no benefits), the government does not inform the public of the intervention. I test the extent of respondents’ approval of the decision by the government. In the figure below, I plot the proportion of approval and disapproval for all three conditions.
Overall, respondents value transparency as they offer higher degree of approval for an overt action, compared to either version of the covert action. Yet, both secret alternatives still garner substantial proportion of approval. This may hint at lower importance of transparency than previously assumed.
In the second part of scenario 1, I provide information about the outcomes of the intervention. Half of the respondents learn of successful outcome, and the other half read that the intervention failed to topple the dictator. Following this information, respondents are asked again to state their degree of approval for the government decision. The plot below unpacks these responses along two dimensions – the three values of transparency (like figure 1), as well as whether the intervention was successful or not.
The results of this analysis raise more doubts about the importance of transparency, and highlight the importance of outcomes. For unsuccessful intervention, the results are fairly similar to the previous test in the sense that an overt action is the least criticized compared to both versions of the covert operation (so being transparent reduced the extent of public disapproval). Yet, once the intervention is described as successful, the level of transparency loses most of its `appeal’. Whether the action is overt or covert (either version), it garners substantial public approval. This result highlights the dominance of outcomes compared to the process in public evaluation of national security decisions.
The survey consists of a second scenario. In this case, the US government faced a civil-war in Africa, one that may threaten the stability of American allies. Thus, the government contemplates sending support in the form of either financial means or a small military force. The other factor was again the transparency norm – both policy options were either known to the public or kept in secret (without the third version from the first scenario). I measured the degree of approval for the government decision, results are displayed in the figure below.
The figure shows the proportions of approval/disapproval for the decision. As figure 1 demonstrated, when asked about the intervention before reading any information about the outcomes, respondents care about transparency. The overt action receives high approval ratings (over 70%) regardless of the policy type (financial or military support). The covert option is less appealing for the public and support is lower. An interesting finding here is that respondents do not distinguish between sending money or soldiers, and view both in a similar fashion. This result adds to the findings above: without information about outcomes, the process does matter for individuals.
Finally, I added a third scenario to this replication exercise. Again, this is the dictator in the Middle-East scenario, and the US government contemplate sending military support to help the rebels. Two factors are in-play: (1) whether the intervention is known or kept in secret (as in scenario 2); (2) some respondents read information that many in the American public oppose any US involvement in this region, and some do not read such information at all. I collect responses to the approval question (again, before providing any information about the outcomes). Results are in figure below.
The responses are described in two categories: the transparency and public opposition factors. In terms of transparency, the public provides higher approval ratings for an action that citizens are aware of. Approval figures are also higher when no information is provided regarding any public opposition to US involvement. Covert action has lower approval ratings but still rather high (59%) when respondents have no additional information. When they read that the public opposes US involvement, and that the government still decided to engage in secret action, approval drops substantially (approximately 33%).
What can we learn from all these survey responses? Overall, transparency (as part of the decision process) matters, but not always. First, I included a general survey item that asks how important is the issue of government begin transparent with the public, and more than 67% of respondents rate it as fairly high. Second, in all three scenarios, whenever I asked about approval without offering information about outcomes, the overt intervention received higher approval compared to a covert one.
At the same time, the most powerful factor that alters the results is when respondents learn of the intervention’s outcomes. A successful action leads to high approval, whether it was known in advance or not (I ran the outcomes factor for all three scenarios and the results remain consistent, see more in the expanded analysis document).
The bottom line of this analysis is that when public members evaluate foreign policy actions, outcome outweigh process in their minds, and they are willing to accept actions that are not transparent (and potentially harm some democratic values and norms) in exchange for successful results. These findings raise some concerns about the dominance of democratic values in citizens’ minds, as well the process of government decision-making, especially when it comes to foreign interventions.
For the analysis of this survey, I compiled a more detailed document with additional tests of the survey data. If you are interested in the R code and associated documents that describe the experiment procedure and analysis, check out my Github page.