Which of the following method is considered to be a more practical way of scanning?
Show
Summary of Key Findings
A range of tools can be used to think about future risks and opportunities in a structured manner. As noted by Daniel Flynn from the Office of the Director of National Intelligence, these tools “are for future planning in a world where the future cannot be known.”1 Such tools are commonly used to help shape policy so that entities (such as governments or organizations) are more resilient and better placed to take effective action (IRM, 2018). As explained by the UK Cabinet Office:
Horizon scanning is therefore not about predicting the future, but focused on the early detection of weak signals as indicators of potential change. The terminology around relevant tools, techniques, and processes involved in horizon scanning has yet to be standardized, which can lead to confusion. In some cases, for example, the overall process of structured reflection on the future is referred to as “horizon scanning” (UK Government Office for Science, 2013), while in others it is termed “foresight” or “future(s) thinking” (FAO, 2013). In this report, the committee has adopted a definition similar to that used by the Organisation for Economic Co-operation and Development (OECD): horizon scanning is “a technique for detecting early signs of potentially important developments through a systematic examination of potential threats and opportunities, with emphasis on new technology and its effects on the issue at hand” (OECD, n.d.a). Horizon scanning can be integrated into a broader futures-thinking or foresight framework. This framework describes the overall broader process of assessing and understanding the policy implications of relevant developments, as well as identifying desired futures and specific policy actions that can help realize them (see Annex 6-1 for more detailed discussion of these terms). ETH Zurich developed a model foresight process as part of efforts to strengthen policy making in Switzerland (Habegger, 2009) (see Figure 6-1). This model has three phases. The first involves the identification and monitoring of relevant issues, trends, developments, and changes, accomplished using the tool of horizon scanning. The second phase is assessing and understanding the resulting policy challenges, which makes use of different tools. The third phase involves envisioning desired futures and identifying specific policy actions for realizing them, based on the development of specific scenarios. FIGURE 6-1Three phases of a comprehensive foresight process. SOURCE: Illustration by Habegger, 2009, based on Schultz, 2006, and Horton, 1999. This chapter considers horizon scanning in depth, starting with an exploration of how it is used as a policy tool. This is followed by an overview of good practices in horizon scanning. This overview considers potential sources of information, the development of criteria to parameterize the scan or to use for evaluating the outcome, and avenues for improving traditional horizon-scanning methods. Also considered are issues related to communicating the results, connecting the results to specific actions, and learning lessons from the past. To demonstrate how horizon scanning works in practice, the chapter then presents case studies of relevant scans carried out in the past, both in the United States and in other parts of the world. Several of these case studies focus specifically on biotechnology, while others have been produced by sectors potentially relevant to this study, such as defense, health, food safety, agriculture, and environment and conservation. Next, the chapter places horizon scanning within the broader context of exploring a number of relevant toolkits, handbooks, and guidance, as well as the application of forecasting, or future thinking, by what is termed “superforecasting.” The chapter ends with the committee’s conclusions outlining a possible mechanism for future thinking and horizon scanning tailored to the U.S. bioeconomy, based on existing best practice and making use of current resources. HORIZON SCANNING AS A POLICY TOOLHorizon scanning, often as part of a foresight process, can help address a wide variety of policy-making needs (see Annex 6-1 for an overview of one such analysis). It can also generate important information (such as the identification of important trends or developments), and help gain lead time in addressing future issues or serve as an input for scenario-development processes (European Commission, 2015; OECD, n.d.a). It can help ensure that policy making incorporates “thinking outside the box” and that it is able “to manage risk by planning ahead for unlikely, but potentially high impact events” (UK Government Office for Science, 2013). More broadly, benefits accrue from bringing together experts and policy makers from different backgrounds and disciplines (Habegger, 2009). It is important to recognize, however, that horizon scanning operates beyond a firm evidence base and relies on the instincts of those involved in the exercise (UK Government Office for Science, 2017). The process of horizon scanning can be considered to encompass two separate approaches: “Continuous scanning activities to keep the overview (often with regular newsletters), regular but discontinuous activities (e.g., every 5 years) and ad-hoc Horizon Scanning for a specific purpose, on demand or at a specific occasion” (European Commission, 2015). A number of different horizon-scanning methods have been identified. For example, the Food and Agriculture Organization of the United Nations (FAO) developed a typology that includes best–worst scanning for prioritizing trends or developments, delta scanning for capturing identified trends and developments from other horizon-scanning processes, expert consultations for tapping specialist knowledge, and manual scanning to identify signals of change to track trends and drivers. FAO also provided examples of how each of the methods is commonly used and provided indicative strengths and weaknesses for each (FAO, 2013). Horizon scanning has been explicitly integrated into policy-making processes in some parts of the world. For example, the United Kingdom has integrated horizon scanning into its central policy making through its Cabinet Office. The United Kingdom uses horizon scanning as part of a larger foresight process to gather information on relevant trends and developments (monitoring) and explore their possible implications. Horizon scanning is additionally used as a mechanism for engaging people in future thinking and generating an environment conducive to yielding insights into the changing policy environment. Similar efforts have been undertaken, for example, in Singapore (Chong et al., 2007), the Netherlands (European Environmental Agency, 2011), and Switzerland (Habegger, 2009). Efforts in Singapore have focused heavily on automating a horizon-scanning process. GOOD PRACTICES IN HORIZON SCANNINGThe Horizon-Scanning ProcessA number of different horizon-scanning processes have been described, including by the UK Government Office for Science (2017), the European Union (EU) Directorate-General (DG) for Research and Innovation (European Commission, 2015), the Institute for Risk Management (IRM, 2018), and several academic groups (Brown et al., 2005; Habegger, 2009; Wintle et al., 2017). An example of a horizon-scanning process is provided in Figure 6-2. In general, these processes share the following features. They start by defining the scope of the scan and then identifying experts likely to have important relevant insights. For example, the IRM process emphasizes the importance of involving a diverse range of participants with open minds (IRM, 2018). Several other models stress that the process can be open-ended, involving as many people as desirable. Of course, increasing the number of people involves additional burdens in terms of tracking and compiling the results and may necessitate a dedicated project manager. Participants are then tasked with compiling a structured scan of a specific issue in a fixed timeframe. For example, the UK process suggests one scan per person per week (UK Government Office for Science, 2017). The issues to be covered can either be pre-identified or identified at the discretion of the participants, thereby drawing on their expertise and insights as to what may be relevant. Each scan describes the trend or development identified, how it relates to the policy or strategy area being explored, why the participants found it important, and what thoughts it stimulated. These descriptions can usefully contain links to original sources or additional information, but preferably are short. For example, the UK process suggests no more than one page (UK Government Office for Science, 2017). FIGURE 6-2An example of a horizon-scanning process. SOURCE: Wintle et al., 2017. Some processes stop at this point, and their final output is a series of collated issue scans over time, although this output is then sometimes fed into other activities as part of a larger process, as is the case in the United Kingdom (UK Government Office for Science, 2013). Other processes go further and provide additional steps that involve discussing, refining, rating, or otherwise reviewing the scans within the horizon-scanning process itself. For example, the process developed by the EU DG for Research and Innovation calls for expert dialogue. Some of the academic processes involve a more comprehensive semiquantitative approach, including the need for in-person interaction through a workshop (European Commission, 2015). Some processes then include additional steps to package and frame the results to facilitate their use in policy making. For example, the IRM process highlights the value of visualization (IRM, 2018). Optimizing a Horizon-Scanning ProcessSeveral factors, such as the sources of information, the decision criteria, methodological tools to tailor the generic process, and the policy impact need to be considered when seeking to optimize a horizon-scanning process (for more detailed discussion of each of these factors, see Annex 6-1). Sources of information—Information for a horizon scan can come from a number of different sources. Some sources, such as publications, quantitative data, and published opinions, may be more traditional. To reach the limits of current thinking, however, less traditional sources, such as news outlets, social media, and prepublication servers, may be needed. The process of gathering information can also be increasingly automated as the topic becomes more familiar. Decision criteria and questions to ask—Either when developing a scan on a topic or when reviewing its potential policy impact, a range of criteria can be applied, such as credibility, novelty, likelihood, impact, relevance, time to awareness (how long before the topic or its impact is widely known), and time to prepare for the development. A number of specific questions for exploring each of these criteria have been proposed (Hines et al., 2018). Methodological tools to tailor the generic process—A number of recent publications describe methodological tools for horizon scanning. Examples include the use of pre-developed scenarios to aid in the identification of important weak signals (Rowe et al., 2017); more structured approaches for matching specific horizon-scanning tools to the needs of policy makers, including better metrics (Amanatidou et al., 2012); the integration of more comprehensive collaborative review processes to identify appropriate responses by policy makers and practitioners (Sutherland et al., 2012); and mechanisms for assessing the value of different information sources to be used for the horizon scan (Smith et al., 2010). Increasing the policy impact—A number of good practices for presenting and communicating the results of a horizon scan have been identified, including having a specific sponsor for horizon-scanning and futuring work; translating results in a more accessible manner; tailoring reporting to policy interests; matching timing to political timeframes; selecting experts to increase policy relevance; focusing on potential impacts of events discussed, as well as the timeframes involved; and structuring the results in a logical manner, whether by groups of issues identified or by relevant policy drivers. Lessons Learned from Past Uses of Horizon ScanningA number of lessons have been distilled from previous uses of horizon scanning in policy making. For example, horizon-scanning experts consulted by the committee3 discussed (1) the use of expert opinion, (2) sources of bias and approaches to managing them, and (3) options for evaluating the effectiveness of horizon scanning. On the use of expert opinion, the speakers observed that individuals’ expertise declines dramatically outside the narrow domain of their area of technical specialization or experience, and pointed out that there is also particular value from generalist, nonexpert input. Relatedly, age, number of publications, technical qualifications, years of experience, memberships in learned societies, and apparent impartiality do not explain an expert’s ability to estimate unknown quantities or predict future events. However, a number of factors tend to lead to better judgments. An example is experts with experience in fields requiring rapid feedback, such as chess players, weather forecasters, sports players, gamblers, and intensive care physicians. People who are less self-assured and assertive and integrate information from diverse sources also make better judgments. It was noted as well that estimates of risk in many domains can be improved by weighting experts’ opinions by their performance on test questions and that relevant training can improve experts’ abilities to estimate probabilities of events. Lastly, group estimates consistently outperform individual estimates, and diverse groups tend to generate more accurate judgments. On biases, the experts who spoke to the committee identified the various types of bias and suggested ways to mitigate their effects on the process and outcome of horizon scanning. Gambler’s fallacy (the belief that past events will unduly impact future events) and the availability heuristic (the potential to be overly influenced by more recent memories and events) can be mitigated by identifying and unpacking assumptions inherent in the process, both in the task assigned and on the part of those involved. Confirmation bias (the likelihood of searching for, interpreting, focusing on, and remembering information that confirms preconceptions) can be mitigated by involving participants from a wide range of backgrounds and expertise, drawn from different communities and locations. Projection bias (the belief that preferences will remain the same over time) can lead to focusing on only a subset of issues or options. It can be mitigated by unpacking assumptions and questioning them, as well as by expanding the range of expertise involved in the process. The bandwagon effect, or “groupthink,” increases the likelihood of failing to explore the full range of options or issues, and can be countered by deliberately involving experts from diverse backgrounds and communities. Anchoring bias (the tendency to rely too heavily on a single piece of information, which is often the first obtained) can be mitigated through the use of advocates both for and against a specific issue, as well as multiple rounds of scoring in different orders. Finally, salience bias (the likelihood of focusing on something more prominent or emotionally impactful, especially when particularly vocal or skilled raconteurs are advocating for specific issues) can be managed through rules on advocating positions that are consistently and rigorously enforced, as well as the use of voting and anonymous feedback. On evaluating the effectiveness of horizon scanning, the experts commented that attempts have been made to review the impact of past horizon scans. As might be expected, these efforts have demonstrated that some issues were identified in a timely manner and deemed impactful, while others that were identified ultimately had a minimal impact (Sutherland et al., 2012). However, given that horizon scanning is not about predicting the future, assessing the “hit rate” of predictions is an inappropriate metric. The absence of an event is not necessarily the absence of impact. Identifying an early signal and taking effective policy action may result in an apparent null outcome. Metrics for a horizon-scanning or futuring effort might therefore be focused more usefully on exploring whether the effort led policy makers to consider more issues or explore more options. Alternatively, useful insights might be gained by comparing the assessments resulting from a horizon scan against those resulting from other tools with respect to facilitating better policy making. Publications from other entities, such as the National Intelligence Council (NIC, 2017), the U.S. Forest Service (Hines et al., 2018), the UK government (UK Government Office for Science, 2017), and the EU (European Commission, 2015), have documented reflections, key considerations, rules for implementation, and improvements made through iterative use of horizon scanning (see Annex 6-1 for a detailed discussion of lessons learned). CASE STUDIES OF HORIZON SCANNINGA number of horizon scans relevant to this study have already been carried out. Both the content of these scans and the communities that produced them could serve as important resources moving forward. The committee noted a paucity of documented horizon-scanning activities performed by U.S. federal agencies. Should relevant federal agencies be carrying out these activities, there is considerable room to enhance transparent reporting of and sharing of experiences from those efforts. This section provides examples of past scans and the actors undertaking them. The scans reviewed include those directly connected to the bioeconomy, those conducted within the U.S. Intelligence Community, those carried out by agencies with a direct role to play in safeguarding the bioeconomy, and those conducted within a U.S. federal agency. Examples of additional horizon scans are described in Annex 6-1, including efforts that have brought together separate horizon scans from different agencies and subject-specific scans in areas related to the bioeconomy, such as health, food safety, and the environment and conservation. Example of a Horizon Scan Connected to the BioeconomyIn 2017, a transatlantic horizon scan was published describing developments in biological engineering likely to have substantial impacts on global society. The process brought together experts in horizon scanning, biosecurity, plant biotechnology, bioinformatics, synthetic biology, the bioeconomy, biodefense, science policy, nanotechnology, conservation and environmental sciences, industrial biotechnology, and the social sciences. These experts used the process described in the section above on the horizon-scanning process to identify 70 potential issues and then prioritized 20 of these issues, covering such sectors as health, energy, agriculture, and the environment (Wintle et al., 2017) (see Table 6-1). TABLE 6-1Issues in Biological Engineering Likely to Have Substantial Impacts on Global Society in the Short, Medium, and Long Terms. The 20 prioritized issues were categorized according to their likely timeline for impact. Highlighted as likely to have an impact within 5 years were five issues, including novel approaches to gene drives (which subsequently received notable backing for development from major science funders) (Wellcome Trust, 2017), human genome editing (2018 saw the birth of the first genome-edited babies) (Cyranoski and Ledford, 2018), and accelerated defense agency research (with novel research programs causing debate within the biosecurity community on the desirability of such research) (Lentzos and Littlewood, 2018). Ten issues were deemed likely to have an impact in 5–10 years, including cyberbiosecurity and corporate espionage and biocrimes (which are directly connected to the aims of this study). Finally, five issues were identified as likely to have an impact in more than 10 years, including securing critical infrastructure needed to deliver the bioeconomy. Example of Horizon Scanning Within the U.S. Intelligence CommunityShortly after the start of each presidential term, NIC publishes “an unclassified strategic assessment of how key trends and uncertainties might shape the world over the next 20 years to help senior U.S. leaders think and plan for the longer term” (NIC, 2017). Comparatively few details are publicly available about the precise methodology used by NIC, but according to the NIC (2017) report, it involved desk research as well as consultations with experts from inside the U.S. government and from around the world. This enabled the identification of, and subsequent reflection on, key assumptions and trends. Assessment of implications was first carried out at the regional level before being aggregated to identify global trends. The results were structured over different timeframes, ranging from the near term (5 years) to the long term (20 years). Analytic simulations were used to explore future scenarios, in particular how uncertainties and trends might combine to alter outcomes. The scale and breadth of the consultations reported were also noteworthy:
These expert interviews and the feedback received were then integrated into a scenario-based, policy-oriented foresight approach. The scenario work and backcasting efforts were used to identify choices and policy decisions that could help realize desirable futures and avoid the undesirable (NIC, 2017). Specific tools used in the preparation of the NIC report that might be important for forecasting work relevant to this study include net assessment and analytic simulations. Net assessment is
Net assessment “uses data that are widely available and creates strategic insights that lead to decisive advantage. It offers paths through the increasingly dangerous landscape of national security.” It often makes use of a specific set of tools. “Scenarios, war games, trend analysis, and considered judgment are the methods most widely used in net assessment studies and analyses” (Bracken, 2006). Analytic simulations, including historical wargaming and analytic path games, have proven useful in military planning for future conflicts. They have allowed commanders to plan for the unknown by both better understanding adversaries and preparing possible responses in advance of events.4 Example of Horizon-Scanning Tools Being Developed by an Agency Connected to Safeguarding the BioeconomyIn 2015, the Office of Technical Intelligence in the U.S. Department of Defense published an assessment of data analytics–enabled technology watch and horizon scanning (TW/HS) for the identification, characterization, and forecasting of known and unknown science, technology, and applications (Office of Technical Intelligence, 2015). According to the assessment report, “data-enabled TW/HS has the potential to improve upon or augment current approaches by expanding the aperture of analyses and decreasing the influence of bias, while at the same time building institutional capacity.” The report includes a structured framework for integrating new technologies (such as data analytic tools) into existing workflows. This framework reflects components of the generic horizonscanning process described earlier, including the following (all descriptions are from Office of Technical Intelligence [2015]): Characterizing decisions (see the above discussion of criteria and questions to ask)—Those undertaking the scan need an understanding of the decision itself; the timeline governing their work; and, most important, the evaluation criteria. This understanding “informs the scope, scale and context of the supporting analysis, which enables analysts to provide targeted, actionable inputs into the decision process in time for the information to be actionable.” Selecting data (see the above discussion of sources of information)—This process “requires careful balancing of relevance and breadth. It is critical to identify sources that are likely to provide signals relevant to the evaluation criteria and to maximize the signal to noise ratio.” Selecting metrics (see the above discussion of methodological tools and lessons learned from past uses of horizon scanning)—“Evaluation criteria are often complex human ideas which cannot be precisely calculated from data. For example, analytics cannot directly assess the maturity of a technology, but they could analyze the amount of activity which references the technology, growth rates of activity, or identify whether sources discuss prototyping or advanced testing to inform a technology readiness level estimation.” Conducting analysis (see the above discussion of decision criteria and questions to ask)—“To enable more effective application of metrics, it is often valuable to develop a taxonomy of the field under consideration. Taxonomies allow for the identification of areas at the same level of abstraction.” Developing decision support products (see the above discussion of increasing policy impact)—“Analysts must integrate the disparate portions of their findings into a cohesive whole in order to make their efforts useful to decision makers… [this] requires understanding what is useful to the decision maker, such as whether the individual metrics or a composite score would be most useful and how to communicate the findings so that they are both clear and most likely to be used effectively.” Leveraging knowledge management (see the above discussions)—“In order to move from a successful TW/HS project to a TW/HS program, it is important to ensure that products can be kept up to date with manageable amounts of effort and to track the accuracy of analysis.” Example of a Horizon Scan in a U.S. Federal AgencyIn 2018, the U.S. Forest Service’s Strategic Foresight Group and the University of Houston’s Foresight Program published a summary of their efforts “to develop an ongoing horizon scanning system as an input to developing environmental foresight: insight into future environmental challenges and opportunities, and the ability to apply that insight to prepare for a sustainable future” (Hines et al., 2018). The process adopted was similar to that described earlier. It included an initial framing phase in which the domain of interest was mapped (including the identification of key activities, stakeholders, and drivers of change), geographic and timeframe boundaries were set, relevant stakeholders and participants were identified, and guiding questions were developed. The scan itself used a four-step process:
The criteria used in the scan to determine the relevance of an issue were those described earlier in the discussion of criteria and questions to ask. The authors identify a number of specific lessons learned from attempting to develop a horizon-scanning process within a U.S. federal agency. The study also includes a discussion of future plans for improving the communication of results, integrating the results into the host organization, and linking the results to effective action, as well as making the process self-sustaining. Examples of Environment- and Conservation-Related Horizon ScansOne example of an international horizon-scanning effort related to the environment and conservation is a 2016 international study by academic authors from 11 countries that focused on issues likely to impact pollinators and pollination positively or negatively in the future and that succeeded in identifying six high-priority issues and nine secondary issues (Brown et al., 2016). A second example is a 2018 international study by academic authors from six countries that identified “15 emerging priority topics that may have major positive or negative effects on the future conservation of global biodiversity, but currently have low awareness within the conservation community” (Sutherland et al., 2019). The latter is the tenth annual review conducted by this group, and its methodology was employed in the scan of biological engineering described previously. ADDITIONAL TOOLS FOR FUTURE THINKINGIn practice, horizon scanning is rarely used in isolation, but is often combined with a range of other tools and techniques. Sometimes, these tools and technique are combined into a stand-alone exercise (such as the integration of Delphi, a consultation process to gather input from a wide variety of experts and sometimes prioritize the results, and other expert review processes discussed in Annex 6-1). Alternatively, horizon scanning can be embedded in a more comprehensive foresight process that feeds the results of the scan into processes for assessing and understanding the consequent policy challenges, connecting them to possible future scenarios, and identifying specific policy actions designed to steer toward desirable outcomes. See Annex 6-1 for further detail on the additional tools discussed here. Forecasting ToolsSeveral studies have catalogued a comprehensive range of forecasting tools. For example, the Handbook of Technology Foresight, published in 2008, explores in depth 19 qualitative tools, 8 quantitative tools, and 9 semiquantitative tools (Popper, 2008) (see Table 6-2). FAO outlined a similar list of tools in 2014, providing a description of each tool, examples of its common use, and its particular strengths and weaknesses (FAO, 2013). And OECD has highlighted four tools as being particularly important: the scenario method, the Delphi method, horizon scanning, and a trends impact analysis (OECD, n.d.a). Many of these tools have been combined into frameworks for forecasting. Box 6-1 describes an example developed by the UK Government Office for Science. TABLE 6-2Foresight Tools Identified by Academic Studies and Intergovernmental Organizations. BOX 6-1The UK Government Office for Science’s Futures Toolkit. SuperforecastingIn 2010, the Intelligence Advanced Research Projects Agency (IARPA) initiated a competition to explore how crowdsourcing can improve forecasting.5 Various tools and approaches for making accurate predictions were tested over 4 years of tournaments. IARPA identified a number of promising tools, but also concluded that (1) some individuals were notably better at making predictions than others, and (2) it is possible to learn how to be better at making predictions. These two conclusions formed the basis of what was to become known as superforecasting. A superforecasting program brings together those with a proven track record in making predictions in a system designed to enhance their abilities and in making use of tools to help interpret the results. Since the conclusion of this program, a successful team of established superforecasters has created the Good Judgment project, which offers superforecasting capabilities and training for commercial entities and public processes.6 RoadmappingRoadmapping “shows how a range of inputs—research, trends, policy interventions, for example—will combine over time to shape future development of the policy or strategy area of interest” (UK Government Office for Science, 2017). A wide range of countries and regions have developed roadmaps for their bioeconomy.7 In 2019, the Engineering Biology Research Consortium published “Engineering Biology: A Research Roadmap for the Next-Generation Bioeconomy.” This roadmap was “intended to provide researchers and other stakeholders (including government funders) with a compelling set of technical challenges and opportunities in the near and long term.” It covers four technical themes and explores five application sectors (see Box 6-2). BOX 6-2Technical Themes and Application Sectors Addressed in the Engineering Biology Research Roadmap. CONCLUSIONSDuring the committee’s webinar on horizon-scanning methodologies, experts highlighted four key questions to consider when developing a horizon-scanning process.
Following discussion of the above questions, the committee concluded that best practices for horizon scanning include the considerations laid out below.
Ongoing horizon scanning might be integrated into the work of different agencies with specific fields of expertise, using the good practices discussed in this chapter. Encouraging such agencies to share their experiences with each other would help to build relevant capacity as quickly as possible. In some cases, horizon scanning for important policy issues may already be under way. Different issues identified in these field-specific scans could then be fed into a centralized meta-review. This approach would make use of good practice in horizon scanning (as described in this chapter) to compare different issues using a common set of criteria and scoring systems and multiple rounds of voting. These ongoing activities could form the basis of a regular report, similar to NIC’s Global Trends report.
One-off horizon scans could be used to answer specific questions or drill down into specific issue areas. Such a process might follow an approach similar to that of the example horizon scan presented earlier in Figure 6-2. It would include modified use of the Delphi method to highlight issues considered most likely to have a policy impact, or highly novel issues that are likely to be omitted from policy-making processes. One issue that could greatly benefit from both one-off horizon scans and continued assessment is the creation and maintenance of bioeconomy-specific satellite accounts (see Chapter 3 for further detail). This combined approach is particularly suitable for the creation of satellite accounts as it serves a policy need, and the bioeconomy is continually changing.
While these horizon-scanning processes are likely to be expert-driven, tools for automated data gathering are advancing and could be integrated into the methodology used for a horizon scan as appropriate. It will be important to involve the widest possible range and diversity of expertise. The meta-review process, resources permitting, might resemble the scope, scale, and nature of NIC’s Global Trends report, aiming to directly engage thought leaders from different communities around the world. Criteria to be applied in assessing potential issues to be fed into horizon scans include credibility (e.g., Is the source reputable? Is it confirmed elsewhere?); novelty (Is the issue new, or has it already been widely reported?); likelihood (What are the chances the issue will actually occur?); impact (Will the issue change the future, and if so, how big will that change be?); relevance (How relevant is the issue to the bioeconomy, and is that relevance direct or indirect?); time to awareness (How long is it likely to be before the issue is widely known, and could this change [or be changed]?); and time to prepare (When is the issue likely to have an impact, what could affect its impact, and when would that intervention need to take place?).
The above conclusions represent the committee’s view of elements for a future-thinking and horizon-scanning mechanism for the bioeconomy. A structured foresight process making use of horizon scanning would help support policy making around the future of the bioeconomy. Chapter 8 considers the establishment of a government-wide mechanism to monitor and oversee the U.S. bioeconomy. Future thinking and horizon scanning should be a tool at this network’s disposal.
Foresight processes build on horizon scanning intended to identify issues that could have a policy impact, feeding into assessment and scenario-based processes for exploring policy options. How horizon scanning is integrated into broader foresight activities will depend on the ultimate purpose at hand. The committee’s Statement of Task on horizon scanning includes both (1) identifying gaps in terms of new technologies, markets, and data sources that could provide insights into the bioeconomy; and (2) identifying and helping to prioritize opportunities and threats with respect to safeguarding the bioeconomy. A structured, flexible, and adaptive foresight process is key to identifying additional strategies that might be needed to safeguard these new technologies and data and assess their implications for innovation and biosecurity. A model for such a foresight process that embraces both tasks can be found in two of the pathways included in the UK Government Office for Science’s Futures Toolkit (see Box 6-1): identifying future research and evidence priorities and identifying and prioritizing future opportunities and threats for action. These pathways could usefully be adapted to take advantage of existing foresight resources and approaches and other tools in use within the U.S. government.
The aim of this process, which would need to be integrated into the specific questions asked of participants, would include identifying “known unknowns” and previously “unknown unknowns.” It would be used to begin to formulate hypotheses about the future of the bioeconomy and to shape future research agendas. It would use desk research, interviews, and workshops to produce an evolving roadmap showing how the issues identified could impact the bioeconomy over time. Such a process would need to involve both subject-matter experts and policy makers responsible for relevant areas (see Annex 6-1 for more detail on exactly what such a process might entail). Horizon-scanning activities would be fed into driver mapping, which could be used to categorize, but not prioritize, drivers. The results of this activity would then be subjected to SWOT (strengths, weaknesses, opportunities, and threats) analysis. That analysis might usefully identify whether the threat or opportunity will impact the bioeconomy in the short, medium, or long term; the potential outcome or implications for the bioeconomy; whether there are control measures that could be implemented; what actions could be taken directly or indirectly to mitigate threats or seize opportunities; and with whom it will be necessary to work to deliver that action. Likely timeframes and impacts also might usefully be addressed using superforecasters. Possible actions, partners, and control measures might be explored using net assessment and analytic pathway games. Annex 6-1. Defining Horizon ScanningIn this report, the committee uses the terms “horizon scanning” and “future thinking”/“foresight” as developed by the Organisation for Economic Co-operation and Development (OECD):
Use of these definitions is consistent with their use in other settings. The Food and Agriculture Organization of the United Nations (FAO), for example, notes that horizon scanning “generally refers to methodological approaches that scan or review various data sources, while Foresight generally refers to the wider group of more participatory methods” (FAO, 2013). There have been numerous other attempts to define horizon scanning (European Commission, 2015; IRM, 2018; OECD, n.d.a; UK Government Cabinet Office, 2013). Common components of these definitions include that the tool
HORIZON SCANNING AS A POLICY TOOLAccording to the Institute for Risk Management, horizon scanning is used as a tool
The European Union (EU) Directorate-General (DG) for Research and Innovation has outlined a series of considerations for developing a horizon-scanning process (European Commission, 2015):
The EU DG notes that determining the needs of a specific horizon-scanning process for each of these considerations will likely have implications for how focused the results will be. The specific needs of each category will also determine the time and resources required (European Commission, 2015). The United Kingdom provides an example of horizon scanning in policy making, having integrated horizon scanning into its central policy making through its Cabinet Office. The UK process considers three policy horizons (see Figure Annex 6-1). Horizon 1 relates to impacts that will be felt today and tomorrow, where “trends and events stand out against the background and their impacts are clearly signaled to policy makers.” These trends and events can be addressed by actions currently being taken. Horizon 2 comprises trends whose impact will be seen in the short to medium term and can be fed into strategic thinking. Horizon 3 encompasses those trends that will grow in importance in the longer term, for which some planning may be needed. The UK process frames horizon scanning as a tool that “looks towards the long term (Horizon 2 to 3) but is not focused exclusively on it; many H3 developments are the long-term outcome of a range of factors, some of which are in play already” (UK Government Office for Science, 2017). FIGURE ANNEX 6-1The United Kingdom’s three-horizons model for future thinking representing short-, medium-, and long-term timescales of outlook. SOURCE: UK Government Office for Science, 2017. GOOD PRACTICE IN HORIZON SCANNINGFactors to be considered when developing a horizon-scanning process include sources of information, criteria and questions used to explore them, and policy impact. Sources of InformationInformation for a horizon scan can come from a wide variety of sources, and needs to be tailored to the area of interest of the individual process. Information sources can be traditional, such as publications, quantitative and qualitative data, and published expert opinions, but it is equally important to consider unique sources that fall on “the margins of current thinking,” ensuring a holistic perspective (Habegger, 2009). As a result, sources can also be less traditional, such as news outlets, social media, and prepublication servers. In addition, the process may need to take into account insights into lifestyles, people’s sociological expectations, or other indicators of potential impact. It will often benefit from including insights from key stakeholders, such as those provided by professional bodies, industry leaders, customers, or those working in the field in question. It is also possible to apply semiquantitative approaches to rating the utility of different sources (Smith et al., 2010). Efforts are under way to move from manual compilation of information using experts to more automated models. For example, Singapore established the Risk Assessment and Horizon Scanning Experimentation Center to develop better tools for data analytics, modeling, and perspective sharing (Chong et al., 2007). Efforts have been made as well to adapt advances in agent-based modeling in order to automate some of the analysis of the output from horizon scans (Frank, 2016). Criteria and Questions Used to Explore ThemWhen a scan of a short timescale on a specific topic is being prepared, it is important for it to describe the trend or development identified, explain how it relates to the policy or strategy area being explored, and detail why the trend or development is believed to be important and what thoughts it stimulated. The process can include links back to supporting materials and additional information. To ensure comparability, some processes suggest that those participating in a horizon scan attempt to frame the issues at a similar level of granularity. For example, very specific developments might have a profound impact in one area but be much less likely to have an impact at the level of a policy development. On the other hand, overgeneralization may offer policy relevance but lack specific ties to trends or developments specific enough to be targeted by policy actions (Wintle et al., 2017). Either when developing a scan on a topic or when reviewing its potential policy impact, a number of specific criteria have been suggested, and specific questions have been proposed for exploring each criterion (see Table Annex 6-1) (Hines et al., 2018). TABLE ANNEX 6-1Criteria and Questions to Be Considered When Conducting a Horizon Scan. There are also more quantitative approaches for comparing criteria. For example, an analytic hierarchy process can be used to weight the criteria applied in a horizon-scanning exercise (Mehand et al., 2018; WHO, 2017). Policy ImpactDuring the committee’s webinar on horizon scanning, speakers indicated the importance of having a specific sponsor for horizon-scanning and futuring work. A sponsor would need to have the resources to sustain relevant work, the ability to feed the results into relevant policy-making processes, and a high-level interest in the work to ensure that neither the process nor the conclusions of the horizon scan would readily be sidelined. Speakers also discussed the importance of carefully considering how the output from foresight processes might best be used to inform decisions, i.e., how the future can be used to inform today’s decisions. That process would likely involve creating a narrative for the future, including through different storytelling approaches. It is also useful to use backcasting (starting with a desirable future and working backwards to highlight decisions and actions that connect it to the present).8 The EU has stressed the importance of people in translating the results of a horizon scan into action. It suggests that while parts of the process might be automated, expert involvement is likely to result in more policy-relevant output. It also stresses the importance of understanding who might take action as a result of the scan, what their drivers and priorities are, and a clear plan to engage them (or ensure their buy-in from the start) (European Commission, 2015). The Institute for Risk Management recommends developing a framework for categorizing separate scans to facilitate comparing and reviewing them. It also stresses the importance of highlighting the potential impact of the events and trends identified, in particular describing potential risks and time to impact, which should help an end user better understand the need to take action and how fast it is necessary to act (IRM, 2018). In its Futures Toolkit, the United Kingdom further elaborates on the importance of a framework for categorizing scans. It proposes two possible approaches: either structuring them according to different change drivers, such as political, economic, societal, technological, legislative, or environmental factors; or preferably grouping them by themes that emerge from the scans themselves. The toolkit highlights two different formats for presenting the results of a scan: a longer narrative summary providing an overview, broad implications, and specific policy implications; and a shorter structured summary providing a few simple details of impacts, issues, and implications (UK Government Office for Science, 2017). CASE STUDIES OF HORIZON SCANNINGExamples of Health-Related Horizon ScansThere have been numerous efforts to use horizon scans to identify and prioritize emerging technology in the health sector. Some examples are published snapshots of a single horizon scan, while others are ongoing monitoring processes, and a few track trends in the use of these tools. Examples include the following:
Examples of Food Safety–Related Horizon ScansFAO identified several organizations that have conducted or continue to regularly conduct horizon scans for food safety (FAO, 2013):
Example of Combining Separate Horizon ScansThe United Kingdom’s Futures Toolkit includes case studies of how seven different government agencies and ministries make use of futuring tools. Each case study sets out the purpose of the work, the tools used, resources required, the work’s sponsor, specific outputs, particular successes, and challenges. Five of these agencies—the Environment Agency, the Forestry Commission England, the Health and Safety Executive, Revenues and Customs, and Natural England—make specific mention of the purpose of their horizon-scanning work (UK Government Office for Science, 2017). The purposes cited differ and include using horizon scanning to identify new and emerging issues and trends; improve the evidence base for decision making and risk mitigation; help identify risks and opportunities; integrate externalities into business planning; and inform strategy, provoke discussion, and shape thinking. LESSONS LEARNED FROM HORIZON SCANNINGIn addition to lessons identified during the webinar held by the committee, several key actors, including the National Intelligence Council (NIC, 2017), the U.S. Forest Service (Hines et al., 2018), the UK government (Carney, 2018), and the European Union (European Commission, 2015), have distilled lessons from their past use of horizon scanning. National Intelligence Council Global Trends ReportImprovements in methodology integrated into the most recent iteration of the Global Trends report produced by the National Intelligence Council include (NIC, 2017)
U.S. Forest ServiceAlso in the United States, efforts to establish a horizon-scanning system in the Forest Service led to a number of key reflections, including the following (Hines et al., 2018):
UK GovernmentBased on the use of horizon scanning in the UK government, 10 key rules have been identified. Some of these rules have been discussed in this annex and in the main text of Chapter 6—for example, (1) that horizon scanning is not about predicting the future but about challenging assumptions and increasing options, (2) that there is a lack of common understanding about what horizon scanning is or the terms being used, and (3) that focusing on impact and explicitly exploring the implications of the trends or events identified are important. Other rules bear emphasizing here, such as the importance of (Carney, 2018)
European UnionSimilarly, the European Union has identified a number of key considerations, including (European Commission, 2015)
ADDITIONAL TOOLS FOR FUTURE THINKINGSuperforecastingAs discussed briefly in the main text of Chapter 6, in 2010, the Intelligence Advanced Research Projects Agency (IARPA) created a program to explore how crowdsourcing can improve forecasting15:
Similar programs have subsequently focused on developing “innovative solutions and methods for integrating crowd sourced forecasts and other data into accurate, timely forecasts on worldwide issues.”16 There have also been programs created “to develop and test methods for generating accurate forecasts for significant science and technology (S&T) milestones, by combining the judgments of many experts”17; and “to develop automated methods that aid in the systematic, continuous, and comprehensive assessment of technical emergence using information found in published scientific, technical, and patent literature.”18 IARPA tested the tools for aggregating crowdsourced forecasting in a 4-year series of tournaments, where
One successful team subsequently identified a number of key findings (Tetlock et al., 2017):
The last two of these findings form the basis of superforecasting (Tetlock and Gardner, 2015). This process brings together in teams those individuals with a proven track record of being able to make more accurate predictions, supported by specialized tools and algorithms so as to further increase their accuracy. A thorough assessment of the performance of superforecasters during the tournaments demonstrated that they were significantly more accurate in making predictions than other participants and that “tight restrictions on time and information did not erode the superforecaster advantage.” They were also better able to differentiate between signal and noise and were the fastest learners in the tournament. These studies demonstrated that while certain types of people are more likely to become superforecasters, certain skills and organizational arrangements can increase the ability to make accurate predictions. Thus, “superforecasters are partly discovered and partly created.” Mellers and colleagues (2015) identify “four mutually reinforcing explanations of superforecaster performance: (a) cognitive abilities and styles, (b) task-specific skills, (c) motivation and commitment, and (d) enriched environments.” The first cohorts of superforecasters were identified during the IARPA forecasting tournaments. Efforts to identify and recruit additional individuals have continued through Good Judgment Open.19 Since the tournaments, the approach has been developed into a commercial service through Good Judgment, which works with governments, the financial sector, and civil society and nongovernmental organizations, providing forecasting, training services, and tools and techniques.20 UK Government Office for Science’s Futures ToolkitIn 2017, the UK Government Office for Science (GO-Science) published a Futures Toolkit that “policy professionals can use to embed long term strategic thinking in the policy and strategy process.” It is intended to be “practical rather than theoretical and … based on GO-Science’s own experience of running futures work and has been developed in collaboration with other government departments and futures practitioners who use these tools regularly in a wide range of settings” (UK Government Office for Science, 2017). The tools in the kit are structured around four common uses for foresight:
As the task assigned to this committee was to “develop ideas for horizon scanning mechanisms to identify new technologies, markets, and data sources that have the potential to drive future development of the bioeconomy,” our focus was on the use of foresight tools to gather bioeconomy-related intelligence about the future. The toolkit describes four tools relevant for gathering intelligence about the future (UK Government Office for Science, 2017):
The tools in the kit are then combined in different ways to meet different needs, as captured in a series of pathways (UK Government Office for Science, 2017):
Given the focus of this study and the committee’s Statement of Task, Pathways 6 and 7 are of particular relevance. These pathways use additional tools, including (UK Government Office for Science, 2017) the following:
Pathway 6, “identifying futures research and evidence priorities,” begins with horizon scanning but feeds the results into 7 Questions, issues papers, driver mapping, and then roadmapping. Pathway 7, “identifying and prioritizing future opportunities and threats for action,” also starts with horizon scanning but feeds the results into driver mapping and SWOT analysis. REFERENCES
Mr. Flynn spoke during a webinar held for this study on June 11, 2019. 23These experts spoke at a webinar held for this study on June 11, 2019. 4This observation was made by a participant in the committee’s webinar on June 11, 2019. 567891011121314151617181920Which of the following is best used for vulnerability assessment?Explanation: White box testing provides the penetration testers information about the target network before they start their work. This information can include such details as IP addresses, network infrastructure schematics and the protocols used plus the source code.
Which type of assessment that is best used to identify classify and prioritize vulnerabilities?A vulnerability assessment is the process of defining, identifying, classifying and prioritizing vulnerabilities in computer systems, applications and network infrastructures.
Which of the following method is used by a tester in VAPT?VAPT Services
Penetration testing, or pen testing for short, is a multi-layered security assessment that uses a combination of machine and human-led techniques to identify and exploit vulnerabilities in infrastructure, systems and applications.
What is manual vulnerability scanning?Vulnerability scanners are used as part of the penetration testing process to discover weaknesses for manual exploitation during the discovery phase. The scanner is used to locate potential vulnerabilities, while the attack phase of the penetration test exploits the vulnerability and confirms its presence.
|