Practices:Expert Panels

From FORwiki

(Difference between revisions)
Jump to: navigation, search

Revision as of 09:00, 8 February 2010


The Expert Panels method is a commonly used method in Foresight to elicit expert knowledge. The panels are typically groups of 12-20 individuals who are given 3-18 months to deliberate upon the future of a given topic.

Contents

The FOR-LEARN Guide to Expert Panels

This is a summary of the article on Expert Panels from the FOR-LEARN guide. To read the full article go here.

Overall Description

The "expert panel" is one of the most frequently used methods in Foresight. Most of the activities in institutional Foresight exercises are carried out by expert panels. The expert panel method is based on the idea of eliciting expert knowledge. The panels are typically groups of 12-20 individuals who are given 3-18 months to deliberate upon the future of a given topic area, whether it be a technology (e.g. nanotechnology), an application area (e.g. health), or an economic sector (e.g. pharmaceuticals). In a Foresight exercise the expert panels have the following functions:
1. Gathering relevant information and knowledge;
2. Synthesizing the information gathered ;
3. Stimulating new insights and creative views and providing a vision of future possibilities, as well as creating new networks;
4. Diffusing the Foresight process and its results to much wider constituencies;
5. Influencing Foresight in terms of follow-up action.

When is this method appropriate?

Technology Foresight is, by definition, a participatory, discursive activity that should be based upon the best available evidence and judgment. These conditions make the use of (expert) panels a natural choice in Foresight exercises. Panels not only open up the Foresight process to potentially hundreds of individuals, they are also ideal forums for in-depth discussions and debate. For these reasons, panels are the "process centers" in many Foresight exercises.

Who is typically involved?

There are two interrelated considerations to take into account when profiling panels:
1. Composition – what mix of knowledge is required to address the panel remit?
2. Balance – what mix of views, positions, value judgments and scientific disciplines should be represented on the panel to ensure even-handed analysis and conclusions?
On a practical level, there are a number of approaches for actually identifying potential participants. These can be divided into:
1. Personal contacts: Using names known to those already involved in the project;
2. Stakeholders: Identifying major stakeholders in the areas of concern and asking them to put forward names;
3. Formal process: Involves more systematic search processes. Types of expertise and stakeholders are identified; a first set of names suggested; these are asked to nominate key people (introducing new names); then a final selection is made of the people who are to be invited to take part.

Approach (Step-by-step Guide)

There are two documents that are used to inform the panels about their tasks prior to starting work: (i) the Proposal and (ii) the Terms of Reference. The proposal document explains what the panel will do, and who (which experts/stakeholders) should be involved. Drawing on the proposal, the terms of reference set out what they should do, how it should be done, and when it should be completed. A short and succinct 'terms of reference document' can be divided into four parts:

  • Background, which provides some information on the Foresight programme and the purpose of the terms of reference document;
  • Description of each phases of the programme, setting out: (i) what needs to be achieved, (ii) how the panel should go about its work, and (iii) a series of milestones;
  • Description of the way in which the panels' work fits into the overall Programme;
  • Account of the human, infrastructure-related (including training) and financial resources available to the panels to support their work.

This document is distributed to all the panel members in the Programme and was used by the sponsor and project management team to monitor the progress of the panels.

Getting Started

The sorts of things that will need to be discussed and decided include:

  • Working practices and panel structure – for example, will the panel work as a whole or through sub-groups? Will particular panel members be assigned to lead on specific areas?
  • What are the data and research requirements? How will data be collected and analysed? Who will conduct research (e.g. project team, consultants, panel members)? What wider consultation will be carried out? What facilitation will be required for specialist methodologies? Panels may need help from experienced Foresight practitioners to be able to answer these questions effectively.
  • What will be the schedule of panel meetings? This includes the total number of meetings and their frequency. These can vary widely between panels, even within the same technology Foresight exercise. The panel (or project team) may also decide to prescribe the topic for each meeting – for example, 'meeting no.3' might be scheduled to deal with SWOT analysis or the like.
  • What will be the schedule of panel outputs, including the final report? In order to track and monitor progress, an agreed-upon milestone chart will need to be formulated (if not already specified a priori in the terms of reference).

Conducting Foresight work

While Foresight work is underway, it is often a challenge to get panels to think creatively about (a) the future, and (b) the means of getting there. People seem to find this difficult, partly due to the unfamiliarity of thinking in this way. It is therefore imperative to ensure that panels take sufficient account of (a) the long-term (short-termism is a common weakness in panels and workshops) and (b) a wide variety of perspectives on any given topic.
Project managers should publish brief progress reports at regular intervals – perhaps every 4-6 months, depending upon the duration of an exercise – whilst analysis prepared for or by the panels (e.g. SWOT analysis, literature reviews) could also be made publicly available. In this way, the evidence base (and assumptions) upon which a panel is working can be scrutinized. Such reporting may also be used as an opportunity to consult with wider communities of actors. Thus, in many technology Foresight exercises, interim reports containing preliminary analysis and findings are published and feedback invited.

Reporting on the panel process and findings

Panels will need to report on their findings, both at the end of their work and in the interim. The main rationale for reporting is to disseminate analyses and findings and to present priorities and recommendations for further action. Reports should therefore be tailored to their intended audiences. Reports are also used to demonstrate that panels conducted their work with integrity, drawing upon the best available evidence to support their findings.

Resources

As regards human resources, a related issue concerns the number of panel members to appoint to each panel. Most Foresight exercises have opted for 12-25 individuals per panel, with the average number being around 15. Typically, a small number of individuals are absent from each panel meeting, and this needs to be taken into account when deciding on the final number.
Considering the financial resources, costs must be taken into account when appointing panels. Financial costs include the following possibilities:
1. Honoraria to panel members and/or the panel chair;
2. Facilitators and/or secretaries;
3. Research and technical services;
4. Travel costs and other communications (e.g. telephone, document courier);
5. Rental of facilities;
6. Questionnaire surveys and/or workshops, report publication.
Time is needed for assembling the panel and any support staff, holding meetings, using methods such as Delphi or scenarios, preparing reports, and disseminating the final results. Realistic estimates must be made of the time and costs required to complete these tasks. This can prove difficult at the outset, and it is common to underestimate, especially with respect to the time needed. Indeed, it is not uncommon for technology Foresight exercises to overrun – usually by only a few months, but sometimes it can be longer.

Pros and cons

The main advantage of working with expert panels is that different types of players who might not normally meet in the course of a panel such as innovators, sponsors, policy makers, academic researchers, users and/or consumers can be brought together. Expert panels provide an environment where diverse viewpoints of stakeholders can be brought together freely. Further advantages are:
1. Availability of expert judgment 'on tap' at the center of an exercise;
2. In-depth and meaningful interaction and networking between different scientific disciplines and areas of expertise that would otherwise be difficult to organize;
3. The ease with which panels can complement other methods used in technology Foresight;
4. The credibility and authority lent to the technology Foresight exercise by the profile of panel members and the visibility of expert/stakeholder panels; and
5. The molding of influential individuals (panel members) into Foresight ambassadors and change agents in support of panel findings. However, the experience has demonstrated that the operation of expert panels is far from routine and unproblematic. Characteristics that may prevent the panel to work effectively include:
1. A dominating personality or outspoken person takes over the panel process so that the outcome tends to be his or her view;
2. Individuals are unwilling to commit themselves on an issue;
3. The superior vs. subordinate relationship hampers free expression of opinion by subordinates;
4. The unwillingness to abandon a position once it has been taken publicly;
5. Committee members are not necessarily familiar with the needs of the Foresight process and may fall into a conventional mode inappropriate to developing a longer term view of the topics under discussion.
Finally, an expert panel cannot produce a statistically significant outcome. The results provided by a panel will not reflect the response of a larger population or even the findings of a different panel. The panels usually consult through surveys, meetings or conferences to gather opinions from wider participants.

Possible variation of the approach

In a typical panel, experts meet face-to-face, normally in private sessions, at regular intervals over a fixed time period. During this time, they use their judgment in interpreting available evidence. They report their findings, usually through a written report that is later disseminated and, ideally, acted upon. However, there are many variants on this typical model of a panel. For instance, 'expert panels' may involve 'lay persons'. In fact, panels may not be 'expert' at all (at least not in the traditional, professional sense of the word). Instead, such panels may be composed of 'stakeholders', i.e. individuals (sometimes representing an organization) with a stake in the outcomes of the panel process. The practical life experiences of such individuals are typically taken as criteria for membership. Another deviation concerns the interaction of panel members, which need not be face-to-face. Indeed, some panels never meet at all. In such cases, interaction may be through the Internet or through a survey process, e.g. a Delphi. This also means that panel numbers need not be limited to 12-15 members but can be much larger. Panels can also meet in public sessions, although this tends to be reserved for those instances where panels wish to consult with a wider public. Finally, panels can, in some instances, be constituted for an indefinite period of time. This often occurs where the desire is to establish an 'independent' authority for dealing with long-standing challenges, e.g. global warming. Such panels report periodically, often on a specific topic or theme.


Sea also

Environmental Scanning & Monitoring
System Dynamics
Structural Analysis
Agent Modelling
SWOT Analysis
Trend Intra & Extrapolation
Modelling & Simulation
Gaming
Creativity Methods
Delphi survey
Backcasting
S&T Roadmapping
Critical & Key Technology Study
Scenario Building
Morphological Analysis & Relevance Trees
Cross-Impact Analysis
Multi-Criteria Analysis

Personal tools