A focus group interview is an inexpensive, rapid appraisal technique that can provide managers with a wealth of qualitative information on performance of development activities, services, and products, or other issues. A facilitator guides 7 to 12 people in a discussion of their experiences, feelings, and preferences about a topic.

In public opinion research, groups are carefully constructed according to specified demographic criteria with the purpose of being able to make certain generalisations to the population as a whole.

Interviews collect narrative information on a theme from persons relevant to a project or program through a more or less structured discussion led by the interviewer. They are not usually used to provide data for a logframe or Performance Monitoring Plan, but they are very useful for formative M&E purposes.

Group discussion is a method consisting of elements of focus groups and individual interviews. They lack the focus on a particular key question, which is a characteristic of focus groups. Contrary to focus groups, they sometimes include people who belong to the same group, but also to other groups.

A questionnaire survey presents written questions in different formats to the entire population of people relevant to the project, or a sample of them.

Questionnaires can be designed to measure outcome variables such as satisfaction with a project, intermediate variables such as exposure to a project, and background variables such as age and sex. We can combine closed and open questions – open questions can be coded into closed categories afterwards, but we often enliven our reports with quotes of original phrases and statements expressed by the respondents.

We suggest choosing web questionnaires for collecting quantitative data, due to accessibility of computers and the Internet within institutions where your beneficiaries are employed.

A questionnaire survey presents written questions in different formats to the whole population of people relevant to the project, or a sample of them. We include web questionnaires in most of our work because they are an economical way of reaching a relatively wide population.

Questionnaires are specially designed to measure the outcome variables. We can target questionnaires either at a specific group of people defined by the client or from the general web population, e.g. via Facebook advertising.

Outcome Mapping  [C]  [C] (Carden, Smutylo, & Earl, 2002) is a new approach to project planning, monitoring and evaluation, which has been developed at the International Development Research Centre www.idrc.ca; designed by IDRC in consultation with Dr Barry Kibel of the Pacific Institute for Research and Evaluation as an adaptation of the Outcome Engineering approach. It can be used at project, program or organisational levels.

Mystery Shopping is a method where persons who are trained in a role of users/customers perceive and then evaluate the quality of services of the service provider and / or its competitors according to predefined criteria. The goal is to comprehend the quality of service, which allows you to identify those elements/aspects of the services that need to be improved. It is designed for those who want to provide first-class service to its users/customers, because it allows direct insight into the true quality of service, and effectively helps to resolve dissatisfaction ratings.

Work evaluation named 360° is a mechanism used for work evaluation, based on feedback from all those with whom the assessed individual is in contact – supervisors, coworkers, partners, subordinates and others in general. This method of gathering opinions for the purpose of evaluation is a great source of motivation for employees, because it offers a good estimate of how his or her work is observed from various perspectives. In traditional estimates, the supervisor meets the employee on a one-on-one basis, in order to discuss his or her work. Contrary, the evaluation 360° uses confidential information from people who can give specific data about how an employee does his job. After that, the employee and the supervisor meet to discuss the collected data.

The results are usually written up in a comprehensive report as a synthesis of the different methods employed.

First of all, an overall ”question-by-question” report for each respondent or respondent group is written. These reports can be provided from written notes and audio recordings, without making full transcriptions.

Individual reports can be provided for each focus group and/or for each subgroup of the population e.g. municipality. An overall report can also be provided.

If your data is spread across an area – for instance if you deal with different municipalities, school, regions, etc., our software put it on a map for you to re-tell your story from a geographical point of view.

Using questionnaires for information gathering enables quantitative analysis of data retrieved from web questionnaires, which ultimately provides the following advantages: 

• it provides a more objective baseline for examining future progress,
• background-variable differences are highlighted (e.g. differences between women and men, different stakeholders, etc.)
• objective methodology can further contribute with expressed opinions , so that they can be discussed within focus groups.

Content Analysis is a sophisticated social science technique which uses specialised software to support the process of extracting hidden meanings from a mass of information, collected through interviews with respondents. For example, we used it very successfully in a previous project with Sida, in order to identify the relationship between gender and poverty, through the eyes of beneficiaries[30].

Readers of our reports often find direct quotations as the best part of the reports; they bring life into the findings. Direct interview quotations are included in the price shown in the financial offer.

proMENTE uses a wide range of different on-site and off-site web analytics tools, such as Google Web Analytics, KISSmetrics, Facebook Insight, Twitalyzer, in order to obtain more comprehensive and useful data.

Web analytics is the measurement, collection, analysis and reporting of Internet data for the purpose of understanding and optimizing web usage. Using web analytics methods we can analyze following data: where are users (visitors) coming from, what they are doing on web site, when and where do they leave, how long do they stay on web site, how often do they come back to the web site etc. Combining web analytics methods with qualitative methods can help to better interpret web statistics, to better focus on user (qualitative) research and to get better certainty of findings. In campaign evaluation, web analytics can help to measure or to better understand the effectiveness of campaign’s concept, identification of target audience and their behavior on web and to verify user feedback.

How is the sample selected for the aforementioned methods? Sampling is a procedure which selects a smaller number out of a larger population (e.g. all citizens of a country or all elected mayors in one region) in such a way that the selected units, persons, geographical areas, etc. are sufficiently typical or representative of the larger population. Sampling involves deciding how many units to include in the sample and how to select them. Details of how the samples would be constructed are discussed with the client.

Often the client is particularly interested in minority sectors of the desired population, for example Roma householders or female mayors. In this case, such sub-groups can be oversampled, i.e. the proportion of them included in the sample is greater than in the population. Where statistical comparisons are made, special techniques have to be applied to correct for this oversampling.

Often, clients have a rough idea of what they are trying to find out in their research or evaluation project but need help to develop more precise research questions which are clear, useful, and can be answered with the chosen methods.

These are usually not the same as any actual questions put to respondents in the field – rather, they are more general, higher level and more directly relevant to the client’s overall goals and interests. In these cases, we sit with clients, read relevant project documentation and scientific literature, and finally agree on between one and four key research questions which will help to focus the research or evaluation project. These key questions will then determine the way the results are structured and presented in our report, in press releases etc.

Sometimes clients ask us to conduct statistical analyses, for example of a questionnaire, but what they really need when the analysis is finished is ”for someone to tell us what that actually means”.

This often happens when a questionnaire was developed without a clear set of high-level research questions. In these cases, we sit with clients to see what really interests them and how we can answer their questions and concerns with the data available. This is part of the report is usually more exciting than a more mechanical statistical analysis which deals with a questionnaire question-by-question or section-by-section.

Most of the research and evaluation we carry out for clients produces some really interesting results which the general public would be interested to know – but most often no-one ever tells them! So when the report is finished or nearly finished, we can provide for clients one additional product: a press release which expresses the most important findings and recommendations in a format familiar to media agencies and with messages which are interesting for the general public. Often clients find that, even if they don’t actually send the release to the press, this half- to one-page document is useful for them in explaining the project and key results to other partners and stakeholders.