EVALUATION TECHNIQUES OF PSYCHOSOCIAL
PROGRAMS AND PROJECTS
INTRODUCTION
The
evaluation of psychosocial programs and projects plays a central role in the
process of rationalization and is a basic element of planning. It is not
possible for them to be effective and efficient, if the triangulation of
evaluation techniques (quantitative, qualitative and participatory) is not
used, which are fundamental and important for improving the evaluation process.
Including
the evaluation techniques both in the programs and projects, shows that a
system has been established that can allow us to collect information and
knowledge, in order to carry out evaluation with relevance.
II. CONTENT
2.1 TYPES OF TECHNIQUES
Most of
the evaluation techniques of the projects come from other areas of knowledge
and have been adapted, with greater or lesser fortune, to the social field.
Even so, evaluation should not be confused with a scientific research process,
since its ultimate purpose is not to establish knowledge, but to allow us to
make decisions that improve our current and future interventions.
The
evaluation of the interventions of the projects must be seen, as well as an
applied inquiry that collects the advances made in other fields of knowledge
with the purpose of assessing the achievements of our development actions and
of increasing their quality and their positive effects.
There
is a methodological triangulation that contains information gathering
techniques, which can be summarized in the following table:
These
techniques have different characteristics while quantitative techniques
emphasize objective measurement, the demonstration of causality and the
generalization of results; qualitative techniques focus on the description and
understanding of human behavior in the frame of reference of the individual or
the social group in which it operates.
For its
part, participatory techniques are among the quantitative and qualitative. The
participatory approach has a fundamental characteristic, which is to start from
the involvement of the population in the development of techniques. However,
depending on the type of participatory technique we use, it can provide
objective data or other interpretative data.
2.1.1 QUANTITATIVE TECHNIQUES
Quantitative
techniques are based on objective measurement. They seek to obtain results that
can be extrapolated to a specific group of the population, either through the
selection of a sample of the population, a statistically significant number of
said population, or through the study of the entire group. The main
quantitative techniques, in the field of evaluation, are:
• The
survey.
•
Direct measurement.
•
Census.
A. The survey: The survey is one
of the most popular instruments in the evaluation of projects, it is generally
carried out on a sample of subjects representative of a larger group and is
carried out in the context of daily life, using standardized procedures of
interrogation, in order to obtain quantitative measurements of a great variety
of objective characteristics of the population. This technique has advantages
and disadvantages which are shown in the following table.
The
preparation of this tool occupies a precise place in the overall evaluation
process. It must be constructed after having formulated the criteria and
evaluation questions. Likewise, if the objectives of the evaluation are clear,
it will be easier to decide and choose the questions that should be asked. We
can distinguish eight fundamental phases of a survey:
·
Formulation of the objective of the
evaluation.
·
Review of project documentation.
·
Selection of the sample size.
·
Design of the questionnaire.
·
Administration of the questionnaire.
·
Selection and debugging of questionnaires.
·
Data processing and analysis.
·
Drafting the conclusions.
B. Direct measurement:
Direct measurement or quantitative observation consists of collecting data
according to a specific scheme and is widely used in contexts of development
projects. For the measurement to be meaningful it is very important to
determine in advance what properties or characteristics are going to be
considered and what are the dimensions that are handled. Direct measurement,
therefore, is directly related to the establishment of indicators.
Frequently,
direct measurement is related to the verification of objectively verifiable
indicators that are included in the project planning matrix. A common problem
occurs when the indicators are poorly stated or impossible to measure. The
steps to follow in direct measurement are the following:
·
Definition of the
objective to be measured by this technique.
·
Delimitation of the
properties to be collected.
·
Construction of indicators
and review of the indicators of the project planning matrix.
·
Contrast and
systematization of information.
·
Analysis of the data.
The use
of this technique facilitates the obtaining of very reliable data on physical
phenomena or processes, but does not manage to delve into the causes that cause
a certain situation. For this reason, direct measurement must be accompanied by
other, more valuable modes of analysis.
C. The Census: it constitutes a
crucial tool for a quantitative study of the social reality.It is
very useful to have carried out the census in the identification phase of the
project and to have incorporated its changes. The existence of this document
where the updated demographic data is collected is undoubtedly a great help in
the evaluation work.
The
census is a project that allows to know, on a given date, the cash of the
population in a territory with some of its characteristics:
•
Distribution by age.
• Sex.
• Age.
• Civil
status.
•
Family situation.
Its
execution is usually based on completing questionnaires, by the people affected
or some of them or even the census agent.
2.2.1 QUALITATIVE TECHNIQUES
These
types of techniques allow obtaining significant information about the different
perceptions and priorities of the conditions and processes that may have
affected the development of a program or project.
In
qualitative techniques, the existence of limitations in their usefulness must
be taken into account, such as the lower incidence of rigorous statistics, the
subjectivity in the data collection, the absence of a specific comparison group
that allows the establishment of causal analyzes, etc. Also, keep in mind that
the properties of validity and reliability of qualitative data will depend on
the technical ability, sensitivity and training of the evaluation team.
A. Structured observation: by
means of this technique, the registration of events can be obtained according
to patterns of behavior, social actions, attitudes and structures. Some of the
instruments used to record the observations are the following: daily, it is a
matter of noting the facts observed in the community and stories that they
express about their daily experiences; notebook, is a more structured tool and
where the annotations are made on the ground and when the facts occur; and maps
or devices such as video cameras, photographs, etc.
B. Interviews: in the open and
in-depth interview, the person who evaluates talks freely and spontaneously
with the person interviewed. This type of interview produces more information
when there is a climate of trust and motivation on both sides.
In
addition, it is important to know that when preparing an interview you must
take into account the choice of the place of the community and the appropriate
time. Also, the way to address the interviewee, the language used and even the
way of dressing.
C. Discussion groups:
Represents a peculiar way of gathering information, since it involves
establishing a group conversation dynamic. The discourse that is obtained is
produced in a social context more similar to the one given in real life than
the face-to-face action in an interview.
The
advantage of this evaluation technique is that they require less time and money
than individual interviews, without losing flexibility and providing the advantages
of group interaction. The drawback is that an artificial situation is created
for observation, also raising problems of generalization and bias.
D. Documentary analysis: it
consists in the search of information through the revision of diverse
documents: bibliography, previously elaborated statistics, various reports or
any type of material not directly produced by the evaluation.
2.3.1 PARTICIPATORY TECHNIQUES
These
techniques are characterized by requiring active and continuous interaction
between the evaluation team and critical agents throughout the whole process of
the project or program being evaluated.
Regarding
its use, it must be characterized by the relationship between the community and
the evaluation team based on the exchange of information and experiences for
mutual learning; professional people must adapt their language, attitudes and
behaviors to the context in which the evaluation is developed; the technical
staff starts to play the role of facilitator of a process through which the
beneficiaries express their vision of reality; and the methodology used should
be open, group, visual and comparative.
Regarding
the physical context, in which most of the participatory techniques are
developed (analysis of critical events, welfare ranking, classification
matrices or punctuation, Venn diagrams, flow diagrams, system diagrams, network
diagrams, maps and photographic monitoring) is called "workshop" and
is usually developed in a closed space, wide, allowing movement and collective
work, as well as the development of activities of group dynamization.
III. ANALYTICAL COMMENT
From my perspective, the techniques used for the
evaluation of programs are of vital importance, since they are necessary to
collect the information and with this power to perform the evaluation. On the
other hand, it is important to bear in mind that there are quantitative,
qualitative and participatory techniques, where it is advisable to use them
together, in order to generate reliable, objective and generalizable data for
the entire population. Finally, it is important to differentiate an evaluation
from an investigation, since the research looks for the knowledge and the
evaluation of the process or program.
IV. CONCLUSIONS
- The techniques in the evaluation of programs can be
quantitative, qualitative and participatory, which make up the methodological
triangle, used to collect information for the evaluation of the project or
program.
- It is important to take into account, what can be
evaluated, in order to select the appropriate technique for the collection of
information. On the other hand, if you want to make a complete evaluation of
the program, it is advisable to use all the techniques, to acquire objective,
reliable and generalizable information for the entire population.
- Finally, it is vital to know the evaluation
indicators that have been chosen, in order to make the selection of the most
appropriate evaluation technique.
V. LINKS
VI. BIBLIOGRAPHY
Gómez,
M. (2002). The evaluation of social services and programs: an element of
control or transformation of social action ?. Journal of Social Services and
Social Policy, (57), 81-94.
Robirosa,
M. (1982). Methods and techniques for evaluating social action projects: their
use in Latin America and the Caribbean. Retrieved from: http://unesdoc.unesco.org/images/0004/000479/047918sb.pdf
Strategic
Plan of the Third Sector of Social Action. Retrieved from: https://consaludmental.org/publicaciones/Guiaevaluacionprogramasctosyeciales.pdf
Tejada,
J. (2004). Program Evaluation. Recovered from: http://www.carcheles.es/export/sites/default/galerias/galeriaDescargas/diputacion/dipujaen/formacion/centro-documental/Evaluacixn_programas_de_formacixn.pdf
Starting from the concept of the evaluation of psychosocial programs and projects as a systematic way of learning from experience and using the lessons learned to improve current activities and promote planning through the careful selection of alternatives for future action, it is necessary to contemplate through the different instruments that qualitative and quantitative techniques provide us for the purpose of the evaluations that are considered.
ResponderEliminarNo technique should be considered as superior to the other, since both complement each other ("mixed" methods), and consequently produce a richer and broader understanding of the achievements of a psychosocial project or program.
"The best instrument in an evaluation is his own person." Ruth Guzmán