© International Development Research Centre 1999
Canadian Cataloguing in Publication Data
Main entry under title:
Enhancing organizational performance : a toolbox for self-assessment
Issued also in French under title : Améliorer la performance organisationnelle.
1. Organizational effectiveness — Handbooks, guides, etc.
2. Sustainable development — Developing countries.
I. Lusthaus, Charles.
II. International Developemnet Resarch Centre (Canada)
HD58.9E61998 658.1 C99-980037-X
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, or otherwise, without the prior permission of the International Development Research Centre. A microfiche edition is available.
IDRC Books endeavours to produce environmentally friendly publications. All paper used is recycled as well as recyclable. All inks and coatings are vegetable-based products.
How to use this guide
CHAPTER 1 – SHOULD YOU CONDUCT A SELF-ASSESSMENT?
Why would you conduct a self-assessment?
Some typical reasons for conducting a self-assessment
Exercise 1. Identifying the reasons
Are you ready?
Exercise 2. Getting ready
Stop or go?
How long will it take?
Before you begin
Who will use the results?
Exercise 3. Mapping the stakeholders
Who should conduct the process?
Creating a team
Who will do what?
What does your team have? What does your team need?
Are there any other roles or responsibilities?
Exercise 4. Building the team
What’s the best way to manage the process?
What issues may arise at preplanning?
Have you forgotten anything?
CHAPTER 2 – PLANNING A SELF-ASSESSMENT
What is the unit of analysis?
What are the main performance issues?
Exercise 5. Identifying preliminary performance
What are the right questions?
Three kinds of question
Prioritizing your set of questions
What are the indicators?
A few words of caution about indicators
Exercise 6. Developing indicators
What are the best sources of data?
What methods will be used to collect data?
Exercise 7. Selecting methodologies
Exercise 8. Completing the self-assessment matrix
Have you forgotten anything?
CHAPTER 3 – CONDUCTING A SELF-ASSESSMENT
Planning for data collection
Collecting the data
Analyzing your data
Data sources and triangulation
Sorting the data
Basis for judgment
Communicating the results
Communicating during the process
Communicating after the process
Hints for conveying results
Writing a good report
What issues may arise at this stage?
How will the results be used?
Why results get shelved
CHAPTER 4 – DIAGNOSING THE PERFORMANCE OF YOUR ORGANIZATION
A framework for assessing organizational performance
Exercise 9. Analyzing organizational performance
Administrative and legal environment
Exercise 10. Understanding the organization’s external environment
Mission: stated and perceived
Exercise 11. Determining organizational motivation
Exercise 12. Examining organizational capacity
Exercise 13. Summarizing the performance issues
Tip 3. Questionnaires
Tip 4. Basis for judgment
APPENDIX 3 – Sample questionnaires
Sample 1. Staff questionnaire
Sample 2. Donor questionnaire
Sample 3. Cover letter for donor questionnaire
Exercise A1. Who is XYZ? What are the strengths and weaknesses of our internal environment?
Exercise A2. Our external context
Exercise A3. Dreaming about the future
Exercise A4. Beginning to bridge the gaps
This page intentionally left blank.
Since the International Development Research Centre (IDRC) was founded in 1970, it has funded research aimed at solving development problems. Our programs are driven by the conviction that sustainable development is possible only if people have the knowledge they need to secure their own well-being. IDRC has chosen to contribute to this goal by strengthening the research capacity of individuals and institutions in the South.
Capacity-building for individual researchers is relatively straightforward. Like many agencies, we at IDRC have learned to provide opportunities for training, collaboration with other researchers, and research-project funding, with considerable success. By comparison, donor agencies appear to be less clear about how to build capacity within institutions and organizations. Over the past few years, IDRC has increased its efforts to learn more about what makes research institutions effective and viable.
In 1995, we published Institutional Assessment: A Framework for Strengthening Organizational Capacity for IDRC’s Research Partners. This publication was translated as Évaluation institutionelle in 1996. These books were the product of collaboration between IDRC’s Evaluation Unit and Universalia Management Group (a Canadian management consulting firm). These books identified and brought together a series of questions on an organization’s external environment, its motivation, its internal capacity, and its performance. This series of questions was intended as a practical guide for assessing an organization’s ability to sustain itself and to meet its goals.
Working with Universalia, we field-tested the framework with IDRC’s Southern partners. We wanted to find out whether it could help diagnose an organization’s strengths and weaknesses, guide an organization in formulating a capacity-building response, and assist in assessing the outcome of an intervention. Our partners responded positively to the framework and contributed their own ideas about how it might be refined and used. Two important themes emerged from the field tests: organizations often prefer to assess themselves, rather than employ external evaluators; and organizations unprepared to conduct a full-scale assessment often wish to conduct a smaller, problem-based exercise focusing on a specific area or situation. This book is a response to these needs. It is a self-assessment guide with flexible tools and techniques that can be adapted and used severally or together.
Having received enthusiastic responses to our assessment framework, we are excited about this publication. We sincerely hope that it will open new possibilities for organizations to improve their performance and sustain their programs and that it will provide the basis for improved, more effective relationships between funding and recipient organizations.
Taken together, Institutional Assessment, Évaluation institutionelle, and this book, Enhancing Organizational Performance, are designed to assist IDRC and its partners in creating and maintaining organizations well-adapted to serving the needs of the world’s poor.
International Development Research Centre
Organizations usually conduct self-assessments to better understand their own performance and to address their strategic issues and thus, ultimately, to improve their performance. Organizational self-assessment is often used as a diagnostic, or a starting point, for organizations implementing an internal change or strategic planning process, or both. It can also be used as a way to engage in dialogue with other stakeholders, such as the Board of Directors or donor agencies.
Enhancing Organizational Performance was developed to accompany our previous publications, Institutional Assessment and Évaluation institutionnelle, which described our conceptual framework for assessing organizational capacity through various possible interventions, such as internal self-assessments or external evaluations by a funding agency.
Our model of self-assessment goes beyond measuring the results of an organization’s programs, products, and services. It integrates these results with the techniques of formative assessment, in which the assessment team becomes involved in helping the organization become more effective in meeting its goals. Focusing on organizational self-assessment, the framework has since been tested in a variety of organizations around the world.
This guide is not definitive. It has evolved as organizations share their experiences with us. As a matter of interest, if you are familiar with our book Institutional Assessment, you will notice that in this guide we have refined or expanded some of the terms used in our original model. This is a true sign of work in progress. We invite you to share your experiences and comments on this guide and the self-assessment process it presents.
We wish to thank the International Development Research Centre’s (IDRC’s) Evaluation Unit, whose preoccupation with providing support to the Centre in building research capacity led to the development of this framework and its testing in the field. We would especially like to thank Terry Smutylo and Cerstin Sander, who reviewed and improved the final drafts of this guide. We are also indebted to the IDRC program officers who identified the contexts in which we could work and who worked with us in many different ways to improve the model. IDRC’s strong and constructive linkages with its partner institutions made it possible to test the approach so effectively.
Most especially we would like to thank the staff and leaders of the organizations with which we worked. It was a privilege and an opportunity to work with
Dr Soekartawi of the Southeast Asia Ministers of Education Organization (Philippines); Dr Souleymane Soulama of the Centre d’études, de documentation, de recherches économiques et sociales, Burkina Faso; Dr Tade Aina, Dr Mkandawire Thandika, and Dr Achille Mbembe of the Council for the Development of Social Science Research in Africa, Senegal; Mr A.V.S. Reddy of the Centre for Integrated Rural Development for Asia and the Pacific (CIRDAP), Bangladesh; Dr Joseph Yao Yao and Dr Aïssata Camara of the Centre ivoirien de recherches économiques et sociales, Côte d’Ivoire; Dr Diagne Abdoulaye of the Centre de recherches économiques appliquées, Senegal.
We extend special thanks to Ms Rosalie Y. Say, Head of Training at CIRDAP, for her valuable contributions to the final draft of this guide; to Ms Julie Valentine, researcher; and to Ms Carroll Salomon, our editor.
This guide and the self-assessment process it presents can be used in a variety of ways. Our main objective was to provide you with a process for conducting an assessment, a framework for assessing issues, and some tools and tips to help you as you address an issue – big or small – that your organization is facing.
The guide is divided into four chapters and four appendixes. The first three chapters describe the major elements of the self-assessment process. The fourth chapter reviews the performance framework developed by Universalia Management Group and the International Development Research Centre and provides checklists and exercises to help you diagnose organizational-performance issues. Appendix 1 contains samples of self-assessment tools that some organizations have found useful in the self-assessment process, and Appendix 2 concerns the design of data-collection instruments. These tools and tips are referred to throughout the text, identified by a . We hope you will copy or adapt some or all of these tools to help you in your own self-assessment process and let us know what you think of them. Throughout the guide, we have included short stories or anecdotes about our experiences with self-assessment that we hope will bring you a sense of how this process works, as well as exercises, marked by a to help with various parts of the process. Appendix 3 contains sample questionnaires. In Appendix 4 you will find some sample self-assessment exercises.
In our work, some organizations used parts of this guide to address a set of organizational questions that managers often ask themselves: What business are we in? Are we having an impact on our client group? Are people happy in our organization? These organizations used the framework and some of the tools and managed to find problem-solving time in their regular meetings.
Other organizations, undergoing more dramatic shifts, opted for a more thorough review. They used the approach presented in this guide in a more systematic way and allocated special time and resources to undergo the self-assessment process. You are the best judge of how you can make the guide useful to your own organization.
In this guide and in our work, the term self-assessment refers to an approach to organizational diagnosis in which the organization has some measure of control over the assessment. The degrees of control range from assessments controlled
entirely by the organization to those commissioned by an entity external to the organization but in which the organization fully participates and owns the results. The control of the process appears at different points on this continuum in the experiences described in this guide.
Although we have described the self-assessment process in a certain sequence in this guide, we do not intend to suggest that all self-assessments follow these steps rigidly or in this order. Many organizations engage in self-assessment at different stages or in a unique framework that makes the best sense for a given organization. Sometimes an organization seeks an overview of its operations; at other times, it may have a specific problem to address, and a full-blown institutional assessment would only get in the way. These two approaches to self-assessment appear graphically in Figure 1.
At a recent meeting and discussion with one partner, we even worked out a quick-assessment model.
The agency’s major source of funding was to be cut within 9 months. The agency had to decide whether it should close down, cut down on staff and programs, or privatize. Each option had major implications for the agency’s programs and services, staffing requirements, and funding mechanisms. To gain an understanding of these implications, the agencyunder-took a self-assessment. With this knowledge, the organization was able to make an informed decision.
The centre had never carried out a comprehensive review of its strengths and weaknesses, although it had conducted numerous project and program evaluations. The need for an in-depth organizational review was triggered by
- The arrival of a new executive director;
- Increased pressure on the organization to identify alternative funding sources; and
- The organization’s desire to respond to the changing needs of its member institutions.
The process was also supported by one of its funders, who saw this approach as a way to build the capacities of the research centre.
EXERCISE 1. Identifying the reasons
Instructions: The following exercise can help members of your organization understand why they would conduct a self-assessment. Ask senior managers and other individuals or groups you think should be involved to reflect on this question and begin a list of their reasons for self-assessment. Their ideas can be shared and can become the basis for future decision-making. This exercise may be conducted on an individual basis, with the information then collated by one person, or it may be carried out as a group exercise, facilitated by either an internal or an external person.
In your organization, identify the three main reasons for undertaking a self-assessment at this time.
Organizations need to have a certain degree of readiness to engage in self-assessment. Although there is no set number of variables to assess, you will want to consider some readiness concepts before beginning the process:
- Cultural readiness – This means that your organization has an organizational culture in which it is acceptable to provide suggestions for improvement.
- Leadership readiness – This means that leaders support the self-assessment and the allocation of resources to the process.
- Resource readiness – This means that you are prepared to commit the resources (people, time, money, and technology) needed to conduct the self-assessment. Although the type and amount of resources you will need to undertake the assessment will not be known until you begin, management’s commitment to making the necessary resources available is indicated. Institutional self-assessments require more time from internal staff than externally conducted assessments do and typically involve more of the people on staff as well. This must be both understood and accepted. The assessment can have long-term benefits in terms of ownership and pace of implementation of change, but these will not be clear at the outset.
- Vision and strategy readiness – This means the organization has a sense of where it is going and how it should get there or has a desire to create a clearer vision.
- People readiness – This means the organization has people on staff who will champion the self-assessment process and be willing to work together through a process that may sometimes be ambiguous and will constantly be changing.
- Systemic readiness – This means the organization has or wants to have systems in place to provide the information needed to complete the data collection and support the self-assessment.
FACTORS THAT CAN AFFECT READINESS
You must have
- Acceptance of the process by leaders in the organization
- A champion
- Adequate internal resources (time and people) to do the self-assessment
- A compelling reason for doing the self-assessment
These are mixed blessings
- Other changes going on at the same time, some of which you cannot control
- An organization with a history of change
- Past experience with evaluations (positive or negative)
It’s nice to have
- Leaders with credibility
- A clear vision in the organization of where it wants to go
- Additional resources (financial) to conduct the self-assessment
These can be major barriers
- Past failures and frustrations with self-assessment
- Superficial motives
- Low levels of skills and capabilities
- Negative incentives for self-assessment
EXERCISE 2. Getting ready
Instructions: This exercise will help you to decide whether you are ready to engage in a self-assessment. Ask a group of managers in your organization to reflect individually on these questions. Then use the list as a tool to collectively discuss the organization’s readiness. There are no clear-cut answers to these questions. Your group will need to reach consensus on whether to proceed with the assessment.
- To what extent do the senior leaders in your organization support the change process? Do staff have confidence in the leaders’ ability to engage in change management?
- To what extent is any individual (professional or manager) willing to champion the process and capable of doing so?
- Is the organization facing the need to make strategic decisions and would a self-assessment help in the decision-making?
- Does the organization have a clear vision of where it wishes to go?
- Are major changes already going on within the organization that might slow down the process or interfere with it?
- Does the organization have access to resources to carry out the process?
- When was the last major organizational change? To what extent was it successful? Did it energize the staff or lower their morale?
- Do people inside the organization have adequate skills to undertake this process?
- To what extent are the leaders and staff comfortable with the use of organizational data? To what extent do organizational data exist?
- Is this a good time for change? Would another time be better? Are there future incentives for change to occur now?
- What are the positive, negative, neutral, or cultural implications of changing? Are people in your organization supported if they try new things?
It is impossible to be absolutely sure that your organization is ready for a self-assessment. The readiness diagnosis will be based on your best judgment. The purpose of this diagnosis is not to achieve a definitive answer but to consider the elements that will contribute to a successful assessment process. Conditions change, and factors may lead to a revised stop or go at various stages in the process.
At this stage in the process, some organizations may, for a variety of reasons, decide that they do not want to proceed – because they are not fully ready, or they wish to begin the process at some later date. It is important to recognize this and not engage in the process without support from inside the organization.
A self-assessment is a commitment of time and energy. It should provide you with knowledge of how to improve the performance of your organization. However, it is difficult to estimate how long it should take, as the process depends on many variables: the number of strategic issues you wish to explore, the scope of those issues, the type of data you need and their accessibility, availability of staff to be involved in the process, and the type of report you expect to present. These factors will affect the duration of the process.
The depth of data collection also depends on the quantity of information you need to understand an issue. This may not be obvious at the planning stage. As you begin to collect data on an issue, you may identify the need for more information. The deeper you go, the more you are led to uncover. The process will need to be guided by considerations of time, resources, availability of new data, and the importance of the issue.
One of the most critical elements affecting the scope of a self-assessment is the way your team will have to present its findings. It is quite time-consuming, for example, to produce a highly detailed report that links each finding to numerous data sources. This may or may not be necessary, depending on the audience and its expectations.
Understanding your audience will help you decide on the type of reporting required and the appropriate level of language. If the self-assessment is primarily for internal use, for example, then your report may comprise a set of brief notes, a summary of main lessons, or simply a memo. If, on the other
hand, you have to inform external stakeholders of the results, you will need to discuss with them the way they wish to be informed (see “Communicating the results” in Chapter 3, p. 38). Remember, you can conduct a thoroughly professional but less formal process to support findings. You and your organization will need to weigh how much you will or can invest in the process.
In our experience, some organizations carried out the process in 12 months; others, with a focus on specific issues for internal use only, completed the assessment in 1 month.
Finally, specialized activities in the self-assessment can be delegated to produce certain data. For example, a financial analysis can be completed by your organization’s accounting department, which can generate both the raw data and the graphics required to interpret them. Other elements can be contracted out. There are many options that you can use and control.
If your organization decides to engage in a self-assessment, you will have to do a number of things before carrying out the diagnosis. At this stage, you will probably want to
The audience for the assessment will influence its scope and the types of result expected. The sooner you identify the final audience, the easier it will be to identify their needs. You will need to consult with the audience throughout the process to obtain a richer set of data:
Stakeholders are individuals or organizations that will be affected in some significant way by the outcome of the self-assessment process or that are affected by the performance of the organization, or both. As shown in Figure 2, most organizations have a wide range of stakeholders, some more influential than others (either because they benefit from the organization, they fund some of its activities, or there are political reasons). Not all stakeholders have the same stake in the organization, and it is important to recognize the level of influence each stakeholder has on your organization. This will guide the data-collection process and allow you to identify the main sources of data.
Part of the planning phase is to determine roles and responsibilities. At this stage, you can form a self-assessment team to guide the process. Self-assessment teams have two major roles: strategic and operational.
In some cases, you will need to have two separate teams; in other cases, one team with two distinct roles. Regardless of the structure, select team members carefully, according to the role to be fulfilled. The strategic role will require people to provide guidelines and directions for the process and will oversee its overall quality. The operational role will require people to be more involved in the data collection, analysis, and reporting. The skills required for these roles are quite different, and several factors will influence your selection of team members: their availability, the complexity of the assessment questions, and the purpose and scope of the self-assessment.
Although the same individuals may play both strategic and operational roles on a self-assessment team, it is important to recognize that certain responsibilities will be more important at some stages of the process and less important at others. The strategic responsibilities are more conceptual; the operational responsibilities, more hands-on.
The size of the self-assessment team will also be important. Enough people should be there to provide a range of views, but not so many that it becomes difficult to make decisions. How decisions will be made and who will make them should be agreed on early in the process and will depend on the types of decision to be made.
CONFIGURING A TEAM
An internal self-assessment team
In one case, the executive director drew up a self-assessment team comprising several young professionals and led by the head of training. The team was responsible for all aspects of the self-assessment process. Its work was reviewed by the executive director.
A combination of internal and external team members
In another organization, the self-assessment process was led by a team of two very senior managers, who contracted external consultants to carry out specific aspects of the process, such as data collection and analysis of some issues. The team then integrated these external reports into its own synthesis.
A steering committee to guide the self-assessment process
In a third organization, the senior managers operated as a steering committee responsible for the strategic aspects of the self-assessment. The managers mandated diverse individuals inside the organization to conduct elements of the process.
The following criteria can help in selecting team members likely to commit to the process and see it through:
Self-assessment is an introspective process. Because the players are involved with the content of the assessment and have some stake in the organization, sensitivities can be strong. A clear definition of the roles and of the process at the outset can help ease tensions. Your organization needs to have a clear process for entering into a self-assessment, for managing the self-assessment, and for dealing with the expected and unexpected. On the other hand, self-assessment is not intended to be a scientific inquiry; rather, it is an organizational-development intervention and must be kept within the context of available resources.
Facilitation skills are important, to ensure voices are heard, to manage differences, and to create opportunities for consensus-building. The experience and knowledge of the team members enrich the self-assessment process. At the beginning, it is important to understand your team members’ levels of skill and knowledge and what they need or want to learn. This information will help you make decisions about how the work of the team should be allocated.
Some team members will have a deeper understanding of the organization and its issues; others will be able to provide comparative data from their experience in other organizations; and some will be more skilled in the self-assessment process. All of these skills and knowledge will be useful to the self-assessment.
Self-assessment is a learning process for all members. The team members you bring together are unlikely to have all the necessary skills for the self-assessment process. You will need to decide what skills or characteristics are most important, using the checklist below.
CHECKLIST FOR SKILLS AND KNOWLEDGE OF THE SELF-ASSESSMENT TEAM
__ YES __ NO Does the team have sufficient skills and knowledge about assessment techniques (how to conduct an interview, how to prioritize questions, where to look for data, how to analyze data, etc.)?
__ YES __ NO Does the team have sufficient skills and knowledge about the functioning of the organization?
__ YES __ NO Does the team have sufficient skills and knowledge about stakeholders’ needs and the environment in which the organization operates?
__ YES __ NO Do team members have the time to devote to the self-assessment, in addition to that needed for their normal work?
__ YES __ NO Do team members have the facilitation skills required to conduct a self-assessment?
In addition to the roles of the self-assessment team itself, the roles and responsibilities of other organizational members may need to be clarified. The Board of Directors, for example, may be required to participate in data collection or follow-up.
One should also consider the roles of external stakeholders – for example, external funders who support or partially support the self-assessment; and external professional consultants who may have been contracted to support part or all of the process.
The earlier the roles and responsibilities of these other players are defined, the easier it will be to proceed. To be effective, the process must be transparent, participatory, and empowering; above all, it must be a joint
learning process. You may need to be creative about using local and external resources. You may want to consider outside resources to save time or to handle very sensitive issues.
HANDLING SENSITIVE ISSUES
Taking advantage of influential team members
In one organization, the executive director’s role was voluntarily minimal during the self-assessment process. However, he did play a crucial role, ensuring that important stakeholders would provide needed data: he was quite influential and respected in his region, and he personally called stakeholders and encouraged them to respond to the questionnaire sent by the operational team. Needless to say, the response rate increased!
Gaining historical perspective
In one organization, in addition to the operational team and the management members, the self-assessment involved a former executive director (the founder of the centre), who was able to provide an historical perspective on many of the issues. This individual’s role was invaluable.
Instructions: This exercise can help you think about the best combination of people to bring into your team. Consider the questions and then complete a grid like the one shown here.
- In your organization, who will be on the team?
- What value will they add to the team?
- What roles are foreseen for them in the process?
A self-assessment team brings together people for a short time. It is important for the team members to discuss the process and set rules for how they will operate.
QUESTIONS TO ASK ABOUT THE MANAGEMENT OF THE ASSESSMENT PROCESS
Questions to ask
How will the team get beneath surface issues in discussions, particularly on sensitive issues?
Competition vs collaboration
How can the team promote a climate in which the members support rather than undercut each other?
Protection of confidentiality
How can the team see to it that all the information is made available to all team members and that confidentiality is maintained?
Freedom of expression
To what extent should team members be free to express feelings and opinions during the self-assessment process?
Wise use of resources
How will the team maximize the use of individual members and the combined resources of all the members?
Win–win conflict resolution
To what extent will the team confront emotional or intellectual conflicts? On what bases will conflicts be addressed?
In a self-assessment, the goal is to present the positive aspects of the organization, as well as the problems that may have been unknown or unclear or that have not been addressed or talked about. The self-assessment process takes for granted a spirit of openness and a willingness to divulge vulnerabilities and to acknowledge and examine organizational strengths and weaknesses.
However, self-assessment is not necessarily a notion accepted in all cultures, and group discussion of the issues may be uncomfortable for people in some cultures or organizations. Fear of the unknown can create tension and sometimes resistance.
In our experiences with self-assessment with a variety of organizations, we found several recurring concerns or obstacles:
Although these problems have no simple solutions, it is important to consider how they could affect the self-assessment process. The team will need to manage conflict using an approach appropriate to the organization. Conflicts can often be prevented by holding regular dialogue sessions and by taking advantage of opportunities to engage people in the process, both formally and informally.
DEALING WITH CONFLICT
At one organization, a member of the self-assessment team had had a difficult experience in the past with an external self-assessment consultant. Because of that experience, the member initially showed a great deal of resistance to the process this time. The new consultant had to acknowledge and deal with this resistance before the self-assessment could actually move forward.
Clarifying required skill set
One institution had many researchers skilled in their own areas of research but less skilled in self-assessment. Although data collection, analysis, and communication of results are important in both research activities and self-assessment, they differ in important ways: in scope, design of instruments, and expectations of reliability. Until these differences were clarified, the self-assessment team faced a group of researchers who were very uncomfortable with being made responsible for the data-collection and data-analysis segments of the self-assessment. A consultant facilitated this process of clarification.
Resolving time conflicts
In one organization, the self-assessment was undertaken officially, and everyone was informed. In practice, however, the staff members responsible for the self-assessment did not have enough time to both conduct the assessment and continue their normal professional activities. Ultimately, the team asked management to resolve this issue.
You may find that developing a self-assessment matrix makes the task of organizing a self-assessment easier. Developing the matrix will help you
SAMPLE SELF-ASSESSMENT MATRIX
The following sections will guide you through the steps of creating a self-assessment matrix in detail. The steps are based on our framework for diagnosing organizational issues affecting the performance of an organization, developed in our earlier work, Institutional Assessment. If you are not familiar with the performance framework, you might want to read Chapter 4 before continuing. Chapter 4 provides a detailed breakdown of the areas of performance you may need to assess, as well as some sample questions.
As you begin the process of planning an assessment, you can expect to undertake some of the following activities:
Organizational assessments can be conducted at many levels. Your assessment may examine the entire organization, a department, or a work unit – these are all, in fact, organizational units you can assess. You must define the unit of analysis at the outset, as many of your subsequent decisions will depend on this decision.
One of your self-assessment team’s first tasks will be to identify the most important issues affecting the organization – issues that the organization needs to resolve to improve its performance. These key issues will be the “strategic questions” of the self-assessment.
Typically, organizational issues centre around effectiveness (how well your organization is performing in achieving its mission), efficiency (how well it is using its resources to reach its mission), relevance (how well your organization’s mission continues to serve the purpose of your various stakeholders), and financial viability (whether you have adequate funding to ensure that your
organization can continue to perform in the short and long terms). You may identify issues in any or all of these areas. Many organizations tend to focus on effectiveness. Research institutions, for example, tend to focus on their research-and-publication issues, as these are the core of their business, their raison d’être. Nevertheless, you should consider all four performance areas.
The self-assessment team or committee will also need to prioritize the issues to decide the depth of analysis each one warrants. Although it is important for you to identify key strategic issues at the planning stage, it is equally important to recognize that issues will become more sharply focused as the process of self-assessment unfolds and that the balance of depth of analysis among the issues can change.
Even before you collect any systematic data, you will be able to find people in the organization with some notion of the performance issues it is facing. Senior leaders are a good source, and by asking them to help identify issues you build their sense of ownership of the process. However, everyone in the organization will have potentially useful views.
One approach to identifying the main issues is to ask several appropriate people inside the organization to reflect on issues they feel are important and to describe why they believe these are important issues. A few examples of issues and the reasons people gave for their concern are given below.
ISSUES AND REASONS FOR CONCERN
Major performance issues
Why these aspects of our performance are a concern
Are we fulfilling our mission adequately?
We are at a turning point in our organization’s history, at which strategic decisions need to be made
Are we using our resources in the best way to fulfill our mission?
The organization is showing a decrease in its ability to fulfill its mission
Are our mission, programs, and services still relevant to those who benefit from them and those who support them?
Stakeholders (staff, funders, Board of Directors) express dissatisfaction with the organization
Are we financially viable or should we pursue additional sources of financing and funding?
EXERCISE 5. Identifying preliminary performance issues
Instructions: We suggest you make several copies of this exercise and distribute them to your team members and the managers of the organization. Ask each person to complete the chart below. Then, as a group, you can compare and discuss your individual responses and “brainstorm” the priorities until you reach reasonable consensus on the main issues.
In the first column, fill in what you believe to be the major organizational-performance issues. In the second column, fill in some of the evidence or reasons you have for believing these are problems.
Once your team has identified the main performance issues, you will need to develop key questions about those issues to guide you in your data collection. A variety of approaches can be used to develop questions. This section suggests some starting points, but you can modify these to suit your needs and nature of your team.
How you write the question will depend on what information you need to understand the issue. An example is given below.
SAMPLE QUESTIONS REGARDING EFFICIENT USE OF RESOURCES
Type of question
Efficient use of resources
How were funds allocated this year?
Were funds disbursed according to standard budget procedures?
Was the planned budget appropriate to the needs of the organization?
When the team is satisfied that it has developed a satisfactory set of questions on the issues, you will need to prioritize these questions according to the following factors:
At this point in the development of your self-assessment matrix, the team will need to decide what type of answers it needs for each of the assessment questions – in other words, what type of information will satisfy the team? This is the time to develop indicators.
An indicator is a measuring device that allows you to clarify and measure a concept. Indicators make a concept more tangible, give you something to measure, and allow measuring to take place over time. They can help you clarify what it is you really want to know. As an example, we often ask about the size of an organization. Size is an important organizational concept, but it is also quite ambiguous. What is a good indicator of size?
The process of discussing and developing indicators can help team members reach a common understanding of what is being measured and why.
Indicators are used in a wide variety of contexts (such as those of planning, monitoring, and evaluation) and in all functional areas (such as those of finance, programing, and infrastructure). Indicators help you to describe organizational performance and the capacity, environment, and motivation driving organizational performance.
Indicators can be quantitative or qualitative. The difference between quantitative and qualitative indicators is sometimes subtle, and they are sometimes mixed together. In general, quantitative indicators are numeric representations of a concept (for example, the number of refereed research articles written as a result of a project). Qualitative indicators are less tangible. They are often an individual’s perceptions of a situation and are not always easy to count (for example, descriptions of the ways people found the research useful).
Indicators are also either direct or indirect. A direct indicator measures the concept itself – if you want to know staff size, for example, you count the number of staff. Indirect indicators measure something that happens as a result of the concept.
AN INDIRECT BUT APPROPRIATE INDICATOR
Some researchers were studying access to clean water in a rural region in Africa. They found that the most reliable indicator of a village’s access to clean water was the school attendance of the girls in the village!
Well, in that region, young girls were sent to collect water for the family. If the only clean water was far from the village, the girls had no time to attend school. Therefore, the more young girls in class, the more likely it would be that village had ready access to clean water.
Indicators may be general or specific: they may work in any situation at any time (for example, staff size), or they may need to be adjusted to a particular context (as demonstrated in the questions about size, listed above). Also, even in a particular context, indicators can be transitory, even seasonal.
INDICATORS MATURE WITH THE ORGANIZATION
In one organization, the executive director made an interesting link between the age of the organization and the level of sophistication of its indicators:
“When we began,” she said, “all we cared about was staying afloat, and our main indicator was having enough money to run our programs. After a few years, we began to serve a broader clientele and started monitoring whether they liked our services (we introduced a client-satisfaction survey). It wasn’t until a few years ago that we started worrying about the costs of treating each client and the increase or decrease of funds from each funder.”
To identify indicators well, you have to develop a clear picture of what you are trying to measure – indicators shouldn’t be the starting point – and even then there are a few other things you need to be careful about.
First, indicators often express the values of the people who develop them. The ways one measures a concept can give that concept importance in an organization and can even change organizational activities, for better or for worse. If, for example, a research institution or university rates researchers or professors on the number of experiments they conduct or articles they write, someone might perceive this as encouraging quantity rather than quality of research. Or, if a funding agency decides that the relevance of a particular social-service agency should be measured by the number of people it serves in the community, this might lead to a reduction in the time spent with each person.
Second, it is sometimes difficult to develop adequate indicators to measure the complex dynamics in organizations – often indicators are too simplistic and need to be combined. Most organizations develop a set of carefully considered indicators and modify them over time as they analyze their results.
Third, indicators may sometimes point out an organizational paradox or contradiction; that is, they give conflicting signals for the same concept. For example, diversification of funding could be seen as both a positive indicator and a negative indicator. On the one hand, it is a sign that an organization is financially viable and not overly reliant on one donor. On the other hand, having several donors (each with its own evaluation frameworks, indicators, and expectations) can lead to fragmented program vision and evaluation requirements, which can make it more difficult to develop coherent programs.
In all the organizations we worked with, the most difficult part of the diagnosis was identifying indicators. Sometimes this was because of an abundance of indicators and the difficulty of finding ones that really mattered. In one case, described below, there was another, far more interesting reason.
INDICATORS CAN LEAD YOU IN DIFFERENT DIRECTIONS
One regional centre had received funding to develop a resource centre (library) to serve the region. This gave rise to debates about the best indicators to measure performance. Some wanted to measure inputs, such as the number of new books acquired and new computers installed. However, given the centre’s mandate to promote regional cooperation and to support local government, others wanted more output-oriented indicators, such as the increase in the number of users from different countries. As you can imagine, chosing inputs rather than outputs as indicators would lead to very different assessments of performance!
The management group reviewed the mission of the organization and decided that the use of the centre by people from different countries was a better indicator of the organization’s success in meeting its objectives. However, not everyone readily agreed to this, as this indicator measured something beyond the centre’s control.
EXERCISE 6. Developing indicators
Instructions: This exercise can help your team discuss and develop indicators. To get a sense of how different people view the same issue, it might be a good idea to work individually on developing indicators and then share your ideas.
Select one of the performance issues you identified in Exercise 5 (p. 20). Develop three to five key indicators you think should be monitored.
Have you monitored these indicators in the past?
Does your organization have the ability to retrieve the information needed to monitor these indicators? If not, what would you have to put in place to collect the information – and is this possible?
Essentially, data can be collected from two sources: documents and people. Document sources can be internal (financial statements, annual reports, human-resource policy, program-planning documents, strategic plans, promotion brochures, evaluation reports, etc.) or external (country policies, legislation, media, donor’s reports, etc.). Here, the important point is to make sure that you review the documents that are appropriate for your strategic questions.
Data can also be obtained through people, either individually or in groups, either directly through conversation or indirectly through questionnaires. Who to meet is always a question. Ideally, you want to meet as many people as possible who can provide you with relevant information, but time constraints, political sensitivity, people’s availability, and geographical location can limit the number of people you have access to. Deciding who will be your best sources of data is like identifying your key stakeholders.
When the time comes to make this decision, refer back to Exercise 3 (p. 9), in which you identified your major stakeholders. For an organizational assessment, you will definitely need information from senior management. You may also decide to collect information from staff or nonstaff (professional, support, board members, volunteers). Depending on your strategic questions, you may also want data from outside people, such as your beneficiaries, members, and donors.
In trying to match your sources of data with the issues, you may have to make some trade-offs. In seeking the best sources of data, you will also need to consider what is realistic, feasible, and acceptable.
You can collect data in a variety of ways, such as through surveys, interviews, document reviews, and focus groups. To use these data-collection methods your team will have to develop the appropriate instrumentation.
OVERVIEW OF DATA-COLLECTION METHODS
- This involves a printed or electronic list of questions
- This is distributed to a predetermined selection of individuals
- Individuals complete and return questionnaire
- This involves personal interaction
- Interviewer asks questions, normally following a guide or protocol
- Interviewer records answers
- This is like a face-to-face interview, but it is conducted over the telephone
- Interviewer records responses
Group technique (interview, facilitated workshop, focus group)
- This involves group discussion of predetermined issue or topic
- Group members share certain common characteristics
- Facilitator or moderator leads the group
- Assistant moderator usually records responses
- This can be conducted in person or through teleconferencing if available
- This involves identification of written or electronic documents (reports, journals, etc.) containing information on issues to be explored
- Researchers review documents and identify relevant information
- They keep track of the information retrieved from documents
WHICH DATA-COLLECTION METHOD WILL WE USE?
Use a surface-mail or faxed questionnaire survey when
- The target population is large (for example, greater than 200)
- You require a large amount of categorical data
- You want or require quantitative data and statistical analyses
- You want to examine the responses of designated subgroups, such as male and female
- The target population is geographically dispersed
- You want to clarify your team’s objectives by involving team members in a questionnaire-development exercise
- You have access to people who can process and analyze this type of data accurately
Use an e-mail or web-page questionnaire when all of the above conditions are met and
- You have the appropriate software and knowledge of this method
- Your respondents have the technological capabilities to receive, read, and return the questionnaire
- Time is of the essence
- You want to provide the option of typing long answers to questions
- You want to reduce production and dissemination costs
Use face-to-face interviews when
- You need to incorporate the views of key people (key-informant interview)
- The target population is small (for example, less than 50)
- Your information needs call for depth rather than breadth
- You have reason to believe that people will not return a questionnaire
Use telephone interviews when
- The target population is geographically dispersed
- Telephone interviews are feasible
Use a teleconference interview when
- The target population is geographically dispersed
- Teleconferencing equipment is in place
Use group techniques when
- You need rich description to understand client needs
- You believe that group synergy is necessary to uncover underlying feelings
- You have access to a skilled facilitator and data recorder
- You want to learn what the stakeholders want through the power of group observation (one-way mirror or video)
Use document reviews when
- The relevant documents exist and are accessible
- You need a historical perspective on the issue
- You are not familiar with the organization’s history
- You need hard data on selected elements of the organization
Before selecting a method and developing instruments, however, you will want to consider the following:
YOU MIGHT BE IN FOR A SURPRISE
In one organization, the support and cleaning staff had never been invited to participate in any aspect of organizational life.
We experimented during the self-assessment process and chose to seek their opinion through focus groups, as some of the staff could neither read nor write. We trained a local person to conduct a focus group in the local language.
Because this was a completely new idea in this organization, we warned the facilitator that he should invite between 20 and 25 persons to get a turnout of 10–12 persons. We also told him that the focus group might not last the whole 90 minutes.
Well, all 25 people showed up, and the focus group lasted 2 hours!
“Never,” said a janitor who attended the session, “had anyone asked me anything about this organization,” even though he had been working in it since the beginning and probably had the biggest corporate memory of the group!
KNOW YOUR AUDIENCE!
In assisting with a self-assessment process in one institute, we had considerable time constraints. Because we were two facilitators and had only one afternoon to meet 16 professional staff members, we decided to conduct two focus groups.
What we didn’t realize was that the culture of this centre was unconducive to group sharing, and the issues we were examining were perceived as very sensitive.
One staff member came to us at lunch and asked us each to conduct, in 2 hours, eight 15-minute individual interviews, as the professional staff would not open up in a group setting. Even these short individual interviews provided rich data for the assessment.
STRENGTHS AND WEAKNESSES OF THE MOST COMMON DATA-COLLECTION METHODS
SURFACE-MAIL OR FAXED QUESTIONNAIRES
- Is highly efficient for routine data collection with a large number of respondents
- Lends itself to quantitative analysis and the use of powerful descriptive and inferential statistics
- Records individual comments and perspectives in the respondent’s own words
- Enables use of a large number of questions
- Is cost-efficient and timely
- In many parts of the world, people are familiar with this type of data collection
- Requires extensive planning and pretesting of instrument
- People might not respond, because of “questionnaire fatigue” – leading to nonresponse bias
- Permits no great depth in responses
- Response rate is typically low
- Misunderstanding of the questions is possible – leading to response bias
- Accurate wording requires a cultural and contextual understanding of respondents
ELECTRONIC QUESTIONNAIRES *
- Is quick to send, and erroneous addresses are caught within seconds
- Is quick and easy for the respondent to complete and return
- Is extremely cost-efficient if you have the technology
- Software enables you to control the visual presentation of web-page questionnaires and eliminates the need for data entry (not with e-mail questionnaires)
- Reminders, follow-up activities, and acknowledgments are quick and easy
- If the person receiving the questionnaire is away, an associate who is reading the mail may inform you – so you are aware of why the questionnaire has not been answered
- Assumes people have access to and can use a computer and e-mail or the Internet
- You cannot control for the visual appearance of the e-mail questionnaires as received
- Not everyone regularly checks his or her e-mail
- E-mail questionnaires demand more from the respondent
- Technology intimidates some people; they will not use it but may request that a hard copy be faxed or mailed
- Because electronic questionnaires are new, you may have to duplicate your efforts and contact some people electronically; others, using hard copy
- Requires a person with the knowledge and skills to develop electronic questionnaires
- Allows for discussion
- Allows presenter to focus discussion
- May be inefficient use of team time
- May fail to focus on key issues
- Gives limited time for in-depth questioning
* The strengths and weaknesses of “surface-mail or faxed questionnaire” (above) also apply to this method of data collection.
STRENGTHS AND WEAKNESSES OF THE MOST COMMON DATA-COLLECTION METHODS – continued
- Shows value placed on individual client
- Allows for in-depth analysis and pursuit of details geared to each respondent
- Few respondents refuse to be interviewed, leading to 100% response and good validity for the sample interviewed
- Personal nature may lead respondents to say things just to please the interviewer
- Requires careful planning of questions when multiple interviewers are used; also requires training
- Validity relies on skilled interviewers
- Is logistically difficult to arrange for efficient interviews
- Is time-consuming for all parties, expensive
- Is often difficult to analyze in ways that give clear directions
- Has many of the advantages of face-to-face interviews but at considerably lower cost
- Telephones permit rapid coding of responses on paper or computer
- Protocol and answers can be computer driven
- Some respondents consider it intrusive
- The interviewer often cannot maintain the respondent’s attention for an extended period
- Uses group synergy to maximize recall and highlight the diversity of perspectives
- Provides rich qualitative perspectives
- Group process can uncover underlying attitudes
- Allows you to reach a large number of people in a short time
- Requires extensive question and logistics planning
- Depends on the team’s having a skilled group facilitator
- Does not easily lend itself to quantification
- Participants sometimes feel that they cannot speak openly
- Is economical
- Provides a good source of general background
- Is unobtrusive
- May bring issues to the surface not noted by other means
- Information may be inapplicable, disorganized, unavailable, or out of date
- Could be biased because of “selective survival” of information
- May require research efforts
- May turn up unreliable statistics
- Provides eye-witness accounts
- Allows comparison of words and deeds
- Is sensitive when used by a trained observer
- Occurs in a natural setting
- May create artificial situations
- May give inadequate sampling of observed events
- Costs a lot in time and personnel
- May reflect observer bias
Working across the self-assessment matrix, your team has identified issues, developed key questions, and designed indicators. The next step will be to determine the best source of data for each indicator and the method to be used to collect the data.
Here’s an example. To help them determine the effectiveness of their research institution, a self-assessment team developed the following question: “To what extent is the institution producing high-quality research?” One of the indicators they developed was the number of research papers published in refereed journals each year. Their matrix looked like the one below.
Working with the indicators you developed in Exercise 6 and the resources available to you, what sources of data and methods for data collection would you use?
SEE APPENDICES 2 & 3
See tips for designing questionnaires, conducting interviews, etc., in Appendix 2. Also, see Sample 1 (staff questionnaire) and Sample 2 (donor questionnaire) in Appendix 3 (p. 113).
EXERCISE 8. Completing the self-assessment matrix
Instructions: Now that you have reviewed the whole matrix, step by step, you are ready to complete the entire self-assessment matrix for your organization.
Make a matrix for yourself like the one shown below. Complete the matrix, using the information you have already compiled by doing Exercises 5–7. It is easier to work across the sheet for each issue, developing questions, indicators, sources of data, and methods.
If you answer “no” to any of the questions below, review this chapter and complete the steps you need to get to “yes,” before continuing with conducting a self-assessment in Chapter 3.
__ YES __ NO Have you identified the major performance issues in your organization?
__ YES __ NO Have you identified the probable causes of these performance problems (external-environment, motivational, capacity factors)?
__ YES __ NO Have you identified a handful of indicators to help you to answer the questions you are exploring?
__ YES __ NO Have you identified the most important sources of data?
__ YES __ NO Are these sources accessible?
__ YES __ NO Will your approach to data collection match your time constraints, the skills of your team, and the culture of your organization?
__ YES __ NO Do you have the appropriate people to participate in data analysis?
Collecting the data will be easier and yield better results if the process is carefully designed. Through careful planning, you will easily see flaws in the design and be able to allocate data-collection resources strategically. Beware, however, of overplanning, which can decrease your team’s flexibility – a very important aspect of a successful self-assessment. Two useful tools for organizing data collection are the work breakdown and the person-day analysis. Actually, these tools are often used to organize an entire project (such as a self-assessment), not just the data-collection part, but this is a good place to introduce the concepts.
SEE TOOLS 5 & 6
See Tool 5 (work breakdown) and Tool 6 (person-day analysis) in Appendix 1 (pp. 90, 92).
The work breakdown should include one line for each group of activities or tasks (work package) small enough to be easily managed and for one person to be held accountable. For example, your self-assessment might include a questionnaire survey, and this will need to be designed, distributed, coded, and analyzed. These tasks may be broken into different work packages, or they may be grouped in other ways that make sense to you. The point is that each work package should have a definable output, a time frame, and a responsible person. Once you have done this for each data-collection task, you can
allocate the tasks to team members as illustrated in the sample person-day analysis (Tool 6 in Appendix 1, p. 92).
Note that a common complaint from people engaged in such planning is that not enough time is available to do all the things the team would like to do. This need not be a major limitation if you think about the available resources and the best ways to use them. You can collect data in many different ways, and all these ways have strengths and weaknesses. Use the constraint as an opportunity to talk about what you really need and of alternative ways to understand the organization. It is better to be selective in data collection and do it well than to try to do everything and do it poorly.
In your planning for data collection, assign team members in ways that will take advantage of their strengths. Some people have experience in conducting interviews; others are familiar with questionnaires and surveys. Make sure that responsibilities are balanced and tasks are shared. Remember that the power of the assessment will be enhanced if participants feel that they have ownership of the processes leading to the results.
Your team will now be ready to begin formal data collection. Actually, because they work in the organization, team members will already have a great deal of information, impressions, and opinions. This type of knowledge is useful for shaping questions and instruments, but it is not systematic data.
The data-collection process should offer an opportunity to begin afresh – to wipe the slate clean of impressions, some of which may be incorrect. Indeed, one of the greatest problems with self-assessment processes is that people base their conclusions on previously held beliefs. One of the strengths of the process, in general, is that people are surprised by what they find when they study their organization seriously. Everyone in the organization will have more confidence in the process if it is clear that conclusions have been shaped by the data they provided.
The team must take several measures to ensure that self-assessment data are valid:
Be sure to capture the data in ways that other members of the team can learn from. Summarize interviews, analyze the information provided by focus groups, tabulate the contents of documents, and so forth – so that other team members can read and understand the material. Because the analysis should involve each team member, you must provide each other with the basic materials you will each need to participate.
Actual data collection is a typical human process that will no doubt occur differently from what you envisaged when you drew up your neatly categorized data-collection plan. It seldom takes place in an orderly, sequential manner. In many respects, the data-collection plan must be flexible to accommodate respondent’s schedules and other obligations. Some people may be traveling when a questionnaire arrives. Others may be unable to attend a focus group at its scheduled time and place. Some individuals may not wish to be interviewed by a colleague, superior, or subordinate.
What do you do in such circumstances? Be flexible and have a team leader who can identify and solve such problems. Data collection is an organic process – it doesn’t have the constraints of a physical or chemical experiment in which all conditions need to be strictly controlled. Try to include everyone on terms that they can deal with. Remember that the data set is rarely complete and never perfect.
Learning from your data can be a challenge. You may find that you collected a great deal of information but have difficulty making use of it. You need to make some decisions along the way, starting at the design stage of your assessment, to simplify the collection and use of data. Keep in mind that to understand what is going on, you don’t need to collect every possible bit of information. You need to decide what is important, ensure variety in your data sources and types, and keep track of your original questions as a basis for sorting the data.
DATA NEEDS AND ACCESS TO SOURCES OF DATA
As the self-assessment process continues, think about your data needs and whether you have access to sources of data:
Access to a suitable spectrum of people, such as
- Clients, stakeholders, and institutional representatives,
- Researchers, teachers, and support staff, and
- Government officials;
Access to key documents, such as
- Institution handbook, calendar, and prospectus,
- Mission statement,
- Annual report and financial reports, and
- Program descriptions;
Opportunities to observe relevant facilities and activities, such as
- Buildings and grounds,
- Program or project sites, and
- Teaching; and
Opportunities to observe the dynamics among people, such as
- Nature of meetings, who attends, and who presides,
- Processes for teaching and learning,
- Nature of dealings with the institution’s clients, and
- How research is conducted and its dominant paradigm.
To understand the importance of the data you are collecting, you should be clear about the sources of the data. For example, if you are collecting data from only one group of people (such as senior management or line employees), then you need to take that into account in your analysis. The data will only reflect the views of that group, and you will have no way of knowing whether they reflect the general opinion of people in the organization. Ideally, you will collect data from more than one group, and you should also try to collect more than one type of data. If you only collect opinions but do not compare them with, for example, the organization’s financial data or project results, then you may miss some key information. If you collect data from different sources, you can triangulate your data to reduce inconsistencies and confirm results from more than one source. You can build a much stronger interpretation with information that comes from several different sources.
Finally, you should go back to your strategic questions and establish a framework to sort the data around the original issues. This is important because the actual questions you have asked may give you data to address more than one issue; it is also important because you can easily lose sight of your objective once you have a mass of information to deal with.
The data-collection process often yields some unexpected results. What do you do with data that fail to fit in with any of the questions or issues in your matrix? Your first consideration should probably be the importance of the information. If it seems clear to you that this is an important issue, or one suggested by several different data sources, you may want to add a question or subquestion to your matrix. If, on the other hand, the information is relatively unimportant – for example, it is the impression of only one person – you will likely just set it aside.
Because you can interpret a set of data in many ways, it is important to take into account potential differences in interpretation early in the process. Three main decision-making methods are generally used to make judgments about the interpretation of data:
SEE APPENDIX 2
See Appendix 2 (p. 93) for tips on analyzing group, interview, and questionnaire data.
Communication will be important throughout and after the self-assessment process. How you handle this issue will depend on who needs to obtain self-diagnosis information and how they wish to receive it. Therefore, very early in the process you should decide with each of the various audiences how they will get information on the self-assessment, both during and after the process. The matrix below can help you keep track of this aspect of the process.
COMMUNICATING RESULTS DURING AND AFTER THE PROCESS
Audience for the process
What the audience needs to know
How I will communicate about the process during the process
How I will communicate about the process after the process
[Example] Staff, Board of Directors, funder
[Example] Briefing at staff meetings, memos, progress reports, summary, verbal presentations
[Example] Final written report, verbal debriefing, videos, displays
Specific procedures should be designed to ensure that everyone involved in the data-collection process is kept informed and has an opportunity to be heard. These procedures might include
After the self-assessment process, your team will likely undertake some of the following activities:
Information can be conveyed in ways that are traditional or creative, or both. The team should be prepared to go further than the diagnosis, but you must keep in mind the issues on which you can reasonably expect action.
If you have agreed to write a report on the self-assessment for a group or groups in your audience, ensure it is clear and that it meets the needs of its audience. Nobody likes to write or read extremely long or complicated reports. A good assessment report:
It is difficult to predict how a self-assessment will go. In our experience in the past, each case seemed unique. However, some themes came up with more frequency than others, and we list these here so that you will know nothing is unusual about your own situation if you come across any of them:
REPORTING ON THE SELF-ASSESSMENT
Allow your institution to frame findings in a way that will be heard
We experimented with the self-assessment process in several countries in the same region. Although overall there were similarities in the culture of the region, the researchers had training in several different traditions.
In one research institution, the researchers had been trained in the Anglo-Saxon tradition; in another, in the French tradition.
Although no significant difference could be found in the depth of the analysis, major differences were found in the ways the researchers chose to report their findings. The ones with Anglo-Saxon training were more direct, even blunt, in their presentation of findings and recommendations. Those with a French training preferred to contextualize the process, the findings, and the recommendations.
Presenting the findings
In one organization, we suggested presenting findings in a very concise way; a brief statement of each finding, followed by a paragraph with supporting data.
The self-assessment team rejected this method for presenting the findings and suggested presenting them in a “softer” paragraph so that they would not “jump out” at the reader. As a rule, people gave feedback in a less direct manner in this region than in North America, and the report had to reflect that.
The message can be shared in many ways
In one organization, the self-assessment team indicated that it had to give a final report to various audiences and had to use different methods of communication for each:
- For one of their funders, the team wrote a substantive report;
- For internal staff, the team presented the main findings at a 1-hour staff meeting;
- For the Board of Directors, the team wrote a short version of the report, with an additional section entitled “Actions to be taken by the Board”; and
- As the 20th anniversary of the centre was approaching, the team included highlights of the report in the organization’s corporate profile, which it used to invite potential donors to support the organization.
Keep us posted
At one research centre, the beginning of the self-assessment process was announced at one of the daily staff meetings. All staff members had an opportunity to ask questions about the purpose, the expected results, and the reasons for starting the process. From then on, every few weeks throughout the process, the staff received a brief progress report on the self-assessment. The executive director received private weekly briefings.
When we shared a summary of these data with the management group, they denied that staff gave these data to the self-assessment team. They indicated that the staff members we had interviewed (8 out of 111 of their professionals) were unaware of the facts. Although the management group said they would provide us with exact data by the following day, they did not do so.
The self-assessment is not an end in itself but a process of diagnosis and reflection that should lead to action. Unfortunately, some organizations stop short of action. They stop once the diagnosis is complete and fail to act on the results. If you need to use the results of your self-assessment to improve your organization, make sure the results do not get shelved or ignored.
Some of the reasons why results get shelved are beyond your control, but others can be avoided through good planning and communication. Here are the four most common reasons:
WHAT HAPPENED NEXT?
One organization continued the self-assessment process with a 3-day strategic planning exercise, in which the self-assessment data were used to develop strategies. Another organization used its self-assessment report to develop a special session of the Board of Directors at its annual meeting. A third organization never fully completed the exercise, because of various changes inside the organization. Little follow-up was conducted, and the draft report may have been shelved.
This page intentionally left blank.
Identifying the main issues helps to define the focus of the self-assessment process. The following sections will help you develop deeper, more detailed strategic questions. We have provided a brief section on each component of the performance model; these have been adapted and refined from our earlier work, Institutional Assessment. Use the following sections in any order to focus on areas you wish to explore.
The International Development Research Centre (IDRC) and Universalia Management Group have constructed a framework to help organizations assess themselves. Our approach can help you clarify important issues and guide the collection of data to help you make decisions to improve your organization’s performance and capacity.
In brief, the framework encompasses four areas:
Your organization’s performance is made visible through the activities it conducts to achieve its mission. Outputs and their effects are the most observable aspects of an organization’s performance.
Ideas about the concept of performance vary considerably. Each interest group or stakeholder may have an entirely different idea of what counts. For instance, administrators might define your organization’s performance in terms of the amount of money brought into the organization through grants, whereas a donor might define performance in terms of your organization’s beneficial impact on a target group.
In our experience, very few organizations have performance data readily available. However, it is usually not difficult to generate this information from existing data or to develop mechanisms for gathering performance data.
Data gathering tends to be mechanical and technical. It is far more difficult to obtain consensus on the merits of particular performance data and indicators. It is even more difficult to arrive at value judgments regarding acceptable levels of quantity and quality for each performance indicator. The real questions are these: How does your organization define good performance? Does good performance help your organization attain its mission? The second of these questions is particularly important for organizations that have very diverse stakeholders.
When you are diagnosing your organization and its performance, the number and choice of indicators are critical. “Wise” organizations try to identify 10–15 key performance indicators that they can regularly monitor to assess their own performance. It is also wise to have a set of other variables to monitor as a barometer to help understand performance. These other variables may include employee morale, timeliness of financial information, economic indicators, absenteeism, and number of new funders.
YOU MAY ALREADY HAVE INDICATORS
Although we have often worked with organizations that never went through the process of thinking about their indicators, we have found that every organization has a unique set of organizationally appropriate indicators. Your organization needs to create its own indicator-monitoring list. Not all the indicators you develop will have the same importance, and you may also find that the appropriate indicators change as your organizational-performance issues do or as the organization evolves.
The effectiveness of your organization is the degree to which it moves toward the attainment of its mission and realizes its goals. Effectiveness, however, is not a simple concept. The basic difficulty in analyzing effectiveness lies in the fact that many organizations make multiple statements about their missions and goals. Sometimes these statements are in the organization’s charter; other times, in their strategic documents. Regardless of where you find these statements, you need a clearly defined guide to the raison d’être of the organization.
A BALANCING ACT
One research institution was caught in a dilemma that many funded institutions can relate to. External donors offered to fund it to carry out environmental-impact projects. Although fascinating and lucrative, these projects would lie outside the mission of the institute, which was primarily to promote economic and social-science research. In addition, the research institute would have to hire an expert in environmental issues to carry out the projects. If the institute accepted this funding, the results of the research would not be fully used by the national government, but the institute would gain revenues from external donors.
The dilemma for this organization was to understand the trade-off between ensuring its own financial sustainability and working toward its mission.
How effective is your organization in working toward its mission?
- The charter, mission statement, and other documents provide the raison d’être for the organization.
- The mission is known and agreed to by staff.
- The mission is operationalized through program goals, objectives, and activities.
- Quantitative and qualitative indicators are used to capture the essence of the mission.
- A system is in place to assess effectiveness.
- The organization uses feedback to improve itself.
SOME INDICATORS OF EFFECTIVENESS
- Number of clients served
- Quality of services or products
- Changes with respect to equality
- Environmental changes
- Quality-of-life changes
- Service access and usage
- Knowledge generation and use
- Collaborative arrangements
- Demand for policy or technical advice from stakeholders
- Replication of the organization’s programs by stakeholders
- Growth indicators in terms of coverage of programs, services, clients, and funding
An organization must be able not only to provide exceptional services but also to provide them within an appropriate cost structure. Performance is increasingly judged by the efficiency of the organization (for example, the cost per service, the number of outputs per employee, the number of outputs per person per year, the average value of grants per person). Whatever the overall size of the unit, performing organizations are viewed as those that provide good value for the money in both quantitative and qualitative terms.
How efficient is your organization in the use of its human, financial, and physical resources?
- Staff members are used by the organization to the best of their abilities.
- Maximal use is made of physical facilities (buildings, equipment, etc.).
- Optimal use is made of financial resources.
- The administrative system provides good value for cost.
- High-quality administrative systems are in place (financial, human resources, program, strategy, etc.) to support the efficiency of the organization.
- Benchmark comparisons are made of the progress achieved in the organization.
SOME INDICATORS OF EFFICIENCY
- Cost per program
- Cost per client served
- Cost–benefit of programs
- Output per staff
- Employee absenteeism and turnover rates
- Program-completion rates
- Overhead – total program cost
- Frequency of system breakdowns
- Timeliness of service delivery
Organizations in any society take time to evolve and develop, but they must develop in ways that consolidate their strengths. Organizations face internal and external crises, and no organization is protected from becoming out of date, irrelevant, or subject to closure. To survive, your organization must adapt to changing contexts and capacities and keep its mission, goals, programs, and activities agreeable to its key stakeholders and constituents.
Has your organization remained relevant?
- Regular program revisions reflect changing environment and capacities.
- The mission is undergoing review.
- Stakeholder-needs assessments are conducted regularly.
- The organization regularly reviews the environment to adapt its strategy.
- The organization monitors its reputation.
- The organization creates or adapts to new technologies.
- Innovation is encouraged.
- The organization regularly undertakes role analyses.
SOME INDICATORS OF RELEVANCE
- Stakeholder satisfaction (clients, donors, etc.)
- Number of new programs and services
- Changes in partner attitudes
- Changes in role
- Changes in funders (quality and quantity)
- Changes in reputation among peer organizations
- Changes in reputation among key stakeholders
- Stakeholders’ acceptance of programs and services
- Support earmarked for professional development
- Number of old and new financial contributors (risk of discontinuance, leverage of funding)
- Changes in organizational innovation and adaptiveness (changes appropriate to needs, methods)
- Changes in services and programs related to changing client systems
ARE WE ADEQUATELY BALANCING STAKEHOLDERS DEMANDS?
A social-service agency had to balance its ways of being relevant to its beneficiaries and funders:
- Beneficiaries – Because of increasing demands from the community to provide more home care for the elderly, the agency was tempted to put more of its resources into this type of service.
- Funders – One of its funders wanted the agency to increase its promotion and advocacy for voluntary work in and beyond the immediate community.
The Board of Directors and management were thus faced with two not necessarily convergent sets of demands from two important stakeholders.
To survive, your organization’s inflow of financial resources must be greater than the outflow. Our experience has shown that the conditions needed to make an organization financially viable include multiple sources of funding, positive cash flow, and financial surplus.
Is your organization financially sustainable?
- Existing funding sources offer sustained support.
- The organization consistently obtains new funding sources.
- The organization consistently has more revenue than expenses.
- Assets are greater than liabilities.
- The organization keeps a reasonable surplus of money to use during difficult times.
- The organization monitors finances on a regular basis.
- Capital assets and depreciation are monitored.
- The organization does not depend on a single source of funding.
SOME INDICATORS OF FINANCIAL VIABILITY
- Changes to net operating capital over 3 years
- Ratio of largest funder to overall revenues
- Ratio of cash to deferred revenues
- Ratio of current assets to current liabilities
- Ratio of total assets to total liabilities
- Growth in terms of number of funders, amount of resources mobilized, assets, capital, and revenues
- Levels of diversification of funding sources
- Partners hired to provide services on a regular basis
EXERCISE 9. Analyzing organizational performance
Instructions: This is a good time to go back and review the performance issues you identified in Exercise 5 (Chapter 2, p. 20). Afterward, fill in the table below to show
- The problematic aspects of your organization’s performance (effectiveness, efficiency, relevance, financial viability);
- Some of the signs (indicative issues) that tell you these are problems; and
- Some questions you could ask to allow you to assess the extent of each problem.
Your organization does not exist in a vacuum. It is located in a country and region to which it is inextricably linked. It operates within a legal and cultural context. These and other external-environment variables influence how your organization operates and what it produces. Such variables can shape the ways your organization defines itself and the ways it defines good performance. You undertake an analysis of the external environment to understand the external forces that help to shape your organization.
The key dimensions of the external environment that influence your organization are the administrative, legal, political, sociocultural, economic, technological, and stakeholder contexts. Each of these components of the external environment is discussed individually in this section.
We have not provided types of indicators for the external environment because these are specific to each organization and its context.
Some organizations’ staff members were a bit perplexed by our suggesting they assess the external environment: “There is very little we can do to change the external environment, so what is the purpose of assessing these external factors?” This statement is only partially true.
Generally, your organization will have a limited ability to change its environment, but the better you understand it, the better you can adapt to it and develop appropriate strategies. Most organizations are influenced by, and can influence, their environment.
Your initial task in mapping out the external environment will be to take a closer look at the current situation in terms of the organizations involved and their roles, influence, and effectiveness. You will need to understand the wider influences, such as laws and policies, that affect your organization.
ADMINISTRATIVE- AND LEGAL-ENVIRONMENT ISSUES
How is your organization affected by the administrative and legal environment?
To what extent
- Is your organization influenced by the rules of other organizations, institutions, and groups to which it relates or might be expected to relate?
- Is your organization influenced by expectations of consumers, policymakers, suppliers, competitors, and other organizations in its external environment?
- Are your organization’s objectives and activities influenced by governments, donors, and other organizations?
- Is your organization influenced by important sector rules and regulations?
- Do administrative norms and values in your country support or hinder the work that the organization intends to carry out?
To what extent
- Do the laws of the country support the role played by your organization?
- Does the legal framework support the organization’s autonomy?
- Is the legal framework clear?
- Is the legal framework consistent with current practice?
- Is the legal regulatory context conducive to your organization’s work?
- Does your organization monitor changes in the legal context which could affect the position of the organization?
The quality of government in your external environment is a key determinant of your organization’s performance and its prospects for change. The legitimacy, accountability, and transparency of this government will be centrally important to your organization.
How is your organization affected by the political environment?
To what extent
- Do the political and ideological trends of the government support the kind of work the organization does?
- Does the government system facilitate collaborative arrangements?
- Does the organization play a role in national or sector development?
- Does the organization have access to government funding?
- Does the organization have access to international funding?
- Does the organization have access to the government’s knowledge and publications?
- Do government policies and programs support the organization?
Every organization is to some degree affected by its sociocultural environment. It is important to develop an understanding of the wider public service and society in which your organization operates. This may include a better understanding of the inherent values of cultural and ethnic groups present in external communities.
How is your organization affected by the sociocultural environment?
To what extent
- Is equity in the workplace a social value?
- Does the organization account for the effect of culture on program complexity?
- Do values found in the sociocultural environment support the work of the organization?
- Does the organization have access to a pool of capable human resources to recruit staff?
- Does the organization analyze and link demographic trends to its work?
The economic environment comprises the forces and trends affecting the availability and worth of an organization’s resources. Is government spending more or less? If government is spending more, do its expenditures pertain to your organization? Is inflation under control or is it running rampant? Inflation decreases available resources. Similarly, it is important for you to know whether donors or other foreign investors are making more or fewer resources available. Government policy will affect all these conditions in the economic environment.
How is your organization affected by the economic environment?
To what extent
- Does the government’s economic policy support the organization’s ability to acquire technologies and financial resources?
- Is money available to do the organization’s work?
- Is the organization supported by donors?
Understanding the types and levels of technology in a society gives insight into the nature of the organizations that operate in that society. For instance, if your organization generally deals with Western models and concepts, you will depend on different technologies than organizations carrying out indigenous research. It is important to understand the level of relevant technology in the organization’s context and to consider the processes for introducing new technology.
Do systems in the wider environment support the technology needed for your organization’s work?
To what extent
- Is adequate physical infrastructure (power, telecommunication, transport) in place to support the organization’s work?
- Is the technology needed for your organization’s work supported by the overall level of national-technology development?
- Does the government system facilitate the organization’s process for acquiring needed technology?
- Is the level of human-resource development in your organization adequate to support new technology?
Stakeholders commonly include individuals, groups, or organizations with an interest or a stake in your organization. They typically include the Board of Directors, beneficiaries, suppliers, governments, staff members, unions, competitors, local communities, and the general public. Each of these groups can justifiably expect (or in some cases, require) your organization to responsibly satisfy its expectations.
Therefore, stakeholder analysis is an important source of information for a self-assessment. It is important to identify your stakeholders and understand their roles and their expectations vis-à-vis your organization. It is also important to assess their perceptions of the issues and challenges facing your organization, the resources (political, legal, human, and financial) they may contribute to help your organization address those challenges, their own mandates with respect to your organization, their possible reactions to some of your organization’s strategies, and the existing or potential conflicts between stakeholders.
Does the stakeholder environment support your organization?
To what extent
- Is the community involved in the organization?
- Are partners involved in the organization?
- Do governments value the organization’s products and services?
- Do governments request or use the organization’s products and services?
- Do similar organizations compete or cooperate with your organization?
- Do donors influence the organization?
- Do funders support the organization?
See Exercise 3 (mapping the stakeholders) in Chapter 1 (p. 9), Tool 2 (stakeholder assessment) in Appendix 1 (p. 86), and Exercise A2 (our external context) in Appendix 4 (p. 124).
EXERCISE 10. Understanding the organization’s external environment
Instructions: This exercise can help the self-assessment team understand how the organization relates to its environment. One interesting way to do this exercise is to have a brainstorming session.
How would you characterize the external environment of your organization in relation to the performance issues you have identified? Is it hostile to or supportive of your performance?
What specific aspects of the external environment do you think you should monitor?
Administrative and legal
Organizations, like people, have different rhythms and personalities. Each has a different purpose, or mission. Some are highly motivated by the opportunity “to do good,” whereas others are driven to perform by other forces, including the personal ambitions of key players. Each organization also has a unique working ambiance, or climate – a combination of purpose, history, and personality. The organizational concepts that drive your organization include its history, its mission, its internal culture, its incentives or rewards, and the widespread values and beliefs about the role your organization plays in society. We consider the first four of these – history, mission, culture, and incentives or rewards – as being the elements of organizational motivation.
Assessing the motivation of your organization can be a very sensitive task, as you are in fact digging into what the organization is. In our experiences with different organizations, we found that strategies to address changes in this area took a long time to implement. The older the organization, the more past history it carried and the more carefully it needed to approach changes in this area. Often, factors related to organizational motivation cause a change to fail.
Your organization’s history is charted in its important milestones – the story of its beginnings, rate of growth, awards of achievement, and notable changes in structure or leadership. Although the organization’s evolution or history is often expressed through formal documents – such as the charter, stated goals, objectives, and plans (strategic or otherwise) – it is also told in unwritten stories that can be highly motivating.
What are the memorable events in your organization’s history?
- The milestones
- The successes
- The crises
Your organization’s mission is its raison d’être. It answers certain questions: Why does this organization exist? Whom does it serve? By what means does it serve them? We often find, however, that these questions have two types of answer: the organization’s written mission statement (stated) and the mission that we hear about from members of the organization (perceived).
The mission statement is the written expression of the basic goals, characteristics, values, and philosophy that shape your organization and give it purpose. This statement is an attempt to distinguish the organization from others by clearly defining its scope of activities; its products, services, and market; and the significant technologies and approaches it uses to meet its goals. The strength of your organization’s mission is the degree to which the stated mission and the perceived mission are the same.
To what extent does the organization have a clear mission that drives members’ behaviour?
- A clear mission drives the behaviour of the organization’s members.
- The organization’s mission is related to its goals.
- The organization’s members have accepted its mission and feel they can ascribe to it.
- The mission is updated and linked to a set of goals.
- The key values and beliefs driving members’ behaviour are linked to the mission.
- New staff embrace the mission.
The culture of your organization is the sum of the values, beliefs, customs, traditions, and meanings related to mission fulfilment. These have developed over the history of the organization, and they make it unique, govern its character, and drive the organization.
The characteristics that distinguish your organization are part of its culture. The culture incorporates all of your organization’s symbols, myths, visions, pride, and the accomplishments of its past and present heroes. Stories about past successes and failures may illustrate values important to your organization (such as team work, gender equity, participation, transparency). These stories form a living history that guides the managers.
See Tool 4 (culture audit) in Appendix 1 (p. 89).
What aspects of your organization’s culture help it to fulfil its mission?
- Documents outline the organization’s values.
- The people in the organization identify with the organization’s values.
- People have good morale in the organization.
- People have a high level of commitment to performance in the organization.
- People in the organization have a positive attitude toward change.
- Functioning systems are in place to reinforce the organization’s values, such as those for promotions, incentives, and training.
- People in the organization exhibit good will toward each other.
THE “REAL” ISSUE
In one organization, poor performance was attributed to the lack of training and appropriate materials and technology. Although these factors affected the organization’s performance, the main issue turned out to be conflicting cultures inside the research centre:
- Some of the staff members had their own incentive systems and rewards, as they were paid by the university with which the centre was associated.
- Other staff members depended solely on the organization’s ability to generate research projects, and these members were more entrepreneurial.
Organizational incentives are the ways your organization’s system of rewards and punishments either encourages or discourages certain behaviours. Incentives are important to individual careers and to the overall success of the organization. It is important to assess how employees at all levels of the organization view the effectiveness of the incentive system.
Does the incentive system in your organization encourage or discourage members’ performance?
- People feel they are rewarded for their work.
- People are adequately compensated.
- Nonmonetary rewards support good behaviour.
- The incentive system is adequately managed.
- The incentive system is under ongoing review.
- People are equitably treated in the organization.
- The organization is consistent between what it rewards people for and what it says it will reward people for.
See Tool 4 (culture audit) in Appendix 1 (p. 89) and Sample 1 (staff questionnaire) in Appendix 3 (p.113).
EXERCISE 11. Determining organizational motivation
Instructions: This exercise can help you identify how your organization’s history and past and present culture affect its performance and are related to its environment.
To make this more specific to your context and to help your team generate more useful data, we suggest you consider Tool 4 (culture audit) in Appendix 1 (p. 89) and Sample 1 (staff questionnaire) in Appendix 3 (p. 113). You could adapt and distribute these instruments and use the data to respond to the question below.
Based on your culture audit and staff survey, how do you think the mission, history, culture, and values of your organization affect its performance?
In our framework, the capacity of an organization is considered in terms of seven main, interrelated areas that we believe are the foundation of the organization’s performance: strategic leadership, human resources, financial management, infrastructure, program management, process management, and interinstitutional linkages. Each of these areas contains various components, which range in importance among organizations.
In our experience, organizations often find capacity issues easier to identify and to address. This may be because it is easier to do something about them, whereas it may be impossible to change the external environment and difficult to change organizational motivation.
Similarly, donors may focus more on capacity-building because (1) organizations often request this type of support; (2) tangible changes can be seen in a relatively short time; and (3) an external body can make a contribution in this area (interventions to change the motivation or external environment may be unfeasible or inappropriate).
SEVEN AREAS OF ORGANIZATIONAL CAPACITY AND THEIR VARIOUS COMPONENTS
Leadership, strategic planning, governance, structure, and niche management
Planning, staffing, developing, appraising and rewarding, and maintaining effective human-resource relations
Financial planning, financial accountability, and financial statements and systems
Facilities management and technology management
Planning, implementing, and monitoring programs and projects
Problem-solving, decision-making, communications, and monitoring and evaluation
Planning, implementing, and monitoring networks and partnerships
Today, organizations need to be able to change quickly. They require leadership to link their internal and external realities so as to improve their own performance. We have used the term strategic leadership to convey these ideas.
In today’s context, leadership is no longer seen in terms of the great-man theory; it requires many of the organization’s members to take on leadership functions. Your leaders must have the ability to create and re-create long-term strategic plans and develop governance systems to support your organization’s survival and development, structures to provide the balance between control and flexibility, and a niche for your organization to ensure it has unique or value-added roles in the economy. Our experience has shown that strategic leadership is an exceedingly complex undertaking.
To us, strategic leadership includes leadership, strategic planning, governance, organizational structure, and niche management. Each of these components is discussed individually in this section.
Both formal and informal leadership can be found in many places inside your organization. Formal leadership is exercised by those appointed or elected to positions of authority. Formal leaders set the direction, ensure that tasks are done, and support resource development. Informal leadership, on the other hand, is exerted by persons who become influential because they possess special skills or resources valued or needed by others.
To what extent does strategic leadership affect your organization’s performance?
- People have goals.
- The responsibilities for leadership and decision-making are known and distributed appropriately.
- The leaders in the organization are concerned to get significant tasks done well.
- The leaders in the organization are respected.
- Staff members are willing to take on leadership.
- People are willing to express new ideas to those in positions of power.
- The organization is appropriately linked externally, and its place is seen as important by peer organizations.
- Leadership is effective in acquiring and protecting resources.
- Leadership practices participatory management, and the Board of Directors and management work well together.
- Leadership is flexible, and it welcomes change.
Strategic planning comprises the ways your organization thinks ahead and responds to its environment to achieve its goals. Strategic planning involves the development and implementation of activities that will lead to the long-term success of the organization. The strategic plan is a written document that sets out the specific values, mission, goals, priorities, and tactics that your organization will use to ensure good performance. It identifies gaps in the organization’s performance and suggests ways to close these gaps.
With good strategic planning, your organization can anticipate and capitalize on opportunities in the external environment that might yield needed resources. It also means being able to predict threats to your resources so that you can intervene (typically, politically) to safeguard the organization’s performance and survival.
To what extent does strategic planning affect your organization’s ability to achieve its goals?
- The organization has a strategic plan.
- The strategy is known to the Board of Directors, senior managers, researchers, and other staff.
- People in the organization generally accept and support the strategy.
- The strategy has helped clarify priorities, thus giving the organization a way to assess its performance.
- The strategy is used to help make decisions.
- The strategy improves performance.
- The strategy supports equity.
- The strategy helps the organization use resources optimally.
- The organization has a process for understanding its clients and users, for clarifying and revising its mission and beliefs, and for working to achieve its goals.
- The organization has a process for scanning the environment to consider potential threats and opportunities.
- The organizational strategy helps the organization to identify its opportunities and constraints in terms of financial resources and infrastructure.
- A process for monitoring the application of the strategy is in place.
- A similar process for understanding client and stakeholder requirements and changes is in place.
- The organization has a process for ongoing review and updating of its strategy to reflect internal and external realities.
The Board of Directors and charter of your organization provide its legal and policy framework and direction. A good Board of Directors has its finger on the pulse of both the internal and the external environments. The board assesses whether your organization’s initiatives are supportable and meet development goals, nationally or regionally; whether your organization is responding appropriately to important forces and trends in its field and within the wider environment; and whether your organization is meeting the needs of those it serves.
To what extent does your organization’s governance affect its performance?
- The governing structure both clarifies and supports the organization’s direction.
- A charter provides an adequate framework for carrying out the mission of the organization and for dealing with external challenges to the organization.
- The governing body scans the external and internal environments to understand forces affecting the organization.
- The governing body responds appropriately to important environmental trends and influences, whether social, political, or economic. For instance, both quality and equality issues are reflected in the minutes and discussion. The governing body supports the principle of equity.
- The governing body operates effectively and efficiently.
The structure of your organization is the system of working relationships that divides and coordinates the tasks of people and groups working with a common purpose. Most people imagine an organization’s structure in terms of the familiar organigram (organization chart). However, structure is far more: it involves the division of labour (including roles, responsibility, and authority), as well the coordination of labour into units and inter- and intra-unit groupings. When you assess the structure of your organization, you will want to see if it facilitates or hinders the achievement of the mission and goals.
Does your organizational structure facilitate or hinder your organization in achieving its mission or goals?
- The organization’s structures support its mission and its goals.
- The roles within the organization are clearly defined but flexibly enough to adapt to changing needs.
- Departmental lines or divisions between groups are crossed easily, particularly when collaboration means an improved product, program, or service.
- Structural authority is used to further equity.
- Staff members have linkages with, and access to, other units in the organization important to their work.
- Coordinating mechanisms are in place to facilitate access to other units within the organization.
- Staff members can easily create important coordinating units.
- The organization encourages efficient ways to coordinate staff and units.
- The organization offers clear lines of accountability (individual, group, and organizational accountability).
- People have the authority to set an agenda to support accountability.
- The organization has efficiently functioning work groups.
- Decision-making is sufficiently decentralized to promote productivity and good morale.
- The structure relating to responsibility for performance makes organizational sense and facilitates the work.
Niche management forces managers to look beyond internal matters to consider the wider environment and the broader issues of the time. It means carving out a particular area in the marketplace to match your organization’s particular expertise. Your organization’s niche helps to clarify its relations to similar local, regional, national, and international organizations. Your organization’s niche helps determine the levels of funding and types of funders that can help build its capacity.
Does your organization undertake niche management to help it to achieve its mission and goals?
- The organization’s constituents (stakeholders) understand its role or area of specialization.
- The role of the organization is clearly defined in its mission.
- The areas of specialization are clear.
- Stakeholders support the areas of specialization.
- The organization undertakes research and development that strengthens its unique role.
The human resources of your organization include all staff (professional, managerial, technical, and support) engaged in any of your organization’s activities. Your organization’s human resources are perhaps its most valuable asset. This is particularly true if the people required to do the core work of your organization are highly trained individuals.
The human-resource-management function of your organization is responsible for ensuring that people’s needs are met. This is not merely an altruistic function – staff members who are reasonably comfortable with working conditions and stimulated by the environment are highly likely to be productive. Human-resource management is responsible for planning; staffing; human-resource development; assessments and rewards; and maintaining effective human-resource relations. Each of these components of human-resource management will be reviewed individually in this section.
Human-resource planning is the first step in effective human-resource management. It involves forecasting the human-resource needs of your organization and planning the steps the organization must take to meet those needs. Human-resource planning should be closely linked to your organization’s strategic objectives and mission. Even in regions with a plentiful, well-educated workforce, human-resource planning is a challenge because the needs of the organization are constantly changing and sometimes do not converge. The challenge is even greater if the recruitment pool is limited or if the people in charge of human-resource management have not been trained to forecast the organization’s needs.
To what extent does your organization’s ability to plan for its human-resource needs affect its performance?
- The right people are in the right jobs in the organization.
- The organization can forecast current and future demands for human resources.
- The organization knows how and where to find the people with the skills needed to fill its needs.
- The organization can link its mission and goals to its human-resource planning.
- The organization has developed a personnel-policy manual.
CHANGING TIMES, CHANGING NEEDS
In changing times, it is not always easy to identify what kinds of people your organization will need. In one nonprofit organization, the mission and goals had been quite stable over the last decade. The organization paid little attention to the analysis of its human-resource demands. It continued to recruit people for the Board of Directors with the same profile as those who first made the organization a success: individuals with great dedication to the mission who were involved in the delivery of services and very interested in daily contact with direct beneficiaries.
Over the most recent 2 years, however, the organization had to privatize. It needed new staff members with a greater interest and skill in management, fundraising, and organizational issues. Although this organization was able to address the forecasting issue, it recognized that the search for a new staff profile was a challenge.
Staffing for your organization means searching for, selecting, and orienting individuals with the appropriate range of knowledge, skills, behaviour, and values to meet your organization’s needs.
To what extent does your organization have adequate staffing procedures to ensure its performance?
- The organization has a staffing system.
- The organization has appropriate job descriptions or equivalents to determine the positions it is staffing for.
- The organization has an appropriate system for selecting candidates (for example, reviewing CVs, conducting interviews, checking references).
- Individuals in charge of selection are appropriately trained to carry out this function (interviewing and listening skills, politeness, good judgment).
- Recruitment and selection materials (advertisements, postings, interview questions) are neutral, free of discriminating elements (gender, religious, etc.).
- Someone who is familiar with the day-to-day functioning of the organization is available to orient new staff members.
Developing human resources in your organization means improving employees’ performance by increasing or improving their skills, knowledge, and attitudes. This allows your organization to remove or prevent performance deficiencies, makes your employees more flexible and adaptable, and can also increase their commitment to your organization. The development of human resources can take several forms, such as training for a job, training for a role inside the organization, and training for a career.
To what extent does your organization have appropriate human-resource development systems and approaches to ensure its performance?
- The organization has a policy for training and development, as well as a budget for training.
- The organization encourages staff members to continue to learn and develop (by providing incentives for learning, by paying training costs, etc.).
- Someone in the organization can identify training needs.
- The organization supports application and transfer of new learning on the job.
- Training is demand driven (responds to needs in the organization), as opposed to supply driven (responding to whatever is being offered – on the market or by a donor).
- The organization can and does assess training and its effects on the organization’s performance.
- The organization has plans for mentoring younger staff members to help them advance in their careers.
- The organization has a way of dealing with succession.
- People see career opportunities in the organization.
The systems and approaches your organization uses to collect information and provide feedback on individuals or teams is an important aspect of your organization’s human-resource management. This means assessing the contributions of each staff member and then distributing rewards (direct, indirect, monetary, and nonmonetary) according to the regulations of your region and to your organization’s ability to pay. The assessment and reward system should aim at allowing your organization to retain good employees, motivate staff, administer pay, facilitate the organization’s strategic objectives, and control labour costs.
ASSESSMENT AND REWARD ISSUES
To what extent does your organization have an appropriate system for assessment and reward that is fair and motivating?
- The organization has a compensation policy that complies with the rules and regulations of the country.
- Staff members see an adequate correlation between compensation and performance.
- Staff members are generally satisfied with their compensation.
- Compensation packages are externally competitive for the sector.
- The organization maintains internal equity of salaries and benefits (that is, equal compensation for work of equal value).
- Compensation differentials are appropriate to motivate staff.
- The organization motivates staff with both monetary and nonmonetary rewards.
The maintenance of effective human-resource relations applies to all the programs and systems your organization has put in place to ensure that employees are protected and dealt with in accordance with the appropriate legislation. It involves all the activities your organization has implemented to address health and safety issues, human-rights issues, and the quality of working life. In unionized settings, it involves collective bargaining. In essence, it represents the organization’s culture, management style, and concrete measures taken to ensure that employees experience feelings of ownership, self-control, responsibility, and self-respect. Exactly what your organization does to produce these outcomes will vary according to the nature of the organization, its leadership style, and its cultural setting.
HUMAN-RESOURCE RELATIONS ISSUES
To what extent does your organization have effective human-resource relations?
- People in the organization feel protected from exploitation (through a collective agreement or through an appropriate set of personnel policies).
- Measures and procedures are established in the organization for dealing with people in emotional or physical distress.
- The organization seeks ways to increase the loyalty and the commitment of the staff. Morale in the organization is generally good.
- The organization has measures in place for dealing with harassment in the workplace.
- The organization has, if appropriate, a health and safety policy.
- Work-related accidents are rare.
Your organization’s ability to manage its financial resources is critical. Good management of budgeting, financial record-keeping, and financial reporting is essential to the overall functioning of your organization. Good financial management ensures that your Board of Directors and managers have the information they need to make decisions and allocate the organization’s resources. It also inspires confidence in your funders.
The people responsible for your organization’s financial management need to be able to plan and budget resources (operating and capital budgets), manage cash, do accounting, and provide financial reporting. The Board of Directors and senior managers should be involved in financial management and be clear about accountability. Your organization also requires skilled people, at both the Board and the staff levels, to carry out the financial analysis and work.
Financial statements are a barometer of an organization’s health. Sound internal financial procedures for administering your organization’s operating funds and program grants offer assurance that monies are being directed properly.
Overall, your budget should support the important goals of your organization. For example, if international exchange of information is one of your organization’s priorities, your budget should allocate the funds needed to obtain electronic-data systems, to host international visitors, and to conduct other related activities in support of this goal.
Financial management includes financial planning, financial accountability, and financial statements and systems, which will be discussed individually in this section.
Financial planning enables your organization to forecast its future operating revenues and expenses and its capital and equipment requirements. The ability to plan revenues provides a framework within which your organization can make decisions about programs and other expenses. This financial planning should take account of your organization’s short- and long-term financial requirements.
Does your organization undertake adequate financial planning to support the organization’s performance?
- The organization undertakes adequate budgetary planning.
- Budget plans are timely.
- Budget plans are updated as financial information comes in.
- Members of the governing body are involved in financial planning and monitoring.
- Human resources are adequate to ensure good financial planning.
- Grants or loans are properly managed.
- Decision-making relies on monitoring and analysis of the ratio of actual to planned budgets.
- Capital and equipment forecasts are made appropriately.
- Reports are provided to senior managers, the Board of Directors, and funders on a regular basis, at least every quarter.
- Timely financial information is given to those who need it.
Proper care of your organization’s finances is a prerequisite for external trust. This means that your organization has appropriate checks and balances and that both internal and external professionals oversee the care of your organization’s assets.
- The auditors of the organization are satisfied with the financial managers’ controls of cash and assets.
- The year-end date is clearly stated.
- The Board of Directors reviews financial statements on a regular basis.
- People on staff and the Board of Directors are competent to interpret financial information.
- Financial information is contextualized in a strategic or business plan.
- The Board of Directors establishes a committee to oversee financial issues.
The financial managers of your organization are responsible for preparing financial statements and ensuring their integrity and objectivity. At a minimum, this means having a bookkeeping system that creates a balance sheet and an income statement. Your financial system must be able to track income, assets, and liabilities and explain expenditures. This implies the ability to control assets and liabilities and manage the cash of the organization.
FINANCIAL STATEMENTS AND SYSTEMS ISSUES
Are financial statements and systems in your organization appropriate to support its performance?
- The organization has an adequate bookkeeping system.
- The organization has adequate staff to record financial information.
- Balance sheets and income and expense statements are prepared at least quarterly.
- A procedure is in place to control and record the assets of the organization.
- Cash-flow statements are prepared.
- Cash is managed to allow the organization to benefit from a surplus and minimize the costs of cash shortages.
Infrastructure comprises the basic conditions (facilities and technology) for work to go on in your organization, such as reasonable space in a building equipped with adequate lighting, clean water, and a dependable supply of electricity. Transportation to and from work is also part of this infrastructure. In developed countries with the wealth and the government structure to support adequate infrastructure, these conditions are often taken for granted. In some developing countries, ensuring adequate infrastructure is a problem organizations need to assess.
Each organization has its own infrastructural assets and liabilities. If your organization has its basic infrastructure in place, this topic will represent a small component of your assessment. If your infrastructure is debilitated, however, with water and electricity problems, then infrastructure will become a major concern.
To understand your organization’s capacity, you will need to consider the extent to which inadequate facilities interfere with the functions or potential functions of your organization. Although single deficiencies in one or more elements of infrastructure may not interfere with day-to-day work, at some point the work will be affected. Typically, the basis of many infrastructural problems is maintenance, which often suffers because of the lack of a recurrent budget for upkeep.
Is your organization’s facilities infrastructure adequate to support the organization’s performance?
- The organizational strategy identifies opportunities and constraints stemming from the facilities infrastructure.
- The buildings and internal services (for example, water, electricity) are adequate to support and facilitate daily work.
- Employees have an adequate transportation system to and from work.
- Communications systems (hardware) function at the levels required.
- An ongoing maintenance budget supports adequate maintenance systems and procedures.
- The organization effectively and efficiently manages the infrastructure, including building and equipment maintenance.
- An individual or a group within the organization is responsible for adequate planning to address ongoing infrastructure concerns.
INFRASTRUCTURE: THE SYMBOL OF AN ORGANIZATION
In one organization, whose mandate was to serve the needs of rural aboriginal communities internationally, the major strategic question was the organization’s identity. As an organization concerned with aboriginal issues, should the majority of its Board of Directors be aboriginal? Should it seek sources of funding specifically for aboriginal projects? These questions were eventually transformed into strategies.
The most significant strategy was the Board of Directors’ decision to relocate the headquarters of the organization to an aboriginal reservation, despite the fact that the organization already had an adequate location (moderate cost, adequate rooms, etc.). For symbolic reasons and to reinforce its mission statement, it moved. So, although infrastructure was not in itself an issue, a gap existed between the infrastructure and the image that the organization wanted to project.
The technological resources of your organization comprise the equipment, machinery, and systems (including library systems, information systems, hardware, and software) that enable it to function. It is important to keep in mind that the instruments of technology are merely tools for enhancing services and products: ideas must inspire the technology.
To what extent do technological resources affect your organization’s performance?
- Technological planning in the organization is adequate.
- Overall, the organization’s level of technology is appropriate to carry out the organization’s functions.
- No one unit is seriously lagging behind the others in the level of technology needed to carry out its work.
- Access to international information is provided to all units through library and infor-mation-management systems.
- Adequate systems and training are in place to manage the organization’s technology.
- Adequate information technologies are in place to manage the organization.
LINKING FACILITIES AND TECHNOLOGY
In one educational organization, one of the strategic questions concerned the organization’s inadequate space. Students were dissatisfied with the size of classrooms or facilities, and the management group responsible for the 5-year plan identified the need for additional space and new classrooms.
However, after consulting a technology specialist, the organization modified its approach. The specialist advised that 5–10 years down the road, approaches to learning could be quite different. With the rise of multimedia, the classroom might not be the primary location for learning in 5 year’s time. Therefore, their strategies had to be creative to ensure adequate space in the short term but without putting substantial funds into building classrooms that might not be required in 5 years.
In this case, to develop a good strategy, the organization had to analyze both the external environment (how learning technology was evolving) and the organization’s capacity.
Your organization’s ongoing programs, services, and products constitute its central endeavour. Indeed, they are the organization’s main attempts to operationalize its mission. The function of program management is to develop and administer these programs in ways that support this mission.
Program management is vital to all other areas of organizational capacity. Ultimately, the strength of your organization’s strategic leadership, human resources, financial resources, infrastructure, process management, and intra-institutional linkages affects the quality of your programs. Program performance is highly visible outside the organization and is often the major focus of organizational assessments.
Good program management requires a cycle of careful planning, implementation, and monitoring and evaluation. All programs go through this cycle, either formally or informally. Each of these aspects of good program management is discussed in this section.
Program planning requires thinking ahead. It involves the following questions: What are our objectives? What must be done to meet these objectives? Who will do this? How will they do it? How long will it take? How much will it cost? How will we know whether we have met our objectives? And so forth.
Program planning has many levels. However, when conducting a self-assessment, you will need to determine the extent to which your organization’s plans are well communicated and used as management tools. This will require written plans.
To what extent does your organization appropriately plan its programs?
- Each program area and each major project has a written plan.
- Program and project plans are linked to the organization’s mission.
- The organization undertakes adequate program-planning and budget-programing activities to ensure that its programs support its mission.
- The organization’s programs and projects are consistent with its mission, needs, strategies, and priorities.
- Program planning takes into account technological, economic, gender, social, and environmental aspects to ensure the applicability of programs.
- Programs are given adequate timelines.
- Programs have adequate budgets.
- Program planning includes an adequate analysis of roles and responsibilities.
- A procedure is outlined to monitor results.
The real work of managers is in making the organization’s program plans work. It is all well and good to have a great plan — making it work is the hard part. Program implementation requires the organization to have people on staff who can put their skills to work. It requires integration of the management skills needed to allocate resources and the technical skills needed to do what needs to be done (for example, to provide health services and do research). Program implementation is the stage at which your organization integrates all its resources to concretely achieve its goals.
To what extent does your organization appropriately implement its programs?
- Staff support the organization’s efforts to get programs accomplished and to get products and services to clients and beneficiaries.
- The staff members providing products and services have good relations with each other.
- Staff members work together to provide good products and services.
- The program team has good problem-solving skills.
- Health and safety for staff and clients are always a priority in implementation.
- Resources are efficiently used to provide the product or service.
- Schedules are adhered to in a reasonable fashion.
- Staff members are motivated to work together to get things done.
- Program meetings are productive.
Programs or projects are central to the life of your organization. Management needs to keep track of programs to ensure that they are meeting their objectives and achieving their intended results. Monitoring and evaluation systems need to be built into programs and projects during their planning stage (see also “Program planning,” p. 75).
MONITORING AND EVALUATION ISSUES
To what extent does your organization appropriately monitor and evaluate its programs?
- Monitoring and evaluation systems are in place.
- Program staff receive feedback on program performance.
- The monitoring and evaluation process includes adequate opportunities to clarify roles and responsibilities.
- The process includes adequate opportunities to review program indicators to measure progress against plans.
- Timelines are monitored to reduce overruns.
- Budgets are reviewed in a timely fashion.
- Programs are reviewed on a regular basis to determine how well they contribute to the organization’s overall strategy.
- Drawing lessons is encouraged.
- Corrective actions are taken when difficulties arise.
- Staff members see monitoring and evaluation as ongoing and normal processes.
Organizational processes are part of your organization’s dynamic. These processes are major components of both stability and change. In the context of a self-assessment, it is important to examine four processes: planning; problem-solving and decision-making; communications; and monitoring and evaluation. Each of these processes is discussed in this section. It is important to consider the nature of these processes and their impacts on your organization’s performance.
Planning helps your organization predict how its members will behave. Your strategic plan sets the overall direction; operational planning is the process by which that strategy is translated into specific objectives and methodologies to accomplish your goals. This involves making the best use of resources of time and people (for example, developing timelines and schedules).
To what extent does the your organization’s planning process contribute to its performance?
- Planning, policy, and procedure development occur in the organization at all levels, from the governing board to departments and individual projects.
- The process of planning contributes to the strategic direction of the organization.
- Plans are clear, and they provide adequate direction for the organization’s members.
- Plans, policies, and procedures are generally followed.
- Planning is part of the organization’s culture.
- Staff members feel that they are involved in planning.
- Planning is linked to monitoring and evaluation.
Problem-solving and decision-making interact and reinforce each other. Both processes must function well at every level of your organization. These processes include the ability to define important problems, gather the data to understand the issue, create a set of alternatives for dealing with a problem, decide on solutions, create the conditions to carry out decisions, and monitor these decisions and the problem’s resolution. Timeliness is a key element in these processes.
PROBLEM-SOLVING AND DECISION-MAKING ISSUES
Do the problem-solving and decision-making processes support your organization in carrying out its functions?
- The implementation of work flows smoothly at every level of the organization.
- Decisions are timely.
- Performance gaps and opportunities are identified quickly enough to resolve them to the benefit of the individuals involved and the productivity of the organization.
- Problem-solving and decision-making mechanisms are in place.
- People on the governing board and within the ranks of senior managers have adequate organizational problem-solving and decision-making skills.
- Problem-solving and decision-making are adequate in departments and for important projects.
- The staff members feel empowered by the problem-solving and decision-making processes.
- The staff members try to solve problems before they become big concerns.
The function of internal communications in your organization is to exchange information and achieve a shared understanding among staff members. Internal communications can serve as the glue holding an organization together or it can break it apart, because both information and misinformation flow constantly in organizations.
Accurate information helps keep your organization’s employees informed and motivated: aside from the specific information they need to carry out their work, the organization’s members also need information to make them feel that they are part of an important effort and work for a wider purpose. Your organization needs mechanisms to help staff members obtain both types of information. Coordinating committees, newsletters, and meetings of various sorts all provide vehicles for transmitting correct messages.
Organizational monitoring and evaluation complement program evaluation and monitoring. Organizational monitoring can help clarify your program objectives, link activities and inputs to those objectives, set performance targets, collect routine data, and feed results directly to those responsible. Organizational monitoring is an ongoing, systematic process of self-assessment.
Is your organization’s performance effectively supported by its communication system?
- People in the organization feel there is adequate, ongoing communication about the organization’s activities.
- Staff members receive information about the organization’s mission and its progress in fulfilling the mission.
- Information circulated in the organization about activities is rarely distorted.
- Mechanisms are in place to correct rumours.
- People have easy access to others they must deal with in the organization and can easily communicate with them.
- Written communication is adequate.
- Meetings are viewed as productive ways of communicating.
- Adequate use is made of communication technology.
- Two-way communication is encouraged.
- Multichannel communication is often used.
- Listening is valued.
- Cultural diversity is a consideration in communicating with others.
Organizational evaluation looks at why and how overall results were or were not achieved. It links specific activities to overall results, includes broader outcomes that are not readily quantifiable, explores unintended results, and provides overall lessons that can help your organization to adjust programs and policies to improve results.
MONITORING AND EVALUATION ISSUES
Is your organization’s monitoring and evaluation adequate to improve performance?
- Policies and procedures are in place to guide evaluation and monitoring.
- Resources are assigned to monitoring and evaluation.
- Monitoring and evaluation are valued at all levels in the organization, as ways to improve performance.
- The organization obtains and uses data to monitor and evaluate its units and activities.
- Use is made of data gathered through the organization’s overall monitoring and evaluation activities.
- The organization has an evaluation plan or performance-monitoring framework.
- Strategy, program, policy, and budgetary documents mention evaluation results.
- People have the skills to perform monitoring and evaluation.
- Monitoring and evaluation processes are valued.
- The organization learns lessons from monitoring and evaluation and makes changes as a result.
Having regular contact with other institutions, organizations, and groups with strategic importance to your organization can result in a healthy exchange of approaches and resources (including knowledge and expertise). Your organization may be forming linkages or may already have linkages with potential collaborators and collegial bodies, potential funders, or key constituents.
Linkages help your organization keep up with advances in pertinent fields and give your organization access to wide-ranging sources of up-to-date information within each area of your organization’s work.
Today, many types of linkages can and need to be made to support your organization’s performance. For example, new information technologies can help your organization to learn about the most recent approaches to programing and managerial issues. They also bring new ways to communicate with potential allies and collaborators in key program and funding areas. Two aspects of interinstitutional linkages are discussed in this section: relationships such as networks, joint ventures, partnerships, and coalitions and electronic linkages.
At the same time as electronic linkages are opening organizations to new ideas and ways of communicating, a similar revolution is occurring with respect to new organizational patterns that support joint work and collaboration.
Many organizations are finding that they are unable to make progress in achieving their missions without the support of similar organizations. Many are forming new types of relationship (either formal or informal) with other organizations to support their objectives.
Networks are informal linkages, groups loosely coupled to serve common interests. The new and joint ventures, partnerships, and coalitions are more formal. The most formal of these relationships are based on contractual agreements. All of these new linkages are breaking down the boundaries of organizations and are changing the way they operate.
NETWORK, JOINT-VENTURE, PARTNERSHIP, AND COALITION ISSUES
Has your organization established or pursued external linkages adequate to support the organization’s performance?
- The organization has adequate formal and informal linkages with like-minded organizations.
- Institutional linkages are adequately supported.
- Institutional linkages contribute efficiently to the organization’s goals and mission.
- Fruitful, ongoing partnerships with external organizations through these linkages bring new ideas or resources, or both, to the organization.
- The organization is using these linkages to communicate information about its work to external stakeholders, including the general public.
Organizational capacity and performance can be increased if an organization makes appropriate use of new electronic technologies. These new technologies can improve your organization’s communications and keep people informed about the latest ideas. The organization’s members can join Internet discussion groups, listservs, and other electronic mechanisms that link people with like minds and ideas. Electronic systems give your organization an opportunity to search the entire globe for new ideas and information.
Has your organization established or pursued electronic linkages adequate to support its performance?
- The organization is electronically linked to the outside world of colleagues, clients, and markets (users) to make these relationships active and beneficial.
- Electronic networks are financially and technically supported.
- Electronic networks effectively respond to the needs, shared interests, and capabilities of the organization.
- Electronic networks support new efficient practices.
- Fruitful, ongoing partnerships with external organizations through electronic linkages bring new ideas or resources, or both, to the organization.
- The organization is using electronic linkages to communicate information about its work to external stakeholders, including the general public.
EXERCISE 12. Examining organizational capacity
Instructions: This exercise can help you begin identifying capacity gaps affecting overall performance in the organization. You might want to conduct this exercise with a group of senior managers, organization members, and the self-assessment team.
Using the chart below as a guide, reflect on the capacities of your organization. Identify capacity gaps and how they affect your organization’s performance. Discuss the seriousness of these issues with the self-assessment team and see if there is some consensus on the priority issues.
EXERCISE 13. Summarizing the performance issues
Instructions: Before you collect and analyze your data, you might find it useful to summarize your performance issues. Using the matrix below as a guide, summarize the major performance issues you have already identified and then develop the key questions and subquestions for each issue.
The five-easy-pieces model is an approach to self-assessment that your organization might like to try. Using this model is one way we have been able to begin a relatively quick and easy discussion:
The purpose of this tool is to help you identify stakeholders and their interest in the organization. It can be used to analyze organizational relevance and assess the environment.
Each stakeholder may have several interests.
Each stakeholder may have several roles in the assessment. These can be listed using the following table format:
PARTICIPATION OR ROLE
The purpose of this worksheet is to provide an analytical tool to help organizations to understand the relationship between performance and the factors that support performance. It can be used at any stage in the diagnostic process at which the team attempts to identify the causes of good or poor performance. It is an evolving tool that can be updated as performance factors emerge.
Use the following worksheet to help you identify the possible underlying causes of performance issues:
FACTORS IN THE ENVIRONMENT THAT AFFECT THIS ISSUE
ASPECTS OF ORGANIZATIONAL MOTIVATION THAT AFFECT THIS ISSUE
ASPECTS OF ORGANIZATIONAL CAPACITY THAT AFFECT THIS ISSUE
The purpose of the culture audit is to provide the assessment team with a list of areas within which data can be gathered respecting organizational motivation. The assessment team can use it to help in planning data collection. Some of the questions could be used in animating focus groups with staff members or people on the Board of Directors – to take the pulse of the organization.
Part 1– Individually answer as many of the questions below as you can.
Part 2 – In a group, discuss your answers and whether one can answer all the questions. Are most people agreed on their answers?
TASK OR ACTIVITY
RESPONSIBLE TEAM MEMBER
Develop data-collection instruments
Rosalie Rosalie and Georges
TASK OR ACTIVITY
RESPONSIBLE TEAM MEMBER
The purpose of this form is to provide a planning tool for the self-assessment team. With this tool, each member of the team is assigned responsibilities for specific tasks and an estimate is made of the time it might take to complete the task assigned. This tool should be used early in the assessment phase to allow the organization to begin to assess the resources the self-assessment will require.
TEAM MEMBER 1
TEAM MEMBER 2
TEAM MEMBER 3
Development of frameworks
Analysis of interview or focus-group data
Analysis of questionnaire data
Analysis of project or organizational data
Formulating findings and conclusions
Validating analysis with key stakeholders
Revising final report
The purpose of this collection of tips is to improve the ability of your assessment team to create data-collection instruments and to collect data. It provides advice on the development of instruments, data-collection methods and data analysis for group techniques, questionnaires, and interviews – three common data-collection approaches. If considered at the planning stage, these tips will help your team determine which type of data collection is appropriate for addressing each key issue.
The following list provides an overview of the procedure:
Plan for 5–10 questions. Effective group questions are carefully defined. They
Successful groups for data collection comprise people who
SEE TIP 4 – Basis for Judgement (p. 112)
Whether interviews are conducted face to face or over the telephone, following certain procedures can help you to get the most out of them. These procedures are as follows:
Your first step in using interviews is to decide what approach to use:
The type of interview will determine your plan:
It is important to do a trial run of any interview. This pilot test will help you to validate
If you are using other interviewers, you will need several training sessions to ensure that they learn to use the protocol.
Revise your interview protocol or guide after you pilot test it with clients or train other interviewers.
It is important that you, as an interviewer, maintain control of the process as well as of the content of the interview.
With key-informant interviews, you are trying to understand the answers to such questions as
Some views may need to be weighted for importance; for example, some clients may be more important than others, either because they are bigger clients or because their needs are more vital. The bottom-line question is what do our elite clients feel about our goods and services. What does this tell us?
SEE TIP 4 – Basis for Judgement (p. 112)
Interviewers often get into trouble because they violate basic rules. The following problems should be avoided:
WHAT TO DO OR AVOID
Have you ever experienced burnout, and what do you do to prevent it?
Avoid double-barreled questions. Ask one question at a time. Do not combine questions and expect an answer.
What are the advantages and disadvantages of working at this university?
Do not combine opposite positions in one question. Separate out the parts, and things will be much clearer.
Do you think that female administrators are as good as male administrators?
The phraseology of this question eliminates the possibility that females might be better. Avoid questions that inherently eliminate some options.
ABC Inc. wants departments to be close to their clients. What do you think of my department's client relations?
Do not precede questions with a position statement. In this type of question, the interviewer states a view or summarizes the position of a current or recent event and then asks for a response. This tends to lead the respondent in a given direction.
Would you favour or oppose murder by agreeing with a woman's free choice concerning abortion?
Avoid questions that use loaded words and are emotionally charged.
The following procedures are recommended for a questionnaire survey:
You should begin by understanding the major questions or issues you wish to address. These will generally be reflected in the questionnaire sections, as described below.
This section includes questions about your client that may be important to your analysis. It should solicit background information you need to address, such as the respondent’s department, region, experience, gender, position, and experience with the supplier.
Ask only what is essential to your subanalyses. If you don’t need to know, don’t ask.
This section is the heart of the questionnaire and requires you to develop dimensions of quality that may be important. The client (respondent) then rates the quality of your outputs along these dimensions.
For this section, choose a title that matches other important dimensions of client service, such as “Timeliness of Delivery,” “Safety,” or “Environmental Responsibility.”
This set of questions will address your client’s perceptions of your service. This section might be merged with the quality or other-considerations section.
These major sections of your questionnaire provide the overall outline. Once you know these major themes, you need to develop actual questions or items.
You must draft actual questionnaire items within each of the sections of your questionnaire. It is difficult to vary the types of questions too often, so economize within each section by asking similar types of questions.
You will need to master six types of questionnaire item before you invent your own. Unproved alternatives are often confusing to the reader. So use unproved alternatives only after you are fully familiar with the types of items described below.
This type of question is useful for the introduction or background-information section.
Use this form when the possibilities are too numerous to list using a multiple-choice item. They work well in a mix with multiple-choice. So, they are also good in the introduction.
This type of item enables you to collect a lot of information efficiently. Rating-scale items are good for rating your goods and services, other considerations, and so forth.
This type of item provides a stronger form of feedback than a rating scale. It forces the client to identify what he or she considers important and helps the researcher to avoid the problem of people just agreeing because it is easy to check a box without feeling that it is important to them.
What aspects of your training course did you like most? Please list three of them.
This type of question is another way to gain an understanding of what your client considers important. It is particularly useful for “mopping up” in the concluding section.
Please write any other comments about the work of school principal and suggestions for training that you consider important:
The Likert scale allows the respondent to agree or disagree with a series of statements. (Note, these are statements, not questions.) The Likert scale is easy to use, if you know how, and like other rating scales it is an efficient way to collect lots of information.
Now, you try a few. Write your own statements for dimensions of your work unit’s outputs. Include items that are worded both positively and negatively.
As you write the items, you should begin considering an overall design for your questionnaire. Follow these rules:
Even the best questionnaire needs testing. You might understand everything in the questionnaire, but your client may not. Here are some tips to help you test your questionnaire.
It sounds like a lot of work. It is! Creating a good questionnaire may take a week of full-time work, even for a professional.
Now you have a questionnaire ready to go! You’ll need to work out a strategy for how and where to send it. The first part of your strategy is to select a sample of people who fairly represent all your clients. Prepare a list of your sample clients.
The second part of your strategy is to decide on the technology you will use to send out your questionnaire.
Questionnaires can be printed, in your office or by a printer, and mailed to respondents. Respondents fill them out and mail them back. Results are manually input into a database or statistical program for analysis.
It is possible to print questionnaires so that they can be read by an optical scanner that picks up the responses automatically. The questionnaires in Appendix 3 were designed to be used in this way. (Note, pictographs can be used to illustrate points.)
The coming wave for internal client-needs assessments is the e-mail questionnaire. This is designed on a computer and sent as a computer file to clients via e-mail. The client receives the file, completes the questionnaire on his or her computer, and sends the file back to you by e-mail.
You also need a follow-up strategy. This may include
When key people in each unit distribute and collect the questionnaire, pyramid networks are great, but personal networks are the best of all for getting returns.
Each client in your sample should receive
Every successful questionnaire comes with a cover letter. The letter should contain six pieces of information:
Questionnaire analysis generally means dealing with large numbers or with a variety of numbers. This usually requires you to use statistical concepts and computers. Many simple statistics programs are available to help you analyze data.
Because the same data can be interpreted in different ways, it is important to take these potential differences of interpretation into account at the design stage. Three main decision-making methods are generally used to make judgments about data:
Your self-assessment team can use one or more of these methods to interpret the assessment data.
This questionnaire is a tool to obtain data from staff on their level of satisfaction with the organization and their perception of its level of performance.
This questionnaire is a tool to obtain data from donors on their level of satisfaction with the organization and their perception of its level of performance.
This sample cover letter can be attached to the questionnaire going out to donors.
As part of our efforts to build our capacity and improve our services, XYZ is presently undergoing an organizational self-assessment and strategy-development exercise. Supported by [donor name], this process will also facilitate our strategic-planning efforts. We are asking our donors to take about 10–15 minutes to complete the enclosed questionnaire. Please return it in the self-addressed, postage-paid envelope at your earliest convenience. We would appreciate your response by [date], as the questionnaire is an important part of our data-collection process. All responses will be treated confidentially.
This page intentionally left blank.
This set of short exercises can be used at a miniretreat to encourage staff members to identify the organization’s level of performance. Each of the exercises can be done separately, although when they are done as a set they provide a holistic assessment of the organization – from where it is now to where it wishes to be. The exercises also help to identify what needs to be done to bridge any gap between the organization’s reality and its vision of the future. The self-assessment team can give these exercises, or this can be done with the help of someone outside the organization.
Note the following:
To develop a common understanding of the five major strengths and weaknesses of XYZ.
To understand the context within which XYZ is operating.
Working in two groups, identify the major external threats and opportunities that have the biggest potential impacts on XYZ. Use the following list of elements that can positively or negatively affect an organization such as XYZ as a guide. You may identify other elements.
EXTERNAL FACTORS THAT MAY AFFECT XYZ
NO MAJOR EFFECT
Support from member countries
Support from host country
Support from international funders
Reliance of our organization on a few donors
Collaborating institutions’ support for XYZ’s work
XYZ’s willingness to use electronic technology in its work
Gap between XYZ’s proclamation of support and its actions
National attitudes toward regional cooperation
National values for sharing with other countries in the region
Policies of international bodies
National leaders’ desire to know about the impacts of XYZ
Competition from other organizations doing similar work
Demand for the type of services XYZ offers
International demand for expertise in integrated rural development
Reputation of the organization
Disenchantment of donors with funding international bodies
Disparities among members in the region, such as language or culture
Link institutions’ satisfaction with services
External groups’ lack of understanding of the target beneficiaries of XYZ
To create a vision of XYZ as it will be in 3–5 years.
Working in a group of five, develop a 5–10 minute presentation. Your presentation should include
To begin to make the vision for XYZ a reality.
With the whole group, take 30 minutes to identify and discuss the following topics:
Charles Lusthaus, PhD, is an Associate Professor in the Department of Administration and Policy Studies, McGill University, and a partner in Universalia Management Group. His expertise is in organizational theory and in institutional evaluation and change. Dr Lusthaus has more than 20 years of experience in organizational development and assessment and has published more than 30 articles on topics related to educational management and policy development. He is one of the authors of Institutional Assessment: A Framework for Strengthening Organizational Capacity for IDRC’s Research Partners and has also made more than 50 presentations at conferences and workshops. Dr Lusthaus is the Faculty Advisor for the Centre for Educational Leadership, McGill University.
Marie-Hélène Adrien, PhD, is a partner in Universalia Management Group. She has extensive experience in institutional and organizational evaluation and change, both in Canada and internationally. She works with medium-sized organizations from the public and private sectors to build individual and organizational capacities in response to organizational change: changing external context, diversified work force, emerging new training requirements, organizational restructuring, and changing stakeholder needs. She is one of the authors of Évaluation institutionelle: cadre pour le renforcement des organizations partenaires du CRDI. Her PhD and an MEd are from McGill University.
Gary Anderson, EdD, is an international authority on evaluation, education, and policy research and specializes in research studies and methodology. With more than 20 years experience in the field, he has designed, monitored, and evaluated education and training programs and has developed methodologies and frameworks for organizational development and analysis. He is the author of Fundamentals of Educational Research (2nd edition). Dr Anderson was the chair of the Department of Administration and Policy Studies, McGill University from 1985 to 1995. His BSc and MA are from McGill University, and he has a PhD in education from Harvard. He is a founding member and the president of Universalia Management Group.
Fred Carden, PhD, is Senior Specialist in Evaluation at the International Development Research Centre (Canada). He has extensive experience in international development and has written in the areas of evaluation, international cooperation, and environmental management. He has taught and carried out research at York University, the Cooperative College of Tanzania, the Bandung Institute of Technology (Indonesia), and the University of Indonesia. He has a PhD from the Université de Montréal and a Masters in environmental studies from York University.