Category: Evaluation

Crisis and Disaster Planning
This is part of a special series, brought to you by the Standards for Excellence Institute, to provide nonprofit leaders with a brief nonprofit governance and management tip weekly over the course of 2020. We hope these short tips will be helpful to you and the nonprofits you serve. If you have suggestions for future topics, please forward these to acmadsen@standardsforexcellence.org.

Not a week goes by when we don’t hear about some nonprofit that is tackling the impact of some type of disaster. Whether it’s a fire, a health emergency like an epidemic or pandemic, a natural disaster, or a violent act, nonprofits big and small are tackling disasters on an on-going basis. International disaster and relief organizations, hospitals, health and human services, food and housing organizations, and animal rescues are often a part of the first responder teams in many emergencies – and many of these organizations continue to provide support long after the immediate event.

At my organization, I am sincerely grateful for my colleagues who head up our efforts for disaster preparedness. The time and effort that is invested in ensuring that our emergency plans are up to date, that emergency supplies and water are replenished, that drills are carried out, that our team is trained, and that emergency communications strategies are available and ready to be employed is essential for ensuring not just the continuity of our services, but protecting the safety of our team.

The Standards for Excellence: An Ethics and Accountability Code for the Nonprofit Sector encourages all nonprofits to have written board-approved administrative policies addressing crisis and disaster planning that are periodically reviewed by the board.

Nonprofit services are vital to many communities. So maintaining capacity to serve others is crucial. Having a crisis and disaster plan can help ensure that nonprofits:
- Prepare for emergencies, to the extent possible
- Respond quickly and clearly to all constituents
- Continue to offer its essential programs
- Resume full operations as soon as possible
- Safeguard and protect vital organizational resources
The Standards for Excellence educational packet on Administrative Policies includes helpful resources and samples of a crisis and disaster plan; roles and responsibilities of the nonprofits’ disaster/crisis response team; a nonprofit risk assessment worksheet; a business continuity plan worksheet; and a sample emergency crisis communication plan.

This educational resource packet and the full series of all packets – including sample policies, tools and model procedures to help nonprofits achieve best practices in their governance and management – can be accessed by contacting a licensed Standards for Excellence replication partner, one of the over 150 Standards for Excellence Licensed Consultants, or by becoming a member of the Standards for Excellence Institute.
February 21, 2020
Ask a Simple Question for Nonprofit Impact
Winning Evaluation Strategies for Smaller Nonprofits

The following post was written by Standards for Excellence Licensed Consultant Elizabeth Galaida and is part of our “Ten Years of Advancing Excellence” blog series, celebrating ten years of the Standards for Excellence Licensed Consultant program. Elizabeth Galaida is a career nonprofit specialist that offers strategic planning assistance and database renovation. Bringing the necessary depth and breadth of skills sets to bear, she specializes in helping small to mid-sized nonprofits grow and thrive. Elizabeth became a Standards for Excellence Licensed Consultant in 2014.

When my son was little, we played a board game called Clue, Jr. The Jr. version is similar to the classic Clue game, but relies on a simple, straightforward process of elimination to solve a simple mystery. The adult version is best played by controlling for both known and unknown variables, using complex, two-step logic, and putting on a good poker face—there, I’ve just given away my game-winning secrets.

When it comes to program evaluation and demonstrating nonprofit impact, small organizations are often asked by funders to play the full version of Clue on a Clue, Jr. budget. Grant makers seeking to determine whether their $5,000 grant was “impactful” ask questions of their grantees that require $100,000 worth of randomized control trials to answer completely. That leads many smaller nonprofits to assume that they simply can’t show impact. The reality is that smaller nonprofits can, and should, be evaluating their programs, but they need to be asking different questions.

Those highest up on the spectrum of scholarly nonprofit research generally seek to establish scientific causality between a given program and its desired outcomes, commonly discussed as “evidence-based practices.” While few funders demand this level of sociological research of their grantees—and even fewer are ready to pay for it—there does seem to be an expectation among many that even small nonprofits should somehow “prove” the direct impact of their programs.

In the end, what usually happens is that the nonprofit cobbles together a report based largely on output data, the grant maker files it, and the discussion is over. However, the Standards for Excellence Institute has long held that all nonprofits can and should strive for excellence and impact, regardless of size. Evaluation for smaller nonprofits can be both effective and cost-effective, so long as the evaluation is mission-focused, includes quantitative and qualitative data, includes participant input and determines whether the programs answer a community need.

An organization’s program evaluation should start first and foremost with the mission. The Standards for Excellence state that nonprofits

“…should have a well-defined mission, and its programs should effectively and efficiently work toward achieving that mission.”

It is hard to argue that smaller nonprofits need major funding in order to accomplish this. Your board should review the mission statement annually, with newer board members bringing fresh perspectives on whether or not it is “well-defined.”

Your mission statement should be specific enough to know whether or not you are achieving it. The board should review all programs annually to determine if they are mission-focused or not. “Mission creep” is a common threat to the effectiveness and sustainability of an organization, diverting precious resources away from where they should be going.

In order to meet the Standards for Excellence, any program evaluation, whether conducted by committed volunteers or a national think tank, should include both quantitative and qualitative data (at the Basics level, data collection should have begun) and include input from program participants.

Quantitative data—“the numbers”—help us see the bare bones realities of our programs. Our brains lie to us all the time. It’s called confirmation bias. We tend to see only the facts that confirm our already-established beliefs. However, the numbers can tell us a different story, one that we may not want to hear. For example, if the graduation rate of your financial literacy program is only half that of similar programs, that number will prompt you to find out why, so you can serve your students better. Qualitative data provides context and meaning for your quantitative numbers. If the financial literacy organization in the above example discovered through participant surveys that the timing of the program made it very difficult for single parents to attend classes, a simple change of class time could lead to higher graduation rates. Which, of course, is also better mission fulfillment and better service to people in need.

Organization staff should build both qualitative and quantitative data collection into their regular, daily activities. That may mean tracking only how many people come through the door, or it may also mean this:
- Demographics
- Achievement, graduation or completion rates
- Changes in knowledge or confidence
- Satisfaction ratings
A cost-effective means of collecting qualitative data and getting participant input is to conduct exit interviews or surveys with your constituents. Make a point to ask constituents frequently for feedback, or schedule feedback collection to occur at program milestones.

Regarding the means of data collection, small nonprofits should remember that the more informal the data collection, the more human time it takes to compile the results. If you have a pile of ticket stubs of differing colors from one concert to count and sort, this is not terribly taxing—until you have to do this five times per week. This is where a scanning device to determine what kinds of tickets were used (such as full price, senior, educational, special groups, etc.) might be worth the expense. For organizations with complex services, or for those with state and federal reporting requirements for specific individuals over time, a database is a must.

Organizations should regularly evaluate the need for their programs. In my staff reviews of accreditation applications, this is the one area of evaluation that organizations frequently overlook. Waiting lists, sold out concerts, and requests from the public and partner service providers are all valid examples of the evidence of need for your programs. Nonprofits should be careful to define “need” in terms of their constituents, and not in terms of what funders want to pay for.

The conversation on nonprofit impact is constantly changing, and some researchers are starting to recognize that nonprofits may operate on a spectrum of best practices that range from “performance measurement” to “evaluation.” Whatever your nonprofit calls it, you can be successful in determining if your programs are meeting your mission, your expectations of quality and the community’s needs.

Read Further:

As Nonprofit ‘Research’ Proliferates, It Must Be Viewed With Healthy Skepticism

Three Kinds of Data that Actually Matter to Nonprofits

Tools and Resources for Assessing Social Impact

Performance Measurement and Management, from Urban Institute’s Measure 4 Change
July 26, 2016
Planning Strategically Through Process Thinking
The following post was written by Standards for Excellence Licensed Consultant Arshad Merchant and is part of our “Ten Years of Advancing Excellence” blog series, celebrating ten years of the Standards for Excellence Licensed Consultant program. Arshad Merchant recently founded Boost Social Sector Consulting to help nonprofit and other socially-minded ventures address critical challenges and pursue greater good. Drawing on 20 years in consulting, Arshad brings substantial experience in strategy, program improvement and organizational development. Arshad Merchant became a Standards for Excellence Licensed Consultant in 2015. Applications for the 2016 Licensed Consultant Training program will be accepted through July 1.
T

his post, originally titled “A Simple Method to Improve College Graduation Rates,” first appeared in the Stanford Social Innovation Review in September 2014.

David Borgal and Greg Johnson faced a challenge. Their Boston-area organization, Bottom Line, helped low-income, first-generation students get into college and graduate. Dave as founder and director of operations, and Greg as executive director saw hundreds of students entering college each year with help from Bottom Line. But not enough of them were graduating. And while the six-year graduation rate of Bottom Line’s participants at 73 percent was well above the national norm of 57 percent (based on data from the National Center for Educational Statistics), Dave and Greg firmly believed the rate could be higher. With this thought in mind, Dave and Greg asked Wellspring Consulting to help Bottom Line increase the graduation rates of its students.

As we began our investigation, we learned about Bottom Line’s Counselors, who were spending time with 475 college students across 15 campuses, meeting one-on-one or with small groups to provide encouragement and guidance. Bottom Line considered this approach to be a core tool for achieving successful graduation rates. However, the approach used by the Counselors was haphazard, drawing primarily upon their own experience and intuition to determine how to support students. Bottom Line had no systematic understanding of how Counselors should spend their time to yield the highest graduation rates.
T

o attack the problem, we employed an approach called Process Thinking, a powerful way to determine how daily activities can be performed to maximize the ultimate goals of a nonprofit organization. Based on the methods of Process Thinking, we began by understanding the needs of the beneficiary of Bottom Line’s services—in this case, the needs of the students. By systematically interviewing the Bottom Line staff who worked directly with students, we uncovered the four most important areas where students needed help to graduate. These were:
1. Staying on track to graduate—which meant selecting a suitable major, understanding what requirements they must fulfill to graduate, and using strategies and support to improve their academic performance
2. Building their employability—through securing part-time jobs, writing a resume, and defining a desired career path
3. Maintaining sufficient financial aid—by renewing their scholarships, staying current on scholarship payments, and making smart financial decisions
4. Managing life—by staying connected with people who cared, maintaining a positive attitude, and resolving problems that might challenge their ability to graduate
Next, we assessed how Bottom Line counselors were spending their time. We found that the majority of counselors’ hours were spent meeting with students who were well-organized and motivated. On the other hand, at-risk students often missed meetings and were reticent to engage with Bottom Line’s counselors. Counselors with the best of intentions would too often overlook the students most in need. And without a method to prioritize their time, they weren’t focusing on those who needed them most.

To address this problem, we started by creating a simple mnemonic for the four dimensions of graduation success, which we called DEAL, where each letter stood for the following:
- Degree (academic performance, on track to graduate)
- Employability
- (access to financial) Aid
- Life (emotional support)
Next, we worked with Bottom Line to create indicators for each of these four dimensions of graduation success. Students would be ranked on these indicators, using a color-coded scale where green meant “on track to graduate,” yellow meant “facing some difficulty,” and red meant “at significant risk of not graduating.”

At the beginning of school, all students started green—presumed to be on track to graduate—and the counselor’s goal was to keep them green. As the school year progressed, counselors and other staff would regularly update student rankings in a database and tracking system that produced a scorecard for each student. Counselors then used these scorecards to determine how and with whom they should spend their time. As a result of this simple approach, Bottom Line’s counselors were finally able to focus on the students who needed the most help to graduate.

DEAL also helped Bottom Line identify areas where extra support was needed beyond what counselors could provide. One such area was in helping students prepare for careers after college. As a result, Bottom Line instituted a new career program. As Dave Borgal said, “DEAL highlighted that employability upon graduation mattered, which caused us to design a program to accomplish that.”

Because DEAL is a simple yet powerful way to align the organization around its central goals, Bottom Line uses DEAL as a key part of its formal training program for counselors. And, DEAL has been instrumental in fundraising. Mike Wasserman, Bottom Line’s former director of development who now leads the Massachusetts offices, said, “Before DEAL, we would say, ‘We help students get through college,’ and ‘We do a lot.’ Now we can show the rubric, a sample intervention, and the toolkits. Our stakeholders can more easily understand the range, scope, and nature of services we provide students.”

With the implementation of DEAL, predicted graduation rates of Bottom Line’s students have risen to 85 percent. Process Thinking, when executed well, enables nonprofits to focus their daily activities where results will be the greatest, accelerate fulfillment of their goals, and ultimately increase their value to society.
June 7, 2016
How Excellent Is Your Program Evaluation?

Nonprofit program evaluation refers to the process of gathering data about a service or program an organization offers to determine its effectiveness. Nonprofits that thoroughly and strategically evaluate their programs show a commitment to the communities they serve by identifying their successes and challenges. More and more charitable donors now expect to see an evaluation plan in the programs they fund.

The Standards for Excellence: An Ethics and Accountability Code for the Nonprofit Sector identifies key areas of effective nonprofit management, and offers guidance on developing and implementing a successful nonprofit program evaluation plan.

Consider the Cost

A good rule of thumb is to outline your program evaluation plan as you are creating the program itself, at the very beginning of the planning stage. This allows you to include the costs of implementing your evaluation in your program’s budget, and many funders consider underwriting the evaluation costs as evidence of a commitment to delivering the highest-quality program. If your nonprofit has already engaged in delivering your program without an evaluation plan in place, don’t worry! It’s never too late to research cost-effective strategies and include them in your budget and planning session for the coming year.

Define Your Program

Nonprofits typically form to alleviate specific problems or address certain issues in society, and define mission statements that include a purpose and broad methods for achieving their goals. When drafting your program evaluation plan, be sure to begin with a comprehensive definition of your program at the beginning to establish the plan’s objectives. Move beyond the mission statement to identify the vital components of your program to help you determine an appropriate evaluation method for each one.

Monitoring vs. Evaluation

Nonprofits should be prepared to include both monitoring and evaluation procedures in their evaluation plans to ensure they are consistently and thoroughly analyzing their effectiveness. Monitoring refers to the essential, ongoing process of collecting information related to your program delivery and operations. Outlining monitoring systems early on ensure you won’t lose or miss valuable information that could be gathered during your activity and later assessed, such as how many people participated, how much time did we dedicate, how much money was spent or generated, or anything else that is important to count.
Evaluation goes beyond monitoring. It weighs the information gathered by your monitoring systems and assesses the impact you can claim as resulting from your program. Whatever evaluation methods you use, be sure to look at how your program has changed the conditions for your targeted population or problem.

Effectiveness vs. Efficiency

The key to determining program effectiveness is the identification of standards, benchmarks, or criteria against which progress or performance can be assessed. Perhaps you are evaluating the year-over-year results of an annual fundraiser, or the outcome of a program against national statistics. Be sure to include criteria in your evaluation plan that you can compare against your data to help identify specific results and areas of improvement.

Efficiency is determined by the ratio of outputs to inputs. Efficient programs offer satisfactory results that are achieved with an appropriate amount of dedicated resources. Paying attention to the efficiency of your program minimizes waste, expenses, and unnecessary effort.

Ready to learn more about creating an excellent nonprofit program evaluation plan? Members of the Standards for Excellence Institute and its Replication Partners have access to an extensive library of educational resources to help nonprofit organizations implement each area of the Standards for Excellence code. Our educational resource packet on program evaluation includes tip, tools and best practices to support your program evaluation development. Become a member and download your resources today!

May 16, 2016