Call for Narrative Submissions
Ever wondered what ethics means for evaluators practicing in Canada?
What are the important moments, dilemmas, and values that demonstrate how evaluators practice ethically?
CES Guidance for Ethical Evaluation Practice
What is this consultation?
To assess and support the rejuvenation of The CES Guidance for Ethical Evaluation Practice, the Canadian Evaluation Society hired external consultants to capture the feedback of CES members. This work follows-up on previous consultations conducted by other CES members to update the CES ethics approach. Working at an arms-length from this previous work, our goals are to:
-
Generate insights into the evaluation landscape in Canada to understand the context of evaluation ethics
-
Capture insights from CES members about their values, priorities, and reflections related to ethics in evaluation practice
-
Create recommendations for the National Board for the ethical guidelines that increase utility and support for you in your evaluation practice.
Who are we?
We (Jennica Nichols, Maya Lefkowich, and Alix Wadeson) are external evaluation consultants based out of Vancouver BC. Working at an arms-length from previous ethics consultations, we were selected by the CES from a Request for Proposals. Feel free to learn more about us here:
Jennica Nichols, CE MPH
Maya Lefkowich, PhD
Alix Wadeson, MPA
What are we asking from you?
We are inviting you to share a story that illustrates what ethics means to to you in your evaluation practice.
Wait! We know stories can be intimidation. And ethics? Ick. Where to even begin...
We created a fun exercise that guides you to reflect on a time when you faced an ethical dilemma or learned an important lesson in your work and helps us understand what ethics look like in your work.
You DO NOT need to be a confident or accomplished storyteller or writer to participate. You also don't need to have a philosophy degree in ethics.
All you need is 35 minutes from start to submission - a perfect break between Netflix episodes!
Why should I participate?
By sharing your experience, you will help ensure that the CES new ethics approach reflects your values and supports your practice.
And, as a thank-you, we will invite you to a members-only skills-building webinar that both shares findings from this consultation, and (b) explores this novel narrative approach to evaluation. This way, you can add this arts-based tool to your evaluation toolbox.
But, I'm not a confident writer or storyteller...
Perfect! You'll be in good company. We are NOT looking for a Pulitzer Prize-winning short story. We aren't even looking for polished spelling and grammar. Just share a brief glimpse into your experience navigating one ethics topic as an evaluator. That's it. And, because this is an exercise in reflective practice, we hope that participating in this consultation creates a meaningful learning opportunity for you that will inform (and strengthen) your future work.
If the topic of ethics is intimidating, that's okay. We are interested in the principles, standards, and/or values that guide your evaluation decision-making. We do not need an in-depth moral stance. You can check out our own examples to give you an idea.
Speaking of ethics, what ethical considerations are you taking in this project?
Great question. It is important that we practice what we preach. Here are 3 steps we are taking:
-
The story submission is anonymous. You will be invited to choose a pseudonym for yourself, any other characters you write about, and/or organizations. Or, you can keep your story in first person (i.e., "I remember") and use pronouns for other characters (e.g., they, she, ze). We will double-check for anonymity as well to make sure you cannot be linked to your story.
-
All of the narrative prompts we created are framed around hindsight. While it might feel uncomfortable to recall ethical tensions in your work, writing from the position of having worked through the issue can be a positive way to identify lessons you've learned and insights you now have.
-
We factored a break into the writing exercise to give you time to step away, return with fresh eyes, and decide what you feel comfortable sharing.
What will happen to my story after I submit it?
-
After December 10th, the external consultants will compile stories from across Canada and sort them by geography and theme.
-
Through chapter/working group consultations and a CES member survey, we will ask for CES members about their perspectives on the fit, relevance, and acceptability of the updated ethics approach. We may use stories or excerpts of stories as rich examples or important contexts to help anchor reflections.
-
Stories or story excerpts may be used as part of knowledge translation initiatives related to this project (e.g., in the final report, accompanying the new ethics approach, etc.)
-
We may create a composite story to capture themes from across multiple submissions to illustrate the complexity of ethical contexts in evaluation practice.
Who can I contact if I have more questions?
If you have questions about this consultation, contact Maya at maya@andimplementation.ca or click on the icon.
If you have questions or concerns about previous consultation work on the updated CES ethics approach, please contact Doaa Saddek, CES President: president@evaluationcanada.ca
How do I participate?
Follow the instructions below. When you're finished writing your story, click the "Submit" button to upload your piece and answer a few questions.
If you want to participate in the survey as well (or instead), click the icon here.
Story Activity
Instructions
Read through the instructions below and get started on your story in your own time.
Submit below by December 10, 2021.
Thinking about your evaluation practice, what comes to mind when you think about ethical evaluation practice? We are interested in your interpretation of ethics. When have you faced an ethical tension, dilemma, or important moment in your work? Read through the instructions below to build this memory into a story from your own perspective.
1. Read through the following prompts and pick the one that most speaks to you:
-
...Afterall, I'm an evaluator, but I'm only human
-
In that moment, none of my options were easy, but one choice was obvious...
-
Just then, I wished I had a magic wand to wave over the project and reveal the right way to move forward
-
...Thinking back on the project now, I realize that I was right the whole time. It would have changed everything if I had just followed my gut
-
...Thinking back on the project now, I realize that I was wrong the whole time. It would have changed everything if I had figured out my mistake earlier.
2. Find a quiet place to think with no distractions (silence your phone & email)
3. Set a timer for 20 minutes
4. Write down the prompt you chose. You could use the prompt as the first or last sentence of your story. Then, fill us in on the rest of the story. Describe a scene or moment in the story that illustrates the ethics issue. Jot down whatever comes to mind first and try not to delete or edit yourself as you go.
If you get stuck: As you walk us through your story (what happened), try answering the following questions to bring your description to life:
-
What was the incident that brought this issue to your attention?
-
Who else was involved?
-
What was at stake for you? What was at stake for everyone else?
-
How did you feel in that moment? Or, how do you feel now looking back on that moment?
5. When the timer buzzes, that's it! Finish your last thought and you're done.
6. Take 5 min to give it a read. Remove any real names or identifiable details (e.g., organization names) from your story. Feel free to use vague pronouns (e.g., they, she, ze) or a description in place of a name (e.g., "The Client" or "The Evaluation Firm").
7. Put your story down and walk away. When you're ready, come back with fresh eyes and make sure you feel comfortable sharing your story. Feel free to add or edit at this stage.
8. Give your story a title
9. Submit your story through the platform below.
Remember: we are looking for a brief glimpse into your evaluation practice, not a polished story. Write whatever you remember as you remember it. Don't over-think the story, spelling, or grammar (we are not grading this). And, keep in mind that with your short piece, you'll help inform how CES defines and supports ethical evaluation practice in Canada.
Plus, we'd never ask anyone to do something we wouldn't do. If you'd like to see an example, scroll down.
Story Examples
Example 1:The Shift
After numerous setbacks and delays, the final client meeting was already going to be tense. I dreaded having to deliver one more blow: the findings did not illustrate the story they wanted neatly gift-wrapped for their funder.
As I waited for the clients to log into the conference meeting, I weighed my options. Do I present a version of story they hired me to tell, or do I share the one they desperately need to hear? When I saw them appear on my Zoom screen I had my answer. While in that moment, none of my options were easy, one choice was obvious…
I cleared my throat. “Thanks for meeting with me today. I wanted to discuss a surprising finding together before I write it up in the final report. This project employed all white staff, recruited all white participants, and had positive impacts – but for all white stakeholders.” I paused as my voice began to tremble. “So, I’m just curious what you think about this.”
“We didn’t hire you to interrogate the demographics of the participants.” One of the program staff immediately jumped in. Her voice was stern and defensive.
“The program is open to anyone. This is just who happened to sign up. Surely you can’t read more into it than that. It’s just an honest - ” Another staff member chimed in trying to diffuse the tension.
“- Besides, this program has positive benefits for the whole community. Now, you’re calling us racist and trying to ruin our chances of getting funded again.” The first staff member cut off her colleague.
I felt a lump in my throat as I choaked down all of the words I wanted to say. “I understand this is difficult to hear,” I did my best to sound unphased. “I think we can name this for what it is - honest mistake or not. Are you open to exploring some recommendations?”
As I waited for their answer, I felt my allegiance shift from my clients to those not in the room.
The scene (what's happening, who is there, what is the context)
Ethical dilemma
Prompt
What happened from my point of view
Anonymous secondary characters
Secondary ethical tension or dilemma
Example 2: Privilege & Evaluation
When I was learning evaluation, I didn’t realize how much of how evaluations are designed or implemented depend on who I was and what I did. I was not good at articulating things that seemed obvious to me (wait? You also don’t think that…?). Soon after finishing grad school, I managed a small team and was tasked with conducting an impact evaluation for a place-based health intervention. Thinking back on that project now, I realize I was wrong. It would have changed everything if I had figured out my mistake earlier.
Ít was my first time managing a team doing an evaluation in Canada. I was very excited and also very nervous. After much convincing on my part, we finally were given access to interview people in the community to understand their health needs and their awareness, knowledge, and experience of using the program. On the very first day of interviewing I realized I had made a mistake. I had not thought about discussing ways to mediate power. So, as we entered the low-income neighbourhood (a place I had been a resident for the last four years), I saw all the ways my team made it clear they were different. Laptops to write notes, designer clothes, introducing themselves by their credentials, gold jewelry, and jargon. I was so embarrassed. How could I have missed this? Data collection was a disaster, one team member cried, and I was responsible for wasting everyone’s time. Worse, I had argued to my boss the importance and feasible of getting resident input… and had failed to deliver what I had promised.
As I headed home through the same neighbourhood after work, I felt like such a failure. I let my team down as I hadn’t prepared them. I let my boss down who took a chance on my idea. I had failed the residents who had so many important insights that could improve the program (and the neighbourhood). After my pity party, I realized that it was my responsibility to do better.
I went to see the program office in the morning to ask the outreach coordinator for some suggestions. I held a team meeting and talked about how I had failed them. I described moments in my own career where power dynamics had silenced me and invited others to share their experiences. I asked for suggestions and ideas about how we could do better. I took the lessons from these two meetings and revamped the consent and data collection processes. The following week we tried again. The data collection email included reminders to be mindful of attire, reminded people that notes would be taken using a notebook, and included an attachment with more accessible words for key things we were asking about. We started with a walking tour of the neighbourhood guided by one of the program volunteers and resident. I supported people to do their first (and sometimes second) interview until they felt comfortable. We collected data at natural gathering spots like in front of the grocery store, at the community centre, and at the playground.
Grounding in a memory or moment in time (being a student)
Prompt
The "set-up" why is this project ethically precarious?
Ethical issue
Anonymous secondary characters, "Boss", "Team members", "Program office."
Hindsight & learning
Abrupt & unfinished ending. When the timer buzzed, that was it. No tidy resolution needed for this exercise.
Thank you for your time & creativity!
For questions, contact Maya Lefkowich - maya@andimplementation.ca