Exploring the Waters of Systems Change: An Interview with Ariel Slifka 

The Massachusetts Community Health and Healthy Aging Funds (the Funds) believes in the power of community engagement and the importance of addressing social determinants of health to create meaningful changes in health equity change. Recently, Caitlin Donnelly sat down with Ariel Slifka, a member of our evaluation team, to discuss her journey and the innovative approaches the Funds using to evaluate the impact of our initiatives.

From Rural Oregon to MACHHAF: Ariel’s Journey 

Caitlin Donnelly: How did you come to Health Resources in Action and the Funds team? 

Ariel Slifka: I’m from Oregon originally and I worked for a local public health department in a rural county. At this point I usually ask people if they’ve ever seen Tillamook cheese or Tillamook ice cream in grocery stores because I was working at the Tillamook County Health Department. It’s a really rural county, very tight-knit community—a lot of dairy farmers. I’m from a town of about 2000 people, so I have a soft spot for rural communities and the way that organizations and people can come together with limited resources and still make things happen. I think that is a familiar feeling for anyone working in community health or public health, or social services in general. I worked in harm reduction, reproductive health, and community coalition building, and we were using the collective impact model through that health department. 

Caitlin Donnelly: Can you say a little more about what you mean by coalition building and collective impact? 

Ariel Slifka: It’s the idea that you’re bringing together different organizations and fields across topic areas, maybe partners that you wouldn’t typically or traditionally work with, under one goal and vision. The idea being that often organizations are either working in silos or they have similar goals, and that if you can collectively come together and merge those resources and staff and capacity, you can have a bigger impact than you could all do individually. It’s not necessarily revolutionary, but it is terminology to kind of capture that type of community-based work. 

I really started to struggle after a little bit when thinking about how to measure success. And that’s what really sparked an interest for me in evaluation—how can we show potential funders the impact of this network building and this relationship building? Especially for someone on the outside that can’t see a clear cause and effect like someone working in the middle of it could maybe see. So, I went back to school and did my master’s in public health, specifically in community assessment and program evaluation, because I felt like I was missing a skill set that could really benefit the type of work we were doing there. 

Caitlin Donnelly: Tell me about joining the Funds team and learning about the evaluation process. What stood out to you about our approach? 

Ariel Slifka: I’ve been part of the evaluation team with the Funds for about a year, so in the grand scheme of things I still feel very new. But I think for me, that first piece that really stood out about the Funds overall was that focus on community engagement and relationship building, especially given my background in that work. That stood out—that it was a core aspect of the funding. And that in combination with the focus on addressing racial and health inequities, and that long-term focus on PSE change(policy, systems, and environment change)—those also felt unique and very valuable to this type of work.  

Another thing that feels unique is having that bigger picture of the collective impact of the Funds as a whole. When it comes to our evaluation, we really want to uplift that bigger story of the collective impact overall. 

Creating an Evaluation Tool that Flows Across Initiatives 

Caitlin Donnelly: We just piloted a new evaluation tool, the PSE-ARI, or Policy, Systems, and Environmental Change Approaches Report & Inventory. Where did the idea for this tool come from? 

Ariel Slifka: Yes. So, the PSE-ARI came from the idea that we have all this rich qualitative data around the stories, the themes from awardees, either through the Virtual Site Visits (VSVs) or the Grantee Reflection and Connection groups (GRCs). But they were missing some of those collective measures, highlighting how and what the grantees are doing within policy, systems, and environmental change.  

Caitlin Donnelly: Can you tell me about the Water of Systems Change framework and how that intersects with the PSE-ARI? 

Ariel Slifka: We spent a lot of time searching for examples of how other groups have done this and found plenty of examples for measuring how an individual initiative is working towards PSE change, but had a harder time finding examples of how that might look across a range of initiatives or a range of awardees. We asked ourselves, “How do you create a survey with questions or indicators that are relevant for grantees doing such different work? How would you measure PSE change across the grantees who are advocating for SNAP policies and another grantee who is connecting residents to housing resources?” These are vastly different types of work, and yet still exist under this umbrella of PSE change.  

We eventually came across the Water of Systems Change framework, which isn’t necessarily an evaluation tool, but it is a framework for thinking about different types of systems change. It gave us a way to see the different components of systems change that’s more tangible but still broad enough to be relevant for grantees. The framework breaks down six interconnected conditions: policies, practices, resource flows, relationships, power dynamics, and mental models. 

Caitlin Donnelly: Why did you choose Water of Systems Change over other frameworks? 

Ariel Slifka Yeah, I think we have been searching for different tools and frameworks; and there were some that were complicated, almost economics style frameworks that didn’t feel relevant for how we talk about systems change within the Funds. 

Coming across this triangle or this iceberg was a light bulb moment. Like, oh wait, this is capturing it. It was the first framework we found that included relational connections. Awardees talk about how important the Funds are for building relationships and coalitions in comparison to maybe other funding that they’ve received. And so, to be able to highlight that in a framework that already exists, it was a very validating moment. Sure, we need to adapt it a little bit, but it’s hitting on all these key points.  

Caitlin Donnelly: You played a big role in developing the actual tool that used the framework. Can you tell me more about that process? 

Ariel Slifka: The Evaluation Workgroup was key. The Funds convene a group of grantees, MA Department of Public Health and Executive Office of Aging & Independence representatives, and Funds Advisory Committee members about five times a year to inform our evaluation approach and activities.  The workgroup is part of our effort to practice equitable evaluation, by ensuring transparency, accountability, and shared decision-making.  We brought the original framework to the workgroup and asked, “Does this look like it represents the type of work that you’re doing? Is this missing anything? Are there key components that you feel aren’t included?” That was helpful. The workgroup is so engaged and thoughtful, and they provided feedback about what was landing and what wasn’t landing. 

Some of the terminology doesn’t quite match the language that we use, even though the concepts are still very similar. For instance, mental models is a term that I don’t see used very much. We expanded that to “mental models, beliefs, and narratives” because we already talk about narrative change. The relationships bucket, we expanded to include community engagement, since that’s a core component of the Funds. We added access to services into the resource flows category because that matches the way we tend to frame it. 

From there, we drafted an outline of what the tool might look like and some sample questions within each of the different categories. We brainstormed with the workgroup, thinking about, “Are these questions specific enough? Would awardees understand what is being asked, and would they have an answer for it?” We wanted it to be applicable to a huge range of work that’s occurring, which was a delicate line to be walking when developing the tool. The workgroup reviewed the full survey and discussed the questions, how they might be interpreted by awardees, and how to make them more applicable and relevant. 

Invaluable in this process was that members of the workgroup, if they had the time and capacity, fully pilot tested the tool, start to finish, with their teams. This process uncovered questions that were coming up while teams went through the PSE-ARI. We got suggestions for more guidance, and it informed the materials that we built out for the grantees when we fully launched the tool, like creating a webinar with more background and including a Word doc template to enable group brainstorming. Huge shout out to the workgroup for their time. I think this was all over the course of over like six months, if not more. It was something we wanted to take our time developing. 

Caitlin Donnelly:  How would you explain the PSE-ARI to someone unfamiliar with the Funds? 

Ariel Slifka: I think the PSE-ARI helps make what is sometimes not easily visible, more tangible. Sometimes the work seems straightforward like “I’m working on this one transit policy that I’m hoping to get passed.” But there are many other layers to pass through to get that policy passed, like working with a coalition of advocates which comes with a lot of relationship building, and that comes with power shifting. It also means that we’re shifting the community narrative on what it means to have access to transit, and that’s a mental model piece. To be able to put a name on all those buckets—I wish I had that tool a few years ago. It shows how awardees work on different aspects within their own initiative, and it also paints a bigger picture: almost every awardee is working on relationship building, or most awardees are working on power shifting. Those are clearly critical pieces to PSE work in the long run. It allows us to almost quantify how important those buckets are. 

Looking Ahead: Analyzing and Sharing Results 

Caitlin Donnelly: The awardees of Cohorts 1 and 2 have submitted their PSE-ARI responses. What comes next? 

Ariel Slifka: Huge thank you to everyone who’s taken the time with their teams to go through the tool. I know that although there weren’t a ton of questions in some of those sections of the PSE-ARI, they were heavy questions of really thinking through how your team has been working within these different buckets. It takes time and energy to go through that. We are slowly working our way through the analysis of responses, so more to come there. 

We will be working with the Evaluation Workgroup, of course. And then also with awardees as a group to do some co-interpretation. We want awardees to identify the important highlights from the PSE-ARI tool, and to tell the story of the Funds’ impact over the last five years. 

Caitlin Donnelly: Before we wrap up, can you tell us about the presentation that will happen at this year’s annual conference for the National Association of County and City Health Officials (NACCHO)? 

Ariel Slifka: Great question. For those who will be at NACCHO, we’re hoping to share some preliminary results and go more in-depth on the process of developing this tool with the evaluation workgroup and the Water of Systems Change framework. If you want more details, that would be a place for a much longer deep dive than this interview. 

Caitlin Donnelly: Thanks, Ariel. I’m looking forward to the findings! 

Learn more about how Health Resources in Action uses the Water of Systems Change framework to build capacity for equitable community engagement here.