The Whole Story: Education, Housing, and What the Qualitative Data Tell Us
Challenges in areas ranging from education to the environment, gender to governance, health to housing don’t exist in a vacuum. Each month, Abt experts from two disciplines explore ideas for tackling these challenges in our monthly podcast, The Intersect. Sign up for monthly e-mail notifications here. Catch up with previous episodes here.
To address education and housing challenges, you don’t need some of the data, you need all of it. Abt’s Cara Jackson and Henry Love explain how qualitative data can help researchers better understand the obstacles that families and communities face, enabling all parties to devise solutions together.
Read the Transcript
Eric Tischler: Hi, and welcome to The Intersect, I'm Eric Tischler. I have associates tackles complex challenges around the world. Ranging from improving health and education to assessing the impact of environmental changes. For any given problem we bring multiple perspectives to the table. We thought it would be enlightening and maybe even fun to pair colleagues from different disciplines so they can share their ideas and perhaps spark new thinking about how we solve these challenges. Today joined by two of those colleagues, Kara Jackson and Henry Love. Tara conducts education research and evidence reviews on a variety of topics, including interventions intended to improve educational opportunities and child outcomes, policies related to school and teacher quality and school network expansion. Henry is a developmental psychologist and educator with 10 years of experience implementing, developing, testing, and evaluating innovative programs to improve the lives of BIPOC families and living in extreme poverty.
Welcome.
Cara Jackson: Thanks. Great to be here.
Henry Love: Oh, thanks. Great to be here.
Eric: We need data to identify challenges and we need it to verify solutions that work. That includes getting insights from people with lived experience. And sometimes the data they can provide is qualitative, not quantitative. Nowhere is that more true than when working with children and families. How do we benefit from qualitative data and how do we collect it? Kara and Henry, I'm going to ask the two of you to sort talk about how you got into your respective fields; you have a very similar stories and I know you really generate and really rely on qualitative data. So, Cara, you want to go first?
Cara: Sure. Thank you. So, my first job out of graduate school was as a federal policy analyst, but I really developed a better understanding of education when I became a classroom teacher. So I taught pre-K and kindergarten in the South Bronx. And I learned so much from that experience. My students were a very diverse group. A lot of their families had recently immigrated. Some of them lived in transitional housing or were moving in and out of foster care placement. So it was a really up close look at what goes on in children’s daily lives, the challenges that they face and I learned a lot of about the contextual factor that can spill over into challenges that they might face at school.
Eric: Now, some of those insights you got, you had to learn over time, right? That wasn't written down on a piece of paper when you showed up to class, right?
Cara: Definitely. It was really all about learning about individual students and their experiences in the different countries that they came from and different experiences that their families had and things that they struggled with, and they weren't experiences that I had had as a child. And so it's very much a learning process.
Eric: Gotcha. And how about you Henry?
Henry: So, similar story. After undergraduate school, I really wanted to move to New York and I couldn't get a job. And the only job I could get was as a teacher and I did not want to be a teacher because my mom had been a teacher for about 50 years. And so I ended up teaching in Central Brooklyn as well as East Harlem. And a lot of my students were also from the South Bronx. I had several students who were really showing really extreme behaviors. I had a kid that would run literally up the walls all the time and like climb, and another kid that was really angry and would break things all the time, like windows and glass. And there was another one who would defecate and hide his feces around the room.
So it really got me questioning, like, what is at the root of these behaviors? Why are they exhibiting them and why, as a school and as an institution, are we not empathizing with their circumstances? And that led me down that process of really understanding their own individual stories, like Kara was saying and really thinking about what was happening. And as I got to do that, it was the exact same thing where you find out that they're maneuvering in between foster care. The kid that's literally running up the walls was sleeping in a shelter. And so that was sort of my entry point into thinking about this intersection between housing and education.
Eric: Right. Thanks! So who wants to share the twist ending to this story before we really get into it?
Henry: So after I decided to go back to school to further study these things, I ended up working on an intervention called Attendance Matters that was focused on improving school attendance for kids in shelter. And so I was on the ground and working with shelter providers, schools and whatnot. And Cara and I found out that the school that she used to teach at was one of the school that I worked the closest with, and the shelter, that was their main feeder for students that were experiencing homelessness. So there was that connection.
Cara: Really small world.
Eric: Yes. So let's talk about the challenges of using the data we've traditionally had to address these issues versus, needing that qualitative data and Henry you want to take that first?
Henry: So one of the challenges I think for us, at least in that particular circumstance and with Attendance Matters, was we had a team of researchers and folks who are coming from a very academic perspective and then folks on the ground, social workers who, for the day-to-day, haven't really been using data to drive their everyday actions. And so I think really helping them tool up to understand how to process data, how to use it. And then looking at attendance data is not the end of the story, but an entry point that’s really guiding us to start asking critical questions about certain kids who had the worst attendances and really starting to see it as not end all be all, but as something that's an indicator or barometer, if you will, of family function.
Eric: Great. How about you, Cara, because data collection is something that you work on, what have you seen? What's working what isn't working?
Cara: So first I would say there's a lot of value in qualitative data. And to Henry's point, you learn a lot from student stories and experiences that isn't immediately obvious from any number that might be before you, and those numbers aren't generated in a vacuum. The context that the data are generated in, it could be very supportive of children and families, or it could be one in which families really struggle to meet basic needs, children suffer from food insecurity or unstable housing. And those qualitative facets of children's life can influence outcomes. And I have two specific examples of the importance of the data generation context. So ,one year our school was scheduled to administer a standardized test during Ramadan and people fast during Ramadan. And there's reason to believe that might influence how they perform on the assessments, right? There's at least one study that shows that school district that increased calories on test days, experience increases in fifth grade pass rates on standardized tests. So it's not hard to imagine that, if you administer an assessment, when some students are fasting that could affect their perform.
And then another example that is a district survey to ask if families were able to support remote learning, but the survey was online, the people who are responding to that survey are more likely to have computers at home and more likely to have internet access. And the missing data there is from people who don't have computers or internet access. So we have to be sort of careful about where the data are coming from and what we're not seeing.
I've written before about the need to democratize the development of evidence and what I mean by that is that we could be seeking community input from the very beginning, before we even collect data, asking the community what questions they have, thinking about whose story is being told. And from what perspective is that story being told and where we might have limited information on certain perspective. And that leads to this question of how do we engage people in a meaningful way? What are the best ways of bringing them?
My school district has done an admirable job of translating surveys into seven or eight different languages, but there's that issue of families with limited internet access or limited ability to engage with the district in the format that they're engaging families with. So engaging families in a meaningful way might look a little bit different than what our preconceived notions are.
Eric: Right. Do we have tips for how to close that communication gap?
Cara: Okay. So I think one way and, as I said, my district has translated the surveys into a lot of different languages. So I think that's an important first step, right? I think that developing evidence requires also making a lot of different decisions about what questions to ask, what outcomes to focus on, who gets sampled, how you handle missing information, how do you analyze things and interpret the results? So none of that is entirely neutral or objective. And researchers are making a lot of decisions that could shape what the evidence seemed to be telling them. And those decisions tend to be made without input from the community's impacted.
So one of the things we do to democratize the development of evidence is to start by asking the community what questions they would want asked and how they would interpret information that they're seeing. Ask them if we're even asking the right questions, are we getting the right information? Are we analyzing it the right way? Are there other contextual factors that we haven't been considering? Where students go to school is based on where they live, and that has implications because we know that neighborhoods impact kids, too. Do their buildings have lead paint? Do they have access to public libraries, to playgrounds, to healthcare facilities, even grocery stores?
Eric: Henry, you want to jump in ?
Henry: I think to Cara's point it's when you... And I think this is often the case when, I think, we talk about quantitative risk versus qualitative data is that, even the quantitative data, it's not objective, free-- biases are informed by what instruments or people are being are selecting, what items they're choosing, how they're choosing to ask those items, right? And then how are communities then responding to those questions that are being asked of them by researchers. And I think one of the things we found on the ground with their intervention too, was in terms of looking at attendance data where you see just numbers and you see kind of patterns, you get a glimpse of the story, but the narrative around the child is what's so important in really contextualizing and understanding their circumstances, which are then resulting in a negative number, right? Or not successful education outcomes.
And so I think so much of that is really contextualizing, when you start changing your perspective and asking questions that are really guided by those most impacted. For us, it shifted away from, in the beginning so much was put on like, "Okay, we need to think about transportation resource and da, da, da." And it was like, for the kids who had the most extreme issues, a lot of it was rooted--in one case I'm thinking of a couple of families who were from Puerto Rico and came as a result of the hurricane and went through that trauma of losing their family, seeing people die, losing all of their things, being displaced, right? And so these things get at much more larger complex questions. And I think that's where the qualitative piece can come in, to sort of fill in some of that, so working in tandem with those quant numbers to really get a full picture of what's happening on the ground.
Eric: And when you bring up fleeing a hurricane, it feels like that’s social determinants of health, where if we can get that data and weave that together, it's no longer purely an educational piece. It helps inform environmental issues, resilience issues, and so it's not even just helping that child, but helping us identify broader trends that we can be addressing, right?
Henry: Right and, as you said something just sparked in my mind, literally thinking about this connection with social determinants of health, is that where the school that we have that connection at in the Bronx is in the middle of Asthma Alley. So I entered this asthma thing in a very different way than I had ever anticipated because there was someone from the Department of Education who was like this really huge asthma advocate. And we realized that kids weren't missing a week of school because of asthma, but it was like, "Okay, they were missing a day a month" and those days a month add up, right? And so you see this little intersecting piece of, like, where are they living? Where are they being placed? And how does that fit into these health-related issues that are also impacting their development and educational outcomes?
Eric: Right. So that's like almost a one to one for environment and education, or is a one to one, within that—housing, because that environment is literally where you're housed. Cara, I think you were going to say something.
Cara: I'm just fascinated that asthma came up because one of my most vivid memories is that one of my students, when I was teaching pre-kindergarten, our pre-kindergarten classrooms were in a trailer outside of the main building and the nurse's office was inside the main building. And one of the parents was very, very nervous about his daughter, who had asthma. And he was afraid if anything happened that she would have an asthma attack. I'd have to like physically take her up and up flights of stairs and into another facility to get her help. So everything is related. And I think when you're responsible for a classroom full of children, in some ways you end up being very responsible for their health as well. And so that just a good example of how everything is connected.
Henry: And as researchers, I think when you're looking at educational outcomes, the first thing that comes to mind is probably not asthma or housing. And so it's how these things connect... And this goes back to like asking those questions and really pulling in members of the community, because I guarantee you that that was the first thing that father probably talked about was his child's health and wellness, right? And the research process is going to determine, like, "You know what, maybe we should think about what we're measuring and what kind of tools are we even going to be asking a community to engage with," as opposed to just assuming that we already know.
Eric: Moving forward, for each of you, what are you working towards, to sort of get to the right questions and get to the right answers? Cara, you want to go first?
Cara: I think a lot of listening is important and making sure that you're being reflective about who you're listening to. Trying to diversify where you're getting your input from and thinking about who the different audiences are for whatever it is that you're working on. So a lot of our work is with clients who are, government agencies but the people that they're trying to help are also part of this bigger picture. And I think that's what both Henry and I are very invested in saying. It's important for us to understand the community that we're trying to serve and not just the clients who are paying for the work.
Eric: And, to your point, an online poll may not be enough to get the input you need.
Cara: Right.
Eric: Henry, how about you?
Henry: I think my biggest takeaway from just like my experiences as a practitioner, as an educator and leading that intervention was there are some really big just systematic things that need to be addressed. And so I think moving away from these little interventions, these band-aid fixes and really helping government, researchers think about these more complex structural issues. Sort of like how structural racism is intersecting with housing policy and then like, what does that mean then? Because then from that you draw out these issues that can be raised about asthma and education outcomes in the schools. And so I think the work that I've been doing now with Abt has really been a lot of work around guaranteed income projects and really thinking about how do we give folks the resources that they need in the way that they would like them to be able to enable the best decisions for them and their families.
Eric: Of course, you need their input for that.
Henry: Right, and I've worked along with communities and with folks to really think critically about that, because I think even how we're studying the outcomes of some of these guaranteed income projects or cash transfers--so many of these interesting things are coming out of from community voice about what they have realized that they're able to do, but also just the way that programs are rolled out. I think there's groups that are like, "Oh, we're going to use a phone to do it X, Y, Z". And it's like, "not everyone has access to that," but if you engage people and you talk to them about the process, they'll tell you, like, "What would be better is a debit card that I can be able to just go in, do X, Y, Z", but that doesn't come out unless you're engaging with community members at every step of the way.
I think the asthma thing has really resonated with me, especially with COVID and because I was doing the intervention as COVID was starting to become a thing. And I think what was so fascinating is that shelter that was feeding that school had people... We were thinking very critically about these issues. And we knew that the virus was going to come to New York, and we knew that it was respiratory. And we knew that it was going to wreak havoc on this community. And it was just interesting because then it goes back to data, and we had the data. We knew that these students didn't have the same type of access to internet and to whatnot, because we had been documenting it. We knew which families had issues because of attendance. We knew who had the asthma risk. And so, in some ways, that particular shelter and that school were very well positioned. I think, in any other one that I worked with to navigate what was coming with the pandemic. And so much of that was defined by data.
Cara: Yeah. It's an interesting point to bring up because I think one of the things that we're seeing now in our district is differential access to the rapid [COVID] test, right? So, if your parents could afford to get test, you've been testing every time they get nervous. And then if your parent can't afford that, then you maybe went to school and found out through, if you were signed up to do the random testing at school, you might have found out there that you had coronavirus. So it's definitely another data issue. It's like, how much information do we have? When do we get it? Do people have equitable access to that information? That kind of thing.
Eric: There's data and equity, right? Equity, data, getting everyone to be involved, and getting input.
Henry: As we were talking and I'm reflecting, I think this notion of like, of The Intersect and in that community in particular, because I think it's so many overlapping things--like that school and that shelter, the freeway runs through it. The school's on one side, this shelter's on the other side of the freeway, but it's literally the poster child for, like, what Robert Moses did with displacement of communities and environmental racism and all of these issues that are layered up. And that's ground zero for this sort of conversation, I think, nationally.
Eric: Right? And that seems like a great place to wrap up a podcast that’s called The Intersect. Thank you both for joining me!
