Had it not been for Covid-19, I would have been reflecting on my experience with the Faros Horizon Center in Athens, Greece. I would have had a chance to interact with the refugee youth the program is serving through design classes and psychosocial support. I would have had a chance to witness in person the hard-working and dedicated staff at Faros trying their very best to meet the boys’ needs. But just like everyone on Earth, my summer had to be redesigned.
With the support of my supervisor Laura Budzyna, and Martha Thompson and Heewon Lee from the MIT D-Lab Humanitarian Innovation team, and the management team from Faros, the NGO that created and oversees the Horizon Center, we redesigned my summer evaluation project. Since I was not able to travel to evaluate the program’s impact on the youth, and the Horizon Center had to teach remotely because of the Covid-19 lockdown, we decided to conduct a process evaluation to measure how effectively the program is being implemented, how adaptable it is to change, and its impact on staff and MIT D-Lab students.
On Adaptability
The most important theme this summer for me is adaptability. Not only did I personally have to be adaptable to the current pandemic situations (from my project’s changing trajectory to waking up at 5 a.m. for a meeting across time zones), I also learned how tremendously adaptable the Horizon Center program is. Eighty percent of the staff respondents feel that the program has been adapting well and efficiently to changes. The recent and ongoing impact of Covid-19 is a great example:in the beginning of March, the program immediately created design activities for the boys to safely complete at home, taught online classes, and disseminated homework via online and social media platforms. This adaptability is made a reality mostly due to the program and staff’s readiness and willingness to change, staff’s dedication and commitment to meet the boys’ needs, and effective collaboration between MIT D-Lab and Faros. I truly believe adaptability is ingrained in the culture of the program.
The flip side of this is that 40 percent of the respondents also felt that the program is too adaptive. Adapting too much and too fast can present a risk for two reasons; one, it can contribute to staff fatigue and frustration around an uncertain mission, and two, youth experiencing trauma from their uncertain living conditions benefit from structure and predictability. While the program is very adaptable, there is more work to be done in order to prevent staff fatigue while still meeting the needs of the boys.
On the Program’s Impact
I was blown away by the impact the Horizon Center program has had and is still making on the staff and students who have participated in it. For Faros staff, not only is the design mindset and process now being used outside of work (and even taught to their own kids!), but also their engagement with the program has helped strengthen their leadership skills, boosted their teaching confidence and for some, created a sense of fulfillment through their services to the youths. For the MIT D-Lab students, the program offered a teaching opportunity for them to demonstrate leadership skills, opened their eyes to the humanitarian context and refugee crisis in Europe, and contributed positively to their desire for a future career related to humanitarian work. Six out of eight students surveyed envisioned engaging with the skills and knowledge they have acquired through the program to pursue a job or career in the humanitarian innovation sector.
On Youth’s Psychosocial Evaluation Measurement
In addition to the process evaluation, my other priority this summer was to help the team research and develop evaluation instruments to measure the program’s impact on the youth. Throughout this process, I was able to connect with several psychologists and researchers from Columbia University to learn about the measurement frameworks currently being used in this particular field. Through my research this summer, resilience seems to be the most salient theme on this topic. Several researchers have developed evaluation instruments to measure resilience in youth, for example, the Child and Youth Resilience Measure.
For the remainder of 2020, as long as there is not a second lockdown to prevent Covid-19, the program’s designers and instructors will prototype and test these instruments with the youth to see which evaluation instrument would be the most effective for which group of boys. I am excited to continue following this project and hope that my work has been beneficial to support the program’s mission and goals. I find the process of measuring the psychosocial and psychological impacts of an intervention on refugee populations to be both challenging and intriguing, particularly when working with youth who sometimes do not fully verbally convey how they feel. Having spent the summer working on this specific priority, I saw myself being even more passionate about the mental health and psychosocial support work in the humanitarian settings and hope to continue my research during my upcoming last year at Columbia University.
On Evaluation
Before being offered the MEL Fellowship with MIT D-Lab, I knew very little about monitoring, evaluation and learning (MEL). Most of my knowledge about this field had been on impact evaluation and its importance in assessing the effectiveness of a particular intervention. I did not know there were so many different types of evaluation, including the type I used to conduct my project over the summer – process evaluation. This is a method of evaluation which focuses on assessing the first three components of the logic model (inputs, outputs, activities) to determine whether the program is being implemented how it is intended to be. Better Evaluation, an NGO whose mission is to “create, share, and support use of knowledge about how to better plan, manage, conduct, and use evaluation,” was my evaluation bible for the summer.
With the support of MIT D-Lab’s MEL team Laura Budzyna and Artie Maharaj, as well as the team of other MEL Fellows, I was able to expand my knowledge in this field,which is crucial for a future development practitioner like me.
Reflection
The last three months have truly been a summer field placement redesigned. Just like the Horizon Center program itself, I learned how to be okay with uncertainty and how to utilize existing resources and tools to handle changes. Just like the Horizon Center staff, whose flexible mindset is one of the biggest reasons for the program’s success, I practiced having a redesign mindset and staying calm when faced with adversity and challenges. Just like the Horizon Center youth, whose resiliency and character triumphed over their unpredictable lives and difficult living conditions, I learned to rise up again when I felt defeated and upset that this summer wouldn’t be the one I expected to have this year. While Covid-19 took away a lot of good things, it also brought plenty of good lessons. At the end of day, there are just too many things to be thankful for, and my experience being an MEL Fellow at MIT D-Lab is indeed one of them.
My final reflection on this experience is that it is okay to redesign; in fact, we should welcome redesigning. We need the redesign mindset in order to prepare for the uncertainty we might face, and the Horizon Center program taught me how to do that well!
About the Author
Khanh Vu is a Vietnamese citizen, pursuing an MPA in Development Practice at Columbia University School of International and Public Affairs (SIPA). At Columbia, she is currently a Program Assistant to the MPA-DP program and a Design Fellow at Columbia Entrepreneurship, Innovation, and Design. Her primary interest lies at the intersection of innovation and mental health and psychosocial support work in humanitarian settings and the design thinking process. Prior to joining SIPA, Khanh spent five years living in Washington D.C and gaining professional experiences in digital fundraising, monitoring and evaluation, and nonprofit communications. Khanh is also one of the co-founders of a wellness mobile application attempting to find a unique solution to the common mental health distress among Vietnamese youth as well as strengthening a wellness ecosystem in Vietnam. When she’s not scrambling around trying to finish a paper, Khanh can be found climbing some rocks inside or outside, biking around the city checking out free samples at farmer markets, or hiking and hiding in the woods roasting marshmallows with her beloved pals.
More information
MIT D-Lab Humanitarian Innovation Program