Birth to Three Program Evaluation(part one of two pages) Introductionpart two > IFSP Introduction The Portage Project Birth to Three program researched whether their early intervention and training programs were accomplishing the intended objectives. They began this inquiry by looking at whether or not their early intervention program followed the four basic premises of the Growing curriculum. The first premise is that intervention is guided by the family. The second, that parent-child interactions are the heart of early intervention. The third is that family rituals, play, and daily routines, are the medium into which intervention is embedded. The fourth, that recorded ongoing observations and conversations are the basis for intervention decisions. Their hypothesis was that the early intervention program data would show support in philosophy and practice for the basic premises. Their training hypothesis focused on the belief that Growing training encouraged positive change in beliefs and actions for participants. These two hypothesis formed the basis for a larger inquiry. Portage Project staff members wanted to see if the intervention and training data would illuminate key defining concepts for the Birth to Three program and Growing training. Staff members used a variety of data collection instruments to answer their research questions. Home visitors collected family resource maps, parent interview questions (from Individualized Family Service Plans-IFSPs), IFSP goals, home visit summaries, parent-child interaction video and video replay. All families receiving services from the Portage Project Birth to Three program were also interviewed. For the Growing training, essay responses and pre and post parent-child interactions video reviews were also collected. The qualitative and quantitative analysis of the data revealed how interventionists and trainers were providing services from the heart of the Growing philosophy. The risk story that emerged from complementary data sources supports the belief that providing services in the context of a strengths-based, ecological framework that has relationships at its core, produces positive changes in families and providers involved with the Portage Project Birth to Three program. Research Questions and Data Sources Staff
members began by asking how they could determine if the four basic premises
were implemented in their early intervention program. The hypothesis was
that these variables would illustrate child and family progress as a result
of participation in the program. Training research questions focused on
whether the Growing training changed participants philosophies and actions.
Early
Intervention Program Research Questions
Training
Research Questions
The data collection instruments were reviewed for their validity and reliability. For purposes of this study the focus was on descriptive, interpretive, evaluative and natural validity. Descriptive validity tells the story of what happened in specific situations. Interpretive validity considers what these events meant to the people involved in them. Evaluative validity focuses on judgements about the worth or value of actions and meanings. Natural validity is the idea that the events and settings are unmodified by the interventionist’s presence and actions. A system of checks and balances can help determine if data is valid. In this analysis the data are contextually rich and meaningful. The stories and data make sense in the context of the early intervention program. The data from the various sources support each other and strengthen the quality of stated conclusions. These tests in combination add to the data’s credibility. To determine if the measures were reliable, dependability tests were used to assess any significant gaps in the research design. This meant that research questions were reviewed to determine if they were clear, and that the multiple interventionists were observed for comparable data collection protocols. Coding checks were completed to determine if different reviewers would have coded the data in the same format. The various methods were audited to determine if the research findings showed a meaningful parallelism across data sources. The research questions were strongly tied to the philosophical paradigms and analytical constructs that are the framework of the Portage Project Birth to Three program. Data Collection and Analysis Methods Data were coded and analyzed for emerging themes. These themes clearly demonstrated the key concepts inherent in the programs, and the effect of the programs on participants.
These explained what types of interventions and interactions were taking place during a home visit. They were completed by the interventionist after each visit. The forms are filled with concrete and complex information about what is important to families and what interactions occur during a visit. Family involvement in the intervention process is usually included on the form. The family’s priorities and concerns are often written as well. Each reviewed home visit summary sheet included data that could fit into at least one of the following categories. Categories were developed after reviewing all home visit summaries, noting recurrent themes. Categories were further refined so that data could be placed into one category at the exclusion of the others. They are as follows: Development: Data about developmental issues, including developmental delay and disability. Gross and fine motor, speech and language, cognitive, hearing, or vision issues were included.IFSP Goals In order for IFSP goals to be included in this analysis they glad to be in existence for at least 6 months. This meant that the goals would have been reviewed at least one time providing more information about goal progression. Goals that had been written in the child’s first year in the program, were often completed in later IFSPs. Goals which were coded as ongoing had not been fully accomplished at time of review although there was some movement toward completion. Goals which had been met and then expanded on the IFSP were coded as expanded. Parent Interview Questions (IFSP) These questions provide data on the family’s daily routines. Parents were able to provide information about what their child/baby is feeling, what they wished their baby would do, and what changes they saw in their child. This is important information that helped the interventionists to more fully understand the parents and the family as a whole. The data from these questions were used to show that the Portage Project Birth to Three program uses an ecological approach to understand and work with families. Family Maps This is a picture drawn by the family showing the important people, places, and activities of the family. It helped the interventionist to understand the family’s thoughts about their support systems, current relationships within the family, community, and with services. The conversation that accompanies the drawing and explanation of the drawing offered interventionists a great deal of information about what is important to the family and why. The map helped clarify important relationships and placed people and activities into perspective for all involved in the process. The family map served as another check on providing services through a relationship based ecological framework. Video and Video Replay Interventionists
taped parent-child interactions during home visits. These tapes were replayed
so that parents could review their interactions with their children. Interventionists
asked parents to name two or three things the parents liked about their
interactions with their children after viewing the tape a second time.
This helped show that parents could look for and talk about positive interactions
with their children. These data also showed that interventionists were
incorporating and encouraging interventions that fit into positive parent-child
interactions.
Parent Interview Families
in the Portage Project Birth to Three Program were interviewed to determine
what their experiences were with early intervention services in Wisconsin.
The interviewer asked for descriptions of services, feelings about programs
and interventionists, and beliefs about why the people and system worked
a certain way. Answers were coded for theme content which provided a clear
picture of what these families experienced in the Portage Project Birth
to Three program.
Training
After participants completed the three day training, they could participate in an additional fourth day (completed at a later date), and write an essay. For their essays participants were asked to choose two of the four basic premises of Growing and describe their actions and philosophies in relation to the premises. Respondents were asked to clarify which actions and philosophies were present before attending the Growing training, and which were as a result of attending the training. Essay data were analyzed by content analysis and comment reporting. Pre/Post Video Review Training participants watched a parent-child interaction video and recorded their observations before training content and materials were given out on the first day of training. Participants watched the same video and recorded observations on the third (final) day of training. Responses were reviewed and coded according to positive, negative, and neutral category definitions. Percentage change in positive responses was calculated from the coded data. Family Information The data included in this study were collected from 19 families whose children were enrolled in the Portage Project Birth to Three program at the age of 1 to 22 months and had been in the program a minimum of 10 months. This time frame was chosen to allow for repeated data collections through continued home visits.
Results and Discussion of Data Families had strong feelings and opinions about the effect of the Portage Project Birth to Three program on their lives. They talked about their relationship with providers, the ecological focus of intervention, and the focus on family strengths to develop and address goals. Relationship Between Provider and Family Many families thought of their home visitors as friends. They found they could talk to their interventionist about family and personal issues above and beyond their concerns for their child. Parents described their home visitors as “nice, caring, down to early, friendly, outgoing, kind, easy to talk to, and helpful”. Families felt comfortable with their visitors and looked forward to their time together. Parents felt that their home visitors were extremely responsive to their needs. The interventionists paid attention to all the family’s concerns and listened to them, whether they related to the child or family. Parents believed that interventionists were sensitive to their particular situations without being judgmental. Families were able to be open and share what was really going on in their families. They liked that home visitors helped them problem solve in a constructive manner. Parents believed that the Birth to Three staff truly cared for their whole family. Families said that their home visitor went above and beyond the call of duty. They cited numerous examples of staff members helping out when the family was in need, regardless of the task. Interventionists would offer help when family crises arose, such as an unanticipated hospitalization. Home visitors went grocery shopping or picked up Christmas baskets for families. They provided transportation and would accompany parents to medical appointments for their children if asked. Home visitors helped families learn about and apply for other programs or services. Families sensed that visitors offered much more of their time than was originally expected. All these actions were appreciated by families and helped to build a trusting, caring relationship between the provider and the families. The Birth to Three staff played an even greater role for parents who felt isolated. These parents saw their providers as helpful confidants. Having someone to talk to helped these parents identify their concerns and generate possible short and long term solutions. One parent was particularly thankful for caring, nonjudgmental and understanding home visitor. She stated that, “I have to thank her because I went from hating my child all the time to trying not getting mad whenever he gets mad. I’m calm and then I don’t hit him. She’s changed me a lot.” The home visitor’s warmth and sincerity was readily apparent to families. Parents felt connected to their home visitors and comfortable with the provider-family relationship. The home visitor was seen as a friend and someone who would go the extra mile to help out. Families stated an abundance of positive comments about their home visitors, showing that a caring relationship was established first, allowing questions, suggestions, and information to filter through it as needed for growth and development of the child and family. Parents continuously stated how much the program had helped their children and their families. They are obviously benefitting from involvement in this relationship. Although parents saw their home visitors as friends, they also looked to them for information and ongoing mentoring. Families liked the developmental information that home visitors provided. Families learned where their child was at and why certain activities would help her reach specific developmental goals. Parents felt that they gathered a diverse selection of activities they could do with their children that did not involve purchasing expensive, “developmental toys.” Home visitors suggested ways to interact and play with their children to enhance their development. Families felt energized and knowledgeable about how to play with their children and looked forward to showing the home visitor how their child had progressed in their development in subsequent visits. Home visitors provided a variety of information outside of the developmental arena. They were often asked medical questions and would search for information for families. Questions about other services and programs, especially transitions to early childhood frequented their conversations. Home visitors connected families to these other programs, offering assistance in making the transitions when the family requested additional support. Parents appreciated the knowledge and experience of their home visitors. They also liked the home visitor’s ability to define problems or concerns and frame strengths and possible solutions. Sometimes parents could not “step outside” their situation long enough to see another way to address a concern. Home visitors helped them see their way through difficult situations. Some parents described the program as a gift to their families. It provided them with help, a positive outlook, support, and a way to identify their family strengths. Parents were impressed with the variety of services available and the transdisciplinary approach used to carry them out. Parents thought this approach helped providers get the correct information and communicate and coordinate services more effectively as opposed to just reading a report. They felt this collaborative approach provided a better level of care and service than they had expected from an early intervention program. Parents talked about how the program had made a difference for their children. Parents felt that they understood their children better because of information and support from their interventionists. This increased understanding helped parents interact with their children positively. Confusion and stress were decreased as a result of involvement in the program. Parent responses were overwhelmingly positive about their child’s play based assessment. They were very comfortable with the assessment process. Parents were encouraged to be involved in the process in any role that fit their needs. They could observe, play with their children, talk with the interventionists, interpret what their children were saying or doing for the interventionist, or any combination of these interactions. Some parents wanted to take on an active role and some preferred to observe and discuss their children. Parents appreciated that they could decide their degree of involvement in their child’s assessment. Almost all of the play based assessments occurred in the home setting unless the parent suggested otherwise. These assessments were scheduled to fit in with the family’s daily routine. Families felt that the providers at the assessment showed respect by asking questions and listening to what parents had to say about their children. Interventionists asked parents how they felt their child was progressing in the different developmental areas being assessed. Families liked the play based assessment content. They enjoyed seeing the transdisciplinary team members playing with their children on the floor. Parents felt that the interventionists showed a great deal of care and concern for their children, which helped them to feel positively about the assessment. The intervention planning process focused on family strengths. Parents were asked what their goals were for their children and their family. They discussed family strengths and concerns with the interventionist. The goals were developed by reviewing family and child related strengths, concerns, and goals, in addition to the play based assessment information and observation. The individualized family service plan (IFSP) was developed to build on family strengths and address concerns to meet the family’s goals. The process of intervention planning was comfortable for families. It was a discussion between parents and program staff, with both offering observations, assessment information, and realistic goals for the children and family. Most parents felt that the information from staff helped them to set appropriate family and child related goals. A few parents would have liked more input from the Birth to Three Program concerning what developmental goals should be set for their children. Families mentioned methods that helped them to clarify the goals for their children and the family as a whole. Family maps and visual charts were used to help families identify strengths and concerns. Maps generated goals that parents had not considered previously. These maps illustrated a holistic view of the family’s life, which helped generate goals that fit with the family’s current situation. Families generated their own goals. They were the major contributors to this process. The family or child related goals that were most important to parents, discussed with the team and included in the plan. Families appreciated that when they did not want a family concern or goal included in the IFSP it was not recorded. In some instances parents wanted to talk about family issues, but did not want these issues, “worked on,” during intervention. Families felt that the Birth to Three program and its staff were responsive to and respectful of what was important to the family, even if it was not directly related to early intervention. Families appreciated that their regular home visits fit into their daily routines. Every parent explained that the home visitor always asked what day and time would be best for the family. Birth to Three staff did not fit the family into their time slots, they fit themselves into the family’s schedule. Home visits were rearranged when family time conflicts arose. The flexible nature of the early intervention program placed the needs of the family’s life first. Parents noticed how the interventionists included the whole family in a home visit. Siblings were included in activities. Families liked that they didn’t have to make special arrangements to participate in early intervention. They did not need to arrange for a baby sitter or set up separate activities when the home visitor arrived. They liked the inclusive nature of Birth to Three. Although the interventions were family centered, the child related concerns that families held were clearly addressed. Home visitors worked with the family to meet the goals they had for their child. Families liked that visitors got down on the floor and played with their child. Parents mentioned various methods employed by home visitors; playing with different toys, teaching sign language, trying out new words, showing different sitting or standing positions, reading, and signing. Observing how the child is learning and changing were also part of home visits. Using a transdisciplinary team approach, the home visitors could incorporate speech and language therapy, physical therapy, occupational therapy, or special instruction approaches to intervention during a visit, as was appropriate. This transdisciplinary approach was appreciated because the parents felt that they had developed a relationship with their visitor who was responsive to the child’s and family’s needs. Families did not want another person and another appointment to schedule, as they felt the time crunch of all their other family responsibilities. Talking with the home visitor about family and child related concerns was a comprehensive piece of intervention. Families articulated a variety of concerns. Sometimes just the act of asking the family how they were would get parents thinking about their greater family issues such as jobs, money, family relationships, health, stress, and time constraints. In addressing these larger family concerns, home visitors followed the lead of the family. Usually this meant they listed and helped problem solve, providing information when asked, but they did not force families to face or address any issues. Nor were these issues included in the family plan, unless the family wanted them written down and addressed there. Some parents wanted more child focus or did not feel the need for a family focus. These tended to be families with other supports. Sometimes parents did not understand that play was the intervention and wondered what the interventionist was accomplishing by playing with their children. Suggestions for Program Changes When asked what they would do if they did not like something about the program, all parents responded that they would talk to their home visitor about changing things. They felt comfortable enough to discuss program concerns with their home visitor. Parents
wanted to continue with the Birth to Three program past age three. Some
also wanted services in the summer. Parents also wished that they had known
about the program earlier and suggested advertising more aggressively;
especially with pediatricians and nurses. Some parents wanted to see more
people at playgroups so that they would have a network of parents in the
area.
Comparison to Other Early Intervention Programs Family interviews were also completed with three other early intervention programs in Wisconsin. Their comments illustrated where the Portage Project Early Intervention program was distinctive when compared to these additional sites. One of the largest differences dealt with the focus of early intervention visits. Portage Project provides an ecological approach, addressing family and child related issues as the family prioritizes them. In the other four sites, families usually did not talk about family issues. The focus of intervention was mainly on the child. Parents did not think the role of early intervention was to work with the whole family, they waw it as the interventionist working with the child. Family related concerns were not the center of intervention. In the Portage Project program, all aspects of intervention, from assessment to planning to implementation, focused on the family’s strengths. In all of the other four sites parents did not remember being asked about family strengths. A few parents thought that family strengths might have been asked about, but they weren’t included in any family plan. The lack of discussion of family strengths is in stark contrast to family interviews from the Portage Project. Parent involvement varied across the four sites. Parents usually were not an integral part of assessment, planning and intervention. They participated at times, but interventionists often took the lead in all aspects of the early intervention program . Interventionists asked questions of the parents during the evaluation and interacted with the children. Service coordinators and interventionists often came to the intervention planning meeting with suggestions for goals and IFSPs partly completed. This did not mean that parents could not contribute to assessment and planning, but that they followed the lead of the interventionist, except when there were strong objections. These four sites did not offer a coordinated transdisciplinary team approach. In families where a variety of approaches (e.g., SLT, OT, PT, special instruction) were warranted, parents would need to set up multiple appointments with multiple interventionists. This was true regardless of whether therapy happened at home or a center. Some programs were trying to encourage parents to switch from home visits to a center based approach because they felt the cost of delivering home based services was too high. To facilitate communication between all the providers involved with a family, a notebook was passed from interventionist to interventionist, explaining a child’s progress during a given session. One program used a “developmental book,” to determine appropriate interventions for the children in her program. Parents felt they learned a great deal about their child’s development, what they should be doing, and why certain activities would help their development, but this learning did not seem to extend beyond child development. The parent interview data from the Portage Project Birth to Three program clearly demonstrated that families felt that their home visitors were a part of their lives. Their relationship was mentioned first, with the child’s development being a part of that relationship. In the other four sites, families liked their providers and felt that their children were growing and developing, but they did not state as strong a connection as the Portage Project families did. The other data collection tools; home visit summaries, IFSPs, family resource maps, and videotape replay, offered further support for the early intervention research questions. These methods illustrated the strengths based, ecological approach to intervention. The developing relationship between interventionists and family members obviously supported the growth and development of the child and family, as well as the continued interactions between the family and the early intervention program staff. Home visit summary data provided more support to the key concept of ecological intervention. Interventions were focused on the family priorities, needs and strengths. As families noted in their interviews, they discussed child related and larger family issues during home visits. The developmental focus that brought the family to the attention of Birth to Three was noted on 81% of the home visit summaries that were reviewed. This showed that even when parents mentioned how much they loved being able to talk about a variety of family concerns, their child’s growth and development was observed and discussed during most visits. Interventionists were able to fit child specific developmental issues into the context of the family, discussing and working on developmental progress through the ebb and flow of larger family issues Home
Visit Summary Categories
Family issues such as housing finances, employment, counseling, support, child care, and legal issues were expressed on home visit summaries 47% of the time. Health issues were noted 30% of the time. IFSP reviews or assessments were included 24% of the time. Information sharing between home visitor and caregiver, where educational materials might be discussed, was noted 18% of the time. Praise and encouragement were in 16% of the summaries. Suggestions or recommendations from home visitors to families were included 32% of the time. The variety of topics showed the depth and breadth of interaction that home visitors had with families. In addition to the developmental progress and suggestions, interventionists included the important family concerns that were discussed during the home visit and kept track of these ongoing issues for themselves as well as for the family. The families received a copy of the visit summary for their own use. The
interventionist often included information about how the caregiver was
interacting with the child to improve developmental outcomes. As the caregiver
learned more about child development and the activities that could facilitate
it for her child the home visit summaries indicated this increased awareness
of child development. Summaries also showed family requests for additional
resources or supports Examples of family members learning about resources
and taking steps to acquire these services were clearly stated. Interventionists
were able to record and monitor the salient issues from the family’s point
of view. This provided a record of progress and a sense of what was discussed
and worked on in the course of home visits.
|
back
to Portage Project Training, Julia
Herwig, USA