Duco Steenbeek

The main aim of this thesis was to investigate the use of a 6-point Goal Attainment Scaling (GAS) system to measure change over time in interdisciplinary rehabilitation practice for children with Cerebral Palsy (CP). This thesis begins (Chapter 1) with a brief introduction of research in childhood disability and more specifically, research in Dutch paediatric rehabilitation practice. The objectives of measures and measurement issues in general are introduced. GAS is a generic individualized evaluative criterion-referenced instrument. It can be used for measurement of changes in individual patients and in groups of patients, generating ordinal data. The definition and classification methods of CP, and common treatment strategies are presented in the General Introduction. The preeminent outcome in rehabilitation of children with CP concerns a child’s activities and participation. The Children and Youth version of the International Classification of Functioning, Disability and Health (ICF-CY) is highlighted and discussed. It provides a useful framework for terminology, rehabilitation care policy and individual treatment goals.

Chapter 2 presents a study to measure the effect of Botulinum Toxin A treatment, performed using GAS. A single-blind randomized multiple baseline/treatment phase study across subjects was used for 11 children with CP. Each goal was recorded weekly with a standard video for a period of 14 weeks. Rating of pre-determined GAS scales was performed blinded. Nine of the 11 subjects showed improvement in 18 out of 33 goals. Seven subjects showed clinically relevant improvement in 11 goals. Testing the difference between baseline and treatment phase measurements for all GAS scores (n=33) resulted in improvement. An improvement was also found at subject level. The change in GAS score was related to the moment of treatment with BTX-A. We concluded that GAS could demonstrate clinically relevant improvement in individual rehabilitation goals, confirming the positive expectations of GAS. One of the key messages was the necessity for future research on the individual psychometric properties of Goal Attainment Scaling in patients with Cerebral Palsy. This study provided the foundation for the projects that followed.

A critical review of the literature about the content reliability, inter-rater reliability, validity, sensitivity to change of GAS and its use in paediatric rehabilitation research is presented in Chapter 3. Nine relevant articles reported promising qualities of GAS in paediatric rehabilitation, supporting the need for its further development. We concluded however, that current knowledge about its reliability, when used with children, was insufficient at that time. Moreover, its added value and responsiveness should be explored more specifically.

These conclusions resulted in the next project which was performed on the clinimetric properties of GAS. We started by developing a training program in the children’s unit of the rehabilitation centre in Breda (Chapter 4). The training consisted of three 2-hour general discussion sessions. In addition I provided the participating physical, occupational, speech-language therapists, paediatric psychologists and social workers with feedback until the GAS scales met the predetermined criteria of ordinality, described specific, measurable, acceptable, realistic abilities and activities in a single dimension, used the ‘can-do’ principle and could be scored within 10 minutes. This resulted in 115 GAS scales constructed and scored by the professionals. Therapists and parents were asked to complete a questionnaire to express their opinions. The development of a GAS scale remained a time-consuming procedure with an average of 45 minutes per scale. The content-criteria of GAS were found to be useful by all participants. Most therapists and parents regarded GAS as a suitable tool to improve the quality of rehabilitation treatment.

The first issue we investigated was the inter-rater reliability. Chapter 5 describes a reliability study in which 23 children with CP and 20 therapists of 3 disciplines were enrolled. The child’s own therapists, and for each therapist an independent therapist from the same discipline, constructed 64 GAS scales each. They scored both their own scale and their partner’s scale, resulting in 128 scores. The scales constructed by the children’s therapists had an inter-rater reliability of 0.82 (Cohen’s linear weighted kappa). The inter-rater reliability for scales constructed by the independent raters was 0.64. The major question that remained unanswered in literature was answered by this study: the inter-rater reliability of GAS was good, if professionals were trained. A key message was that these results also suggested that scale construction by the child’s own therapist as opposed to an independent rater has a positive influence on the inter-rater reliability of the scales. This finding is reassuring regarding the phenomenon of “therapist bias”.

The content of GAS scales was compared to the PEDI and GMFM-66 using the ICF-CY (Chapter 6). Twenty percent of goals were not covered by the PEDI or GMFM-66. Complete goal attainment was compared to changes on the standardized measures. Thirty nine out of 64 GAS scales were scored as complete goal attainment and of these 16 individual PEDI scores did not show change on the related scale. A key conclusion was that the instruments were complementary in construct and individual responsiveness. A substantial number of achieved goals could have been missed if only generic measures were used, even if the ICF-item matched the items of the standardized measures. Moreover, by measuring complete goal attainment alone, smaller changes could have been missed. This final study of the thesis contributes to the knowledge of the added value of GAS and contemplates the questions regarding the responsiveness of GAS, as first discussed in Chapter 2.

Chapter 7 highlights the lessons learned from this thesis. Positive attributes of GAS include quantifying the relevance of the change measured, facilitating the quality of rehabilitation care, the adaptability to the ICF-CY, the comparability of goal attainment across goals and individuals, the inter-rater reliability and the ability to measure progress that would otherwise not have been measured. Issues regarding GAS include reservations about the content validity related to the professional insight, the low construct and criterion validity and questions about the content reliability of GAS. Computation and data analysis should be performed preferably without the use of Kiresuk’s T sum formula. Future directions for research and implications for practice are discussed.

List of abbreviations

Botox: Botulinum toxin type A
BTX-A: Botulinum toxin type A
CanChild: Centre for childhood disability research in Canada
Children’s RAP: Rehabilitation Activities Profile for Children
COPM: Canadian Occupational Performance Measure
CP: Cerebral palsy
CI: Confidence interval
FSS: Functional skills scale (of the PEDI)
GAS: Goal Attainment Scaling
GMFCS: Gross Motor Function Classification System
GMFM-66: 66-item version of the Gross Motor Function Measure
GMFM-88: 88-item version of the Gross Motor Function Measure
ICF: International Classification of Functioning, Disability and Health
ICF-CY: International Classification of Functioning, Disability and Health, Children and Youth version
MACS: Manual Ability Classification System
MAS: Modified Ashworth Scale
MCID: minimum clinically important difference
NetChild: Network for Childhood Disability Research in The Netherlands
OT: Occupational therapy/therapist
PEDI: Pediatric Evaluation of Disability Inventory
PEDI-NL: Dutch version of the Pediatric Evaluation of Disability Inventory
PERRIN: Pediatric Rehabilitation Research in The Netherlands
PT: Physical therapy/therapist
QoL: Quality of life
SD: Standard deviation
SDC: Smallest detectable change
SMART: Specific, measurable, achievable, resource sensitive and timed
SPSS: Statistical package for the social sciences
ST: Speech therapy/therapist
WHO: World Health Organization