DISCUSSION:Thisstudy explored the relationship between typically developing children’s verbalreasoning skills and performance on national curriculum reading comprehensiontests, cognitive and language skills. The study additionally investigatedpsychometric properties of the assessment such as inter-rater reliability andperformance on different scenarios. The findings, limitations and clinical implications are discussed below.
Relationship between language and cognitiveassessment and verbal reasoning skillsThe current study found that despite a small sample size, therelationship between verbal reasoning skills and vocabulary was found to besignificantly significant. There was a positive correlation indicating thatchildren who have a larger vocabulary are able to score higher in the LFTassessment. These findings add to the previous literature around verbalreasoning skills and vocabulary (Calvo 2004, Cain 1996, Harrison 2004).
Lepola et.al (2012) report thatvocabulary knowledge has been one of the best predictors of narrative listeningcomprehension and verbalreasoning skills (Snowling & Stevenson, 2004, Ouellette and Rodney2006). This current study shows parallels with the literature aroundchildren who have speech, language and communication needs. Cain et.al. (2004)found that children who had poor vocabulary knowledge had limited verbalreasoning skills.
They found it difficult to infer meanings of novel words usingcontext, and were unable to draw information from the texts. Mitchelland Riggs (2000) found that in order for children to develop strong inferenceskills, there is a needin the increase of vocabulary in particular, mental concepts. When presentingchildren with LFT style tasks/ picture books, it could be important to extractkey vocabulary to support the child’s ability to access their own schematas toincrease their verbal reasoning skills. Within this study the results concluded that therewas not a statistically significant relationship between cognitive skills andverbal reasoning skills. This result does contradict the current research anddue to the sample size should be interpreted with caution. Possible reasons behind the results in the study couldbe around the type of cognition assessed. The Ravens Progressive Matrices(Ravens, 1998) is designed to assess general intelligence and although workingmemory is a skill participant’s use within the assessment, it is notspecifically targeted. Working memory has been widely discussed in theliterature around verbal reasoning and inference skills.
The work of Graesseret.al (1994) and Calvo (2004) emphasise the importance of working memory inorder to keep the mental representation in the mind in order to access theappropriate vocabulary and background knowledge to effectively infer. It shouldalso be noted that some studies have also found that children still effectivelyinfer, despite lower intelligence when around a topic of their interest (Barneset.al. 1996). The current study therefore adds to the mixed literature aroundcognitive skills and verbal reasoning skills. When looking at syntax (comprehension and expressionof grammar) and verbal reasoning skills, the small sample indicated astatistically significant relationship between the different variables.
Therewas a high correlation between receptive grammar and verbal reasoning skills (rs=.796) and use of grammatical structures andverbal reasoning skills (rs=.611). There was not a significantrelationship between sentence formulation and verbal reasoning skills (rs=.399).
This could be interpreted that beingable to comprehend grammatical structures helps to support verbal reasoningskills. Lepola et.al. (2012) study found that ‘listening comprehension at agefour was a predictor of verbal reasoning and inference skills at age 5 and 6’ (p.275)in particular sentence memory was an important feature of listeningcomprehension at that age. This is an important factor when thinking about whatskills are important for the development of verbal reasoning skills. Relationship betweennational curriculum reading comprehension scores and verbal reasoningassessments. Theresults indicated that the relationship between national curriculum readingcomprehension results and verbal reasoning skills was not statisticallysignificant.
Seen in Figure 6, there was a weak correlation between the twoassessments. It was however noted that the participants who scored higher inthe verbal reasoning assessment, scored high on the reading comprehension test.Samplesize may have impacted on the results of the assessments. The results couldalso indicate that there is a threshold a higher performance in readingcomprehension tests, in which children need to have adequate verbal reasoningskills. LM1 Within this current population, no participant scored the ceiling ineither assessment.
This could be due to arguments emphasising that reasoningand inference skills develop between six and eleven years (Paris and Lindauer,1976 and Paris et.al., 1977), cited in Kispal (2008) literature review. Whenlooking at the reading comprehension test, the questions were examining a rangeof skills such vocabulary knowledge, sequencing and explaining information, andmake inferences and predictions from texts. The results were given as a totaland therefore the assessments were not interpreted in terms of which areas eachparticipant scored highly on and may not be sensitive to a participant’s verbalreasoning skills. The presentation of questions between the LFT assessment(Parsons & Branagan, 2005) and reading comprehension differed. The majorityof the students scored in the top end of the assessment in paper one.
Thestudents scored lower in paper two which required an increased demand oninference and prediction skills (see Table 5). Within the reading comprehensiontest booklet paper one (An octopus under my bed) there were five questions outof nine, which were multiple choice. The majority of the multiple choicequestions required inference and prediction skills. Within paper two, there wasan increased demand on using open ended questions when asking inference andprediction questions. The LFT assessment (Parsons & Branagan, 2005) usesopen ended questions throughout the assessment. Shohamy (1984) found that whenassessing reading comprehension skills, multiple choice questions were’consistently easier than open ended questions within different texts'(p.157).
All participants scored higher in the paper one assessment which had moremultiple choice questions to support answers. It maybe important to think about the type of teaching the participants received inorder to complete the reading comprehension tasks. They may have had specificteaching and practised tests around how to answer these type of booklets.Outlined in the literature, the level of teaching and support in enhancing verbalreasoning and inference skills is vital. Lennox (2013) emphasises that there are many concepts that childrencannot discover independently and therefore the adult as the mediator plays akey role in supporting new understandings. Background knowledge ofdifferent assessments Within the study, the results highlighted thatthere could be a statistically significant difference between the participant’sscore between the scenarios based on their background knowledge. Althoughassessments were administered on a small sample, the results do link in withthe current literature around background knowledge and verbal reasoning skills.
Lennox (2013) emphasises that “background knowledge and reasoning skills areneeded to predict, hypothesis, explain, imagine, infer, problem solve andevaluate.” (p.386) Within the Level-A questions, the differencebetween scenarios was small. In order to answer the questions accurately thestudents are required to use the information in front of them. The differencebetween the scenarios were larger within LFT Level-C questions. These questionsrequire higher-level thinking skills in which participants are required to drawon their background knowledge. For example, a participant responded to theLevel-C questions ‘Which is better, TV or cinema/puppet show?’ differently. Inthe cinema question they scored 3/3 responding, “movies because you get towatch it bigger and dark”.
Within the puppet show the participant scored 1/3responding, “TV because puppet shows might be quicker.” The answer in thecinema scenario was specific in which the participant drew on their experiences.The participant’s answer in the puppet show demonstrated some understandinghowever the question was not answered as accurately due to lack of backgroundknowledge.
Pressley & Afferbach (1995)report that a reader has to have experiences in order for higher-level thinkingto develop; “the richer a children’s word experiences are, the richer thechild’s schematic knowledge is that they are able to draw on.” (p.54). There was four instancesin which the participants scored higher in the puppet scenario than the cinemascenario although did not affect overall levels. Other arguments have discussedthat it is not only about background knowledge but also to do with being ableto access it in order to answer inference questions accurately (Cain &Oakhill 1998).
Therelationship between raters of the LFT assessment Inter-rater reliabilityis important in order to support the quality of assessments and increase theusefulness within the scoring process (Morgan et.al. 2014). There was two typesof measures when looking at the inter-rater reliability within this study; correlation and ICC.
Thepresent literature (Rankin & Stokes 1998, Morgan et.al 2014 and Solarovaet.al 2014) emphasise the importance of dual measurements of reliability inorder to look at the strength of linear association and agreement. From theassessment data it can be found there is a strong inter-rater reliabilitybetween the three therapists. In Figure 12 there is a normal distribution ofscores for rater three. This rater is a co-author of the LFT programme. As theauthor, co-wrote the scoring criteria, the normal distribution of scores may bedue to truly understanding the rationale behind the scoring criteria.
Theresults however did show that nevertheless two therapists who are currently practicingin the field correlated scores with the author. It should be noted however thatall three raters were experienced Speech and Language Therapists and hadexperience in clinically administering and scoring this assessment, even when aparticipant’s answer was not presented in the scoring criteria. The highcorrelations could be seen as an indication that the scoring criteria withinthe LFT assessment is generally useful in being able to assess a child’s verbalreasoning skills. The current study does not however look at whether there isan agreement between an experienced therapist and a professional who has limitedexperience in scoring this assessment. It should be notedhowever that there were 7/12 participants in which the level on the LFT wouldhave differed, impacting on the recommendation of whether the child would requireadditional specific support.
This difference may have an impact on outcomes forstudents with Speech, Language and Communication needs. It was seen in theresults, that there was wider distribution of scores when the raters scored Level-Cquestions. As seen earlier, these questions require the ability to ‘predict,reflect on and integrate ideas and relationships’ (Blank et.al 1978a ). Perhaps, due to thevariability of these answers in this level, the current scoring criteria doesnot aid a therapist in being able to effectively score the results. Solarovaet.al (2014) highlight that the more items a scale comprising a raw score has,the more difficulties there is in order to reach a complete agreement ofscorers.
It may therefore be important to think about making the scoring more concreteand increasing the examples within the current assessment to allow moreagreement amongst raters. Limitations of thestudy:Thisstudy was limited by a small group size, which is a risk to the reliability ofthe results. This was particularly evident in comparison of language andcognitive assessments. Button et.
al (2013) report that low statistical powerwhich is due to low sample size can negatively affect that a statisticallysignificant result is a true effect. Thesmall group size also had a limited range of ages of participants. Due to thetiming in the year, there was one pupil aged 6;0 years within year two. Futureresearch needs to be conducted on a larger sample across the spread of the yearin order to look at the extent of which age is a factor within the developmentof verbal reasoning skills and whether the statistically significant resultsare able to be replicated.Otherfactors such as gender and English as an additional language were notinvestigated within the current study. Again due to small sample size, thecurrent data was unable to be analysed in terms of whether there were differencesamongst gender and whether bilingualism has an effect on verbal reasoningskills. Due tothe sample size, there was one outlier in the data included in the results.
This participant was lower than average within all assessments however due to asmall sample size was included. This participant may have had an impact on theoverall data and therefore the results may not be as reliable. If this researchwas to be replicated, a larger sample would allow for outliers to be excluded.Itshould also be noted that only one school was recruited in a borough of Londonthat was ranked in the top quarter in poverty rankings (trust for London REF). Waldfogeland Washbrook (2012) report that in the UK, there is a 20 month gap invocabulary at school entry between the wealthiest and poorest. This couldpotentially not be representative of the typical population.
It would therefore be important to conduct further research in differentgeographical areas in order to look at any additional factors such associo-economic status.Theschool is situated within a multi-lingual population. From the current data, 6/12of the participants spoke English as an additional language. Although the questionnaireidentified what languages were spoken, the researcher did not look at how longthe child had been exposed to English. Cummins (1984b) examined the relationshipbetween fluency and everyday conversation and the ability to use language foracademic purposes. He found that children who start learning a second languageafter school admission may acquire a good level of fluency in everydayconversation but may take between five and seven years before they have caughtup with the average monolingual children on measures of academic achievement. Itshould also be noted that the language assessments used are not standardised onthe bilingual population. Theschool provided the researcher with the quantitative scores of the nationalcurriculum tests.
As a result, the researcher was unable to look qualitativelyat the way the participants had answered targeted questions in the booklets inorder to look at whether there are comparisons to be drawn between thestudent’s ability to draw information from the text was due to verbal reasoningskills. Clinical ImplicationsThese findings may have relevant implications for professionals insupporting a child’s verbal reasoning skills using the LFT programme (Parsons& Branagan, 2005). Firstly it may be important to select a scenario thatthe child has experience of. The professional might also want to take intoaccount cultural experiences with regard to a child’s score.
Torr & Scott(2006) emphasise the importance of ‘relatingthe language to the child’s personal situation’ (p.161). The data coincides with the current research, around pre-teaching of vocabularyto support verbal reasoning skills. When implementing the LFT programme(Parsons & Branagan) in intervention, professionals may want to draw outthe vocabulary from the picture presented to the child if they are using’picture and talk/ text’ styles. Cain et.al (2007) further emphasise this pointparticularly when attempting to increase an understanding within picture books.
These findings further support previous research (REF) around teachers creatingopportunities to talk around books, and discuss inferences within the text froma whole class approach at an earlier age. Taggart et.al (2005) highlight that’story time can be an opportunity to develop children’s thinking’ (p.ix). Itmay be important to continue to support children in key stage one in makingmore inferences, when there are no other demands involved e.g. reading the text.This may scaffold a child’s ability to perform higher in their readingcomprehension tests.
The participants’ scores in the LFT (Parsons & Branagan, 2005) assessmentwere largely different to the examples in the book (see appendix item X). The authors of thebook may want to revisit the scoring criteria in order to provide more detailedscoring guidelines, ensuring modern examples are included. Future researchdirections:- A future study with similarassessments with a larger sample size across different schools in different geographicareas. – For future data to be replicated and exploredin terms of gender, socio-economic class and English as an additional language.
– To further look at the relationshipbetween different cognitive skills (e.g. working memory) and verbal reasoningskills.- To further look at agreement amongst professionalswho do not have experience with the assessment in order to look at whetherthere is still a strong agreement when using the scoring criteria. CONCLUSIONSThefindings from the present study indicate that vocabulary, and syntax (receptiveand expressive grammar) may correlate with verbal reasoning skills. There was alimited correlation between cognitive skills and verbal reasoning skills. Therewas not a significant relationship between national curriculum readingcomprehension result and verbal reasoning skills, however participants whoscored higher in the verbal reasoning assessment, scored high on the readingcomprehension test. Whenlooking at the properties of the LFT assessment, there were differences amongstassessment scenarios based on a child’s experience.
This may be an importantfactor for educational professionals when considering using the assessment withindividuals, particularly individuals with English as an additional language withcultural experiences. The assessment currently does have an agreement amongstdifferent raters, however the scores between the Speech and Language Therapistsand the author of the programme varied largely when marking higher-level reasoningquestions. A stricter criteria would be beneficial for accurate scoring of theassessment, particularly for professionals who have limited experience with theprogramme.
The LFT assessment however is a quick, easy to administer assessmentthat would benefit from further analysis of its psychometric properties inorder to support standardisation as a diagnostic tool.