SciCombinator

Discover the most talked about and latest scientific content & concepts.

Concept: Validity

177

A common approach for determining musical competence is to rely on information about individuals' extent of musical training, but relying on musicianship status fails to identify musically untrained individuals with musical skill, as well as those who, despite extensive musical training, may not be as skilled. To counteract this limitation, we developed a new test battery (Profile of Music Perception Skills; PROMS) that measures perceptual musical skills across multiple domains: tonal (melody, pitch), qualitative (timbre, tuning), temporal (rhythm, rhythm-to-melody, accent, tempo), and dynamic (loudness). The PROMS has satisfactory psychometric properties for the composite score (internal consistency and test-retest r>.85) and fair to good coefficients for the individual subtests (.56 to.85). Convergent validity was established with the relevant dimensions of Gordon’s Advanced Measures of Music Audiation and Musical Aptitude Profile (melody, rhythm, tempo), the Musical Ear Test (rhythm), and sample instrumental sounds (timbre). Criterion validity was evidenced by consistently sizeable and significant relationships between test performance and external musical proficiency indicators in all three studies (.38 to.62, p<.05 to p<.01). An absence of correlations between test scores and a nonmusical auditory discrimination task supports the battery's discriminant validity (-.05, ns). The interrelationships among the various subtests could be accounted for by two higher order factors, sequential and sensory music processing. A brief version of the full PROMS is introduced as a time-efficient approximation of the full version of the battery.

Concepts: Psychometrics, Skill, Validity, Reliability, Sound, Music, Test

168

Abstract Purpose: To develop a patient reported outcome measure of active and passive function in the hemiparetic upper limb. Methods: Potential items for inclusion were identified through (a) systematic review and analysis of existing measures and (b) analysis of the primary goals for treatment in a spasticity service. Item reduction was achieved through consultation with a small, purposively selected multi-disciplinary group of experienced rehabilitation professionals (n = 10) in a three-round Delphi process. This was followed by a confirmatory survey with a larger group of clinicians (n = 36) and patients and carers (n = 13 pairs). Results: From an initial shortlist of 75 items, 23 items were initially identified for inclusion in the arm activity measure (ArmA), and subsequently refined to a 20-item instrument comprising 7 passive and 13 active function. In common with the six measures identified in the systematic review, a five-point ordinal scaling structure was chosen, with ratings based on activity over the preceding 7 days. Conclusions: The ArmA is designed to measure passive and active function following focal interventions for the hemiparetic upper limb. Content and face validity have initially been addressed within the development process. The next phase of development has involved formal evaluation of psychometric properties. Implications for Rehabilitation In clinical practice or research, outcome measures in rehabilitation need to have face and content validity. Following stroke or brain injury, goals for rehabilitation of the hemiparetic upper limb may be: to restore active function, if there is return of motor control or to improve passive function making it easier to care for the limb (e.g. maintain hygiene) if no motor return is possible, measurement of both constructs should be considered. This study describes the systematic development of the ArmA, a measure of active and passive function in the hemiparetic upper limb.

Concepts: Traumatic brain injury, Psychometrics, Validity, Upper limb, Motor skill, Content validity, Face validity, Armas

121

The list of motives by Kanin (1994) is the most cited list of motives to file a false allegation of rape. Kanin posited that complainants file a false allegation out of revenge, to produce an alibi or to get sympathy. A new list of motives is proposed in which gain is the predominant factor. In the proposed list, complainants file a false allegation out of material gain, emotional gain, or a disturbed mental state. The list can be subdivided into eight different categories: material gain, alibi, revenge, sympathy, attention, a disturbed mental state, relabeling, or regret. To test the validity of the list, a sample of 57 proven false allegations were studied at and provided by the National Unit of the Dutch National Police (NU). The complete files were studied to ensure correct classification by the NU and to identify the motives of the complainants. The results support the overall validity of the list. Complainants were primarily motivated by emotional gain. Most false allegations were used to cover up other behavior such as adultery or skipping school. Some complainants, however, reported more than one motive. A large proportion, 20% of complainants, said that they did not know why they filed a false allegation. The results confirm the complexity of motivations for filing false allegations and the difficulties associated with archival studies. In conclusion, the list of Kanin is, based on the current results, valid but insufficient to explain all the different motives of complainants to file a false allegation.

Concepts: Psychology, Validity, Criminal law, Motivation, Pleading, Legal terms, Computer file

54

Background The consensus definition of severe sepsis requires suspected or proven infection, organ failure, and signs that meet two or more criteria for the systemic inflammatory response syndrome (SIRS). We aimed to test the sensitivity, face validity, and construct validity of this approach. Methods We studied data from patients from 172 intensive care units in Australia and New Zealand from 2000 through 2013. We identified patients with infection and organ failure and categorized them according to whether they had signs meeting two or more SIRS criteria (SIRS-positive severe sepsis) or less than two SIRS criteria (SIRS-negative severe sepsis). We compared their characteristics and outcomes and assessed them for the presence of a step increase in the risk of death at a threshold of two SIRS criteria. Results Of 1,171,797 patients, a total of 109,663 had infection and organ failure. Among these, 96,385 patients (87.9%) had SIRS-positive severe sepsis and 13,278 (12.1%) had SIRS-negative severe sepsis. Over a period of 14 years, these groups had similar characteristics and changes in mortality (SIRS-positive group: from 36.1% [829 of 2296 patients] to 18.3% [2037 of 11,119], P<0.001; SIRS-negative group: from 27.7% [100 of 361] to 9.3% [122 of 1315], P<0.001). Moreover, this pattern remained similar after adjustment for baseline characteristics (odds ratio in the SIRS-positive group, 0.96; 95% confidence interval [CI], 0.96 to 0.97; odds ratio in the SIRS-negative group, 0.96; 95% CI, 0.94 to 0.98; P=0.12 for between-group difference). In the adjusted analysis, mortality increased linearly with each additional SIRS criterion (odds ratio for each additional criterion, 1.13; 95% CI, 1.11 to 1.15; P<0.001) without any transitional increase in risk at a threshold of two SIRS criteria. Conclusions The need for two or more SIRS criteria to define severe sepsis excluded one in eight otherwise similar patients with infection, organ failure, and substantial mortality and failed to define a transition point in the risk of death. (Funded by the Australian and New Zealand Intensive Care Research Centre.).

Concepts: Inflammation, Systemic inflammatory response syndrome, Intensive care medicine, Acute respiratory distress syndrome, Validity, Multiple organ dysfunction syndrome, Septic shock, Sepsis

48

Conspiracy theories are ubiquitous when it comes to explaining political events and societal phenomena. Individuals differ not only in the degree to which they believe in specific conspiracy theories, but also in their general susceptibility to explanations based on such theories, that is, their conspiracy mentality. We present the Conspiracy Mentality Questionnaire (CMQ), an instrument designed to efficiently assess differences in the generic tendency to engage in conspiracist ideation within and across cultures. The CMQ is available in English, German, and Turkish. In four studies, we examined the CMQ’s factorial structure, reliability, measurement equivalence across cultures, and its convergent, discriminant, and predictive validity. Analyses based on a cross-cultural sample (Study 1a; N = 7,766) supported the conceptualization of conspiracy mentality as a one-dimensional construct across the three language versions of the CMQ that is stable across time (Study 1b; N = 141). Multi-group confirmatory factor analysis demonstrated cross-cultural measurement equivalence of the CMQ items. The instrument could therefore be used to examine differences in conspiracy mentality between European, North American, and Middle Eastern cultures. In Studies 2-4 (total N = 476), we report (re-)analyses of three datasets demonstrating the validity of the CMQ in student and working population samples in the UK and Germany. First, attesting to its convergent validity, the CMQ was highly correlated with another measure of generic conspiracy belief. Second, the CMQ showed patterns of meaningful associations with personality measures (e.g., Big Five dimensions, schizotypy), other generalized political attitudes (e.g., social dominance orientation and right-wing authoritarianism), and further individual differences (e.g., paranormal belief, lack of socio-political control). Finally, the CMQ predicted beliefs in specific conspiracy theories over and above other individual difference measures.

Concepts: Scientific method, Measurement, Sociology, Psychometrics, Factor analysis, Validity, Confirmatory factor analysis, Conspiracy theory

46

We hypothesized that individuals may differ in the dispositional tendency to have positive vs. negative attitudes, a trait termed the dispositional attitude. Across 4 studies, we developed a 16-item Dispositional Attitude Measure (DAM) and investigated its internal consistency, test-retest reliability, factor structure, convergent validity, discriminant validity, and predictive validity. DAM scores were (a) positively correlated with positive affect traits, curiosity-related traits, and individual preexisting attitudes; (b) negatively correlated with negative affect traits; and © uncorrelated with theoretically unrelated traits. Dispositional attitudes also significantly predicted the valence of novel attitudes while controlling for theoretically relevant traits (such as the Big 5 and optimism). The dispositional attitude construct represents a new perspective in which attitudes are not simply a function of the properties of the stimuli under consideration, but are also a function of the properties of the evaluator. We discuss the intriguing implications of dispositional attitudes for many areas of research, including attitude formation, persuasion, and behavior prediction. (PsycINFO Database Record © 2013 APA, all rights reserved).

Concepts: Scientific method, Prediction, Futurology, Prophecy, Psychometrics, Validity, Reliability, All rights reserved

31

Working dog organisations, such as Guide Dogs, need to regularly assess the behaviour of the dogs they train. In this study we developed a questionnaire-style behaviour assessment completed by training supervisors of juvenile guide dogs aged 5, 8 and 12 months old (n = 1,401), and evaluated aspects of its reliability and validity. Specifically, internal reliability, temporal consistency, construct validity, predictive criterion validity (comparing against later training outcome) and concurrent criterion validity (comparing against a standardised behaviour test) were evaluated. Thirty-nine questions were sourced either from previously published literature or created to meet requirements identified via Guide Dogs staff surveys and staff feedback. Internal reliability analyses revealed seven reliable and interpretable trait scales named according to the questions within them as: Adaptability; Body Sensitivity; Distractibility; Excitability; General Anxiety; Trainability and Stair Anxiety. Intra-individual temporal consistency of the scale scores between 5-8, 8-12 and 5-12 months was high. All scales excepting Body Sensitivity showed some degree of concurrent criterion validity. Predictive criterion validity was supported for all seven scales, since associations were found with training outcome, at at-least one age. Thresholds of z-scores on the scales were identified that were able to distinguish later training outcome by identifying 8.4% of all dogs withdrawn for behaviour and 8.5% of all qualified dogs, with 84% and 85% specificity. The questionnaire assessment was reliable and could detect traits that are consistent within individuals over time, despite juvenile dogs undergoing development during the study period. By applying thresholds to scores produced from the questionnaire this assessment could prove to be a highly valuable decision-making tool for Guide Dogs. This is the first questionnaire-style assessment of juvenile dogs that has shown value in predicting the training outcome of individual working dogs.

Concepts: Scientific method, Psychometrics, Validity, Test validity, Criterion validity, Construct validity, Dog, Test

30

Wisdom is a field of growing interest both inside and outside academic psychology, and researchers are increasingly interested in using measures of wisdom in their work. However, wisdom is a highly complex construct, and its various operationalizations are based on quite different definitions. Which measure a researcher chooses for a particular research project may have a strong influence on the results. This study compares four well-established measures of wisdom-the Self-Assessed Wisdom Scale (Webster, 2003, 2007), the Three-Dimensional Wisdom Scale (Ardelt, 2003), the Adult Self-Transcendence Inventory (Levenson et al., 2005), and the Berlin Wisdom Paradigm (Baltes and Smith, 1990; Baltes and Staudinger, 2000)-with respect to content, reliability, factorial structure, and construct validity (relationships to wisdom nomination, interview-based wisdom ratings, and correlates of wisdom). The sample consisted of 47 wisdom nominees and 123 control participants. While none of the measures performed “better” than the others by absolute standards, recommendations are given for researchers to select the most suitable measure for their substantive interests. In addition, a “Brief Wisdom Screening Scale” is introduced that contains those 20 items from the three self-report scales that were most highly correlated with the common factor across the scales.

Concepts: Scientific method, Psychology, Research, Psychometrics, Validity, Construct validity, Prime number

28

OBJECTIVE Recent advances in optics and miniaturization have enabled the development of a growing number of minimally invasive procedures, yet innovative training methods for the use of these techniques remain lacking. Conventional teaching models, including cadavers and physical trainers as well as virtual reality platforms, are often expensive and ineffective. Newly developed 3D printing technologies can recreate patient-specific anatomy, but the stiffness of the materials limits fidelity to real-life surgical situations. Hollywood special effects techniques can create ultrarealistic features, including lifelike tactile properties, to enhance accuracy and effectiveness of the surgical models. The authors created a highly realistic model of a pediatric patient with hydrocephalus via a unique combination of 3D printing and special effects techniques and validated the use of this model in training neurosurgery fellows and residents to perform endoscopic third ventriculostomy (ETV), an effective minimally invasive method increasingly used in treating hydrocephalus. METHODS A full-scale reproduction of the head of a 14-year-old adolescent patient with hydrocephalus, including external physical details and internal neuroanatomy, was developed via a unique collaboration of neurosurgeons, simulation engineers, and a group of special effects experts. The model contains “plug-and-play” replaceable components for repetitive practice. The appearance of the training model (face validity) and the reproducibility of the ETV training procedure (content validity) were assessed by neurosurgery fellows and residents of different experience levels based on a 14-item Likert-like questionnaire. The usefulness of the training model for evaluating the performance of the trainees at different levels of experience (construct validity) was measured by blinded observers using the Objective Structured Assessment of Technical Skills (OSATS) scale for the performance of ETV. RESULTS A combination of 3D printing technology and casting processes led to the creation of realistic surgical models that include high-fidelity reproductions of the anatomical features of hydrocephalus and allow for the performance of ETV for training purposes. The models reproduced the pulsations of the basilar artery, ventricles, and cerebrospinal fluid (CSF), thus simulating the experience of performing ETV on an actual patient. The results of the 14-item questionnaire showed limited variability among participants' scores, and the neurosurgery fellows and residents gave the models consistently high ratings for face and content validity. The mean score for the content validity questions (4.88) was higher than the mean score for face validity (4.69) (p = 0.03). On construct validity scores, the blinded observers rated performance of fellows significantly higher than that of residents, indicating that the model provided a means to distinguish between novice and expert surgical skills. CONCLUSIONS A plug-and-play lifelike ETV training model was developed through a combination of 3D printing and special effects techniques, providing both anatomical and haptic accuracy. Such simulators offer opportunities to accelerate the development of expertise with respect to new and novel procedures as well as iterate new surgical approaches and innovations, thus allowing novice neurosurgeons to gain valuable experience in surgical techniques without exposing patients to risk of harm.

Concepts: Surgery, Psychometrics, Validity, Cerebrospinal fluid, Neurosurgery, Hydrocephalus, Content validity, Special effects

28

The importance of compassion is widely recognized and it is receiving increasing research attention. Yet, there is lack of consensus on definition and a paucity of psychometrically robust measures of this construct. Without an agreed definition and adequate measures, we cannot study compassion, measure compassion or evaluate whether interventions designed to enhance compassion are effective. In response, this paper proposes a definition of compassion and offers a systematic review of self- and observer-rated measures. Following consolidation of existing definitions, we propose that compassion consists of five elements: recognizing suffering, understanding the universality of human suffering, feeling for the person suffering, tolerating uncomfortable feelings, and motivation to act/acting to alleviate suffering. Three databases were searched (Web of Science, PsycInfo, and Medline) and nine measures included and rated for quality. Quality ratings ranged from 2 to 7 out of 14 with low ratings due to poor internal consistency for subscales, insufficient evidence for factor structure and/or failure to examine floor/ceiling effects, test-retest reliability, and discriminant validity. We call our five-element definition, and if supported, the development of a measure of compassion based on this operational definition, and which demonstrates adequate psychometric properties.

Concepts: Definition, Psychometrics, Validity, Reliability, Cronbach's alpha, Suffering, Test-retest, Comparison of assessments