Validation is a process for measuring the accuracy, reliability, and consistency of a testing instrument. At PeopleKeys, we never launch a product without first gathering practical and theoretical research to validate that it works as advertised. We have conducted countless internal studies, and have worked closely with independent agencies to run specialized reliability and validation tests to confirm the results of our research. Below is one of the studies related to DISC Validation.

Texas State University – DISC Personality System Validation Study By Larry Price, PhD, Professor of Psychometrics and Statistics

Study overview

The purpose of this study was to examine the psychometric properties of IML’s DISC Personality System Analysis, specifically within the context of the workplace environment. Cohen and Swerdlik (2005) define personality assessment as ‘the measurement and evaluation of psychological traits, states, values, interests, attitudes, worldview, acculturation, personal identity, sense of humor, cognitive behavior styles, and related individual characteristics.’ This offers the notion that personality is several dimensions. Hence, this study will use both classical psychometric analytic methods and newer approaches that are suitable for multidimensional constructs.

Item development in relation to content validity

Items were developed based on the original theoretical work of William Marston, MD as published in the Emotions of Normal People (1928). The items on the IML DISC were developed to tap four latent dimensions of personality as espoused by Marston. Item content and wording was refined through an iterative procedure to allow for the maximum potential for tapping a particular construct or dimension. A balance was achieved between the number of items on the final scale, as well as the amount and degree of redundancy the items exhibited. During the item development and refinement process, the IML development team displayed evidence of adhering to the Standards for Educational and Psychological Testing published by the American Educational Research Association (AERA), American Psychological Association (APA), and the National Council on Measurement in Education (NCME) (AERA, APA & NCME, 1999).

Forced-choice item format

The forced-choice rating technique is used on the IML DISC where the subject responds to four sets of words or phrases and is required to select one of the four sets that MOST describes his/her behavior in the work environment and one of the four sets that LEAST describes his/her behavior. Highland & Berkshire (1951) conducted an extensive study of forced-choice rating forms in connection with rating instructors. Their findings were that:

• Highest reliability coefficients were attained when item formats were composed of four words or statements, two being favorable and two unfavorable; the rater selecting the most and the least descriptive.
• Evidence for the highest criterion validity was obtained when the item formats were composed of four words or statements.
• When item formats were composed of four words or statements, all favorable, the rater revealed the smallest evidence of response bias.

These findings provide psychometric support for the approach used in the IML DISC. The participant’s measurement is ‘ipsative,’ meaning that the subject’s responses and presumed strength of a measured trait are relative to the measured strengths of other traits in the same person. Hence, there is little normative or reference data in this manual for comparisons with others. Users of this type of scale have to be mindful of the underlying theoretical nature of the ipsative measurement and scoring approach so as to use the scores or classification information in an appropriate manner.

Sample demographics

The primary goal of this study was to investigate the psychometric aspects of the IML DISC within the context of the workplace environment. Therefore, a purposive sampling strategy was employed for the data collection process. The sample obtained was cross-sectional, representing seven different regions of the United States in 26 separate locations. Specifically represented in the sample are the Northeast, Midwest, South, and Southwest regions of the United States. The breakdown of the sample by ethnicity and gender is shown in Tables 1 and 2. Participants in the sample ranged from age 21 to 85 and had attained a minimum of 12 years of formal education with most having 16 years. 331 survey instruments were completed and returned for analysis.

Score and scale reliability

Reliability refers to the consistency and stability of scores obtained on a measurement instrument. These are represented by two measures:

• The examinee’s true score on the underlying psychological construct and
• Errors of measurement that represent the inaccuracy of measurement on a particular testing occasion or over time in a test–retest situation.

Internal consistency

Internal consistency estimates of reliability represent how inter-related items are and how much they measure the same construct. Since the DISC has 4 different dimensions, intercorrelations are provided for each scale (see Table 3). The Cronbach alpha is a measure of the internal consistency of items. A high Cronbach alpha (0.70–0.85) indicates that the items are all measuring one similar ability and are therefore, highly intercorrelated. As seen in Table 3, the internal consistency of the DISC scales is very high, with a measure showing that just 14–24% of the variation in scores for each scale is due to error. Correlations between the DISC scales were also done to provide information about how distinct the scales are or how much they are related to each other. Given that they are meant to be measuring different, distinct scales, we do not expect the scales to be highly inter-correlated. Coefficients below 0.35 reflect a ‘low’ degree of association; those between 0.36 and 0.65 reflect a ‘moderate’ degree of association and those above 0.65 reflect a ‘high’ degree of association. The ‘+’ sign shows that it is a positive correlation, that is, as the value on one variable increases, the value on the other increases as well. The ‘–’ sign shows a negative correlation in which value on one variable decreases as the other increases. Based on the results in Table 4, the following conclusions can be made:

• Dominance has a moderate, negative relationship with Compliance. • Dominance exhibits a strong, negative relationship with Steadiness.
• Influence has a strong, negative relationship with Compliance.

In other words, individuals who are high in Dominance or Influence tend to score lower on Compliance. This provides support for the personality types described, since individuals who are more dominant, direct, and driven or influential, outgoing, and flexible tend not to be highly controlled, compliant, or perfectionistic.

Validity

Validity refers to the degree to which tests measure what they claim to measure. In the case of IML’s DISC, it is also linked to the theoretical framework proposed by William Marston.

Content validity

Content validity evidence refers to themes, words, item formats, tasks, and guidelines for administration and scoring. Evidence for content validity was established by qualified experts at IML by compiling a series of statements and traits that are universally accepted as the basis for the four DISC styles. The descriptors and statements were also compared with other leading DISC publications of modern times to establish universal evidence of agreement for the ‘core’ observable behaviors or traits.

Construct validity

This is a measure of how the subscales and items conform to the theoretical construct on which the proposed test score interpretations are based. Firstly, the DISC is multidimensional in nature and composed of four subscales representing a separate but correlated dimension of personality style. Hence, we want to investigate the factor structure of the IML DISC instrument in relation to Marston’s theoretical framework using the Q – Technique. This allows us to check how many factors are being measured by the items in the test and if all the items relate to one or more of the four DISC subscales. The Q – Technique factor analysis method is particularly appropriate for the study of classifying individuals based on preferences or systems of typology – such as in personality assessment. Typically, Q – Technique studies employ a small number of subjects because we are more interested in the notion of ‘intra-individual differences’ (within subjects) rather than ‘inter-individual’ (between subjects) differences. As expected, analyses provided evidence of a four-factor dimensional structure underlying the IML DISC instrument. Across all analyses, an average of 70% of the total variance was explained by the four factor DISC model. As a general rule, the guidelines for the percentage of variance accounted for in a factor analytic model to be evaluated as ‘acceptable’ in the psychometric literature is 70%. Figure 2 – The IML DISC Multidimensional Four-Factor Oblique Structural Equation Mode

Model estimation

Data on the four-factor structure of the DISC instrument were assessed using confirmatory factor analysis. After screening the data for multiple dimensions and finding this assumption tenable, the method of maximum likelihood was used to derive parameter estimates with all subsequent analyses performed on the covariance matrices. In order for a likely model to be identified, scales on each latent variable were established.

Assessment of measurement models

Several fit indices were used as indicators for the goodness-of-fit of the DISC structural equation model. Fit indices used were: (a) the overall chi-square statistic; (b) the root mean square residual (RMR), a measure of error; (c) the Akike Information Criterion (AIC); (d) the root mean square error of approximation (RMSEA; Steiger, 1990); and (e) the goodness of fit index (GFI). For the four-factor DISC model evaluated herein, the following fit statistics were observed: (a) χ2 (244, n=200)=272.65, p > .05. The χ2 value shows that the data fits with the model. (b) RMR = .08. This value shows that there is an acceptably small error between the data and the implied model. (c) AIC = 348.67. This value shows that the data fits with the model. (d) RMSEA = .02. The RMSEA provides values that represent the goodness-of-fit of the model if it were estimated in the population. RMSEA values between .05 and .08 are viewed as acceptable with values closer to zero indicating a more closely approximated model fit in relation to the population. (e) GFI=.90. The GFI have values ranging from 0 to 1with values above .90 indicating a good fit of the empirical data to the implied model. All the goodness-of-fit statistics show that the data fits with the postulated DISC model.

Multidimensional scaling

Multidimensional scaling (MDS) is a multivariate statistical technique that helps researchers to identify key dimensions underlying individuals’ responses. To accomplish this, MDS offers a scaling approach that allows for inferences about the underlying dimensions (in geometric space such as was posited in Marston’s original work) from a series of similarity or preference judgments or statements. The results of MDS provide a map representing the respondent’s perceived personality style. Figure 3 below provides the MDS perceptual map obtained from this study. The perceptual map demonstrates that the four DISC dimensions are relatively removed from each other, and therefore, fairly distinct from one another, as posited in Marston’s original theory. Figure 3 Perceptual Map of DISC Dimensions Specific problems associated with personality assessment inventories There are a number of common problems associated with personality assessment which can affect the validity of individual profiles. However, this can often be minimized through professional feedback, interpretation, and consultation, including a discussion of the context within which the assessment occurs. Problems include the following:

• False responding, due to uncooperativeness or wanting to project a favorable image.
• Situational specificity, in which individuals tend to respond based on memories of most recent experiences or thoughts about how they would act in a specific situation rather than across a range of situations.

False responding IML’s standard DISC administration procedure uses a timed approach, thereby reducing the propensity for participants to submit false responses on items or respond in a socially desirable manner. Some items are also framed negatively, which checks for the possibility of individuals intentionally skewing their results or responding in a particular way (e.g., checks 1 for the first answer in every item). Situational specificity While a person’s general characteristics remain the same, human behavior fluctuates within certain boundaries, depending on the situation. Hence, when completing the DISC profile, you are asked to answer within the context of a certain environment (e.g., work or home). This is essential for a valid interpretation and understanding ofscores. Conclusion This study was designed to investigate the psychometric properties of the IML DISC instrument to validate its usefulness in a variety of situations. Collectively, the results prove that the IML DISC is a robust measure of personality style and can be used in schools or workplace environments to help individuals better understand their personality style. If you want to gain more HR knowledge and information, please LIKE our Facebook & Linkedin! Your support means a lot to us. Further Reading : How to successfully develop leadership and management capabilities? Human Resource Optimization – a New Paradigm for Developing Talent 3 Simple Questions that Blow Your Mind – The Unconscious Bias