Validation of the iWAM instrument

The Background, reliability and validity of the iWAM instrument


Patrick Merlevede

History and Development of the Tool

The iWAM tool was developed by Patrick Merlevede, a Belgian cognitive scientist and management consultant. Merlevede, the CEO of JobEQ, Belgium, is the author of a number of popular books on emotional intelligence, mentoring and business consulting, including `7 steps to emotional intelligence’.

Statistical information on the construction of the iWAM Questionnaire

The forty questions in the iWAM compare to two-hundred test items in a conventional questionnaire. The power of iWAM comes from the principle that the respondent five statements for each question instead of responding to each statement independently as in a "classic" test. As a result, iWAM collects far more information than is generally obtained from an instrument containing only forty items.

 

Standard Groups

The iWAM tool uses the principle of standard groups in order to give a relative indication of where an individual scores in comparison to others. A standard group indicates how a population will typically score for a pattern. For instance, if we take the pattern `initiation’, the question will be how the population’s scores will be distributed. Once we know this, we can then comment on whether an individual scores high or low on that pattern.

The standard groups are produced by identifying and testing a representative sub sample of iWAM test takers who are representative of the country’s professional mix, in terms of gender, age, role types and ethnicity. The Australian standard group is currently being updated and will include approx. 2,300 iWAM test takers from a wide range of occupational groups.

Organisation of the iWAM Instrument

The Questionnaire is comprised of forty eight metaprogram or thinking styles which are clustered within sixteen pattern categories

iWAM as a contextual instrument

iWAM is not a personality measure, where measures remain relatively stable over time. The iWAM instrument produces a snapshot in time of the test taker’s thinking styles and this will be reflective of their motivations to work at the time of completing the Questionnaire.

The iWAM is designed as a context specific instrument, where the measures are analysed within the work context and the results are relevant to that context only. The iWAM methodology presupposes that people change over time, between contexts and after significant interventions or life experiences. The instrument is designed to detect these subtle changes in pre and post intervention measurements.

Independence of measures

The Myers Briggs Type Indicator (MBTI) which uses the simple metaprograms presents metaprogram categories as a continuum (Briggs Myers, I, McCauley, M, Quenk, M, Hammer, A, 1998). Here, an individual’s score will be located at a point along a continuum between two polar opposite concepts. For example, a person may be seen to be either introverted or extraverted and be situated somewhere along that continuum.

Within the iWAM instrument, all metaprogram measures, including two opposite measures, are structured as independent measures, rather than two points at either end of a continuum (Merlevede, 2005). For example, the first pattern category in the iWAM is Action Level. This two pattern set describes (OF1+) Initiation or a person’s motivation to act and be proactive. The second pattern in this category is (OF1-) is Reflecting and Patience, which measures a person’s motivation to think things through, reflect and be patient before taking action. In the iWAM design an individual may be low on both patterns, high on both patterns or somewhere in between, rather than at one specific a point along a continuum between measures.

Validation of iWAM

Further studies into the validity of the iWAM instrument were carried out in Belgium during 2005. Here a random sampling of the feedback of six hundred and seventeen persons, who completed the iWAM Questionnaire, was conducted by the developer, Patrick Merlevede, with respondents from fourteen countries. Respondents were asked to rate their agreement with the iWAM test results for each of the forty eight individual iWAM patterns choosing Yes=Agree, No= Do not agree or Unsure. The survey revealed an average rating of 89.33% agreement for users commenting on their perceived accuracy of their iWAM report.

Another question in this survey asked respondents to indicate the number of iWAM patterns with which they disagreed. The data indicated that on average test takers disagreed with up to 10% of the ten categories. Further analysis revealed that respondents agreed in 85.27% of the cases and were unsure in the remaining 4.55%. This represents high end user validation of the accuracy of the iWAM reports.

At the University of Newcastle, NSW, the iWAM tool has been used as the primary instrument in Doctoral research with medical students in the School of Medicine. The study has revealed significant measurable characteristics of competent and satisfied medical students.

Test- re-test reliability

The reliability of the iWAM tool was initially tested in 2000 by administering an iWAM and using interviews as the re-test method. A benefit of this process was that the interviewers were able to ensure that the test-taker was holding the same context in mind, during both the initial test and the re- test procedure. The interview used was the Language and Behaviour Profile (the oral version for eliciting metaprogram measures, upon which the iWAM Profile is based). The study (n=70) conducted in France revealed a correlation of r=0.90 between the iWAM patterns elicited and the LAB profile metaprogram equivalents, detected in the re-test procedure.

A further formal test re-test study was conducted with students (n=64) from the School for Professional Studies at Saint Louis University, USA between February and April, 2007. The time between the tests was approximately five weeks (38.85 days, SD = 16.77 days).

The results of this study demonstrated that when comparing the test-retest scores for the metaprogam patterns for the entire participant group, the average absolute scores between test and re-test moved only 2%. Further analyses using T- testing, revealed that there were no statistically significant differences between the means (p < 0.05) for any of the forty eight iWAM patterns between the test and re-test process. Also, F-tests revealed that the variance demonstrated significant differences for only three of the forty eight patterns. According to Merlevede (2007) these results are consistent with previous studies associated with the iWAM standard groups, where findings concluded that the standard error for the patterns was less than 5%. The test re-test study (2007), while not the method of choice for an instrument of this design, demonstrated in that the iWAM patterns are generally robust.

Reliability: falsification of answers

The iWAM questionnaire was built to prevent test falsification and this has been achieved by eliminating socially acceptable answers, by using a ranking system to answer questionnaire items and examining the consistency of all answers.

iWAM is available in twelve languages, each of which has been carefully translated by a native speaker of the language, ensuring that the Questionnaire have worldwide uniformity and reliability. iWAM has also been constructed to detect falsified or inconsistent answers.

Results can be verified by presenting the equivalent LAB Profile question to the test taker. The iWAM system notifies end users how many test items have been unchanged and an information analysis report is available to check the consistency of the responses to similar questions, with comments on the reliability of the test. The Information Analysis Form available with all Questionnaire results displays the ranking given to each test Item by the test taker, checking the overall consistency of the answers.

iWAM and Factor Analysis

The intention and design of the iWAM instrument is based on modelling (and then reproducing) excellent performance in the workplace. The tool was designed to measure the fine distinctions in consciousness that could then be translated into measurable human behaviour. The detailed results of excellence could then be mapped into a template that may be taught to others, to increase their standard of performance. In order to achieve this purpose, the tool needed to consist of multiple and detailed independent measures, rather than larger units of measure evidenced in tools such as DISC and MBTI.

The iWAM instrument consists of 16 over-riding pattern categories. Although the patterns within each category may measure apparently opposite ways of thinking (e.g. Internal frame of reference and External frame of reference) each is an independent measure. In this case individuals may potentially score high or low on both opposing and independent measures within a Category. The inter correlations between measures within each pattern category are weak (r=+/- 0.22- 0.27) confirming the independence of each of the iWAM Metaprogram measures.

Contact Us for further information






      web design by Logo Pogo