TECHNICAL REPORT
INTERPERSONAL DYNAMICS INVENTORY (IDI)
Bo Ekehammar
Professor
Stockholm University
Purpose
The purpose of this Technical Report is to present a psychometric assessment of the Interpersonal Dynamics Inventory, in the following referred to as IDI. The company that owns IDI is IDI Profiling AB, referred to as IDI Profiling in the following.
The IDI instrument
The IDI is an instrument that maps and describes roles at work. Roles are the result of complex social processes and how people understand the organization’s task. IDI taps role by measuring the impression a person makes on key stakeholders in a specific context. The IDI helps people and organisations become aware and bring insights to the social dynamics and how it affects results and culture.
The origin of the instrument can be traced back to Zackrison’s (1977) work and PhD dissertation at the University of Michigan, USA. After that, several statistical analyses were carried out as well as grammatical modifications of the inventory (see Larsson, 2013). The present IDI is the result of this work and denoted IDI 2.0 (see Grant & Larsson, 2020), where the scientific and theoretical rationale of the IDI is presented further as well. There have been two technical reports on the IDI instrument (Ekehammar, 2013, 2017). The first was directed to test (a) if the IDI ratings were distributed as expected on the dimensions Directiveness, Affiliation, and Adaptability and (b) whether the reliabilities were satisfactory for both the self-ratings (how the target person rates her-/himself) and external ratings (ratings made by other persons of the target person). The results showed that the psychometric properties of the two first dimensions Directiveness and Affiliation are excellent and that the properties for the adaptability scale is acceptable/good). The second technical report analyzed whether the number (6, 8, and 10) of external raters affected the inter-rater reliability of the external ratings. The results showed that the number of raters did not affect the inter-rater reliability systematically but that the ratings on the dimensions Directiveness and Affiliation had a systematically higher inter-rater reliability (varying between .81 and .89) than the dimension Adaptability (varying between .65 and .75).
The present report tries to find out if a change in the presentation of items (word-pairs) in May 2019 has affected the psychometric properties of the IDI. Thus, before this date, the items were presented in the same order for all participants but after this date every participant receives a new random presentation order of the items. The data analyzed in this report are thus based on a random presentation order.
An important change was made in August 2019. Before that date the target person was instructed to select respondents from a variety of social contexts – excluding partners and persons from the own family. The procedure was based on the assumption that the instrument should tap the target person’s average, stable social style (trait). From August 31, 2019 the instructions are to primarily choose respondents that interact with the target person in a specific well-defined context (state). In short, the focus changed from trait to state in order to move the applications of the instrument more toward the target person’s immediate work-environment in order to maximize the effect of mapping.
The present IDI version, and the one I have evaluated, comprises 42 items (word-pairs).These items are answered on a 7-point Likert scale. However, only 33 of these items are used to calculate the scores that are used for the IDI result whereas the remaining 9 items are included to collect data for a possible modification of the IDI. Thus, the following psychometric analyses are based on the 33 potent items of the IDI. These items are chosen to cover three aspects (dimensions, factors) of people’s behavior that, in IDI terminology, are denoted Directiveness (how much the target person is perceived as being striving for influence or control over other people), Affiliation (how much the target person is perceived as forming close personal relations with other people), and Adaptability (how much the target person is perceived as being flexible by adapting his/her behavior to the needs of other people or to various situations). There are 11 items aimed at measuring each of the three dimensions. Examples of words measuring high Directiveness are Competitive, Demanding, and examples of words measuring low Directiveness are Cooperative, Obedient. Examples of words measuring high Affiliation are Open, Sensitive and examples of words measuring low Affiliation are Closed, Cold. Examples of words measuring high Adaptability are Flexible, Compliant, and examples of words measuring low Adaptability are Predictable, Rigid.
The target person is the person who is being rated by (six to ten+ persons in the present case) other people (external raters), who make their ratings independent of each other. The target person also rates himself on the IDI and these ratings are here denoted self-ratings. Thus, in an actual IDI test situation, the target person has to choose at least 6 other persons who know him or her quite well and in addition rate himself/herself on the same IDI items independent of the external raters. The psychometric assessment will include both the external ratings (which are the most important for the IDI) but also the self-ratings. The items of the IDI consist of word pairs where one of the words hits one end of the dimension that the word attempts to measure whereas the other word in the pair is an antonym or a word that defines the other end of the continuum or some of the other factors (for Directiveness, e. g, powerful – obedient). Sometimes, the word defining “high” on the continuum is placed to the right and sometimes to the left so the scoring of some of the items has to be reversed.
The output presented to the target person is not the continuous scores on Directiveness, Affiliation, and Adaptability. Instead, the data are presented, first, in a four-field graphical presentation where a person scoring low on both Affiliation and Directiveness is denoted Processor, one scoring high on both is called Motivator, one scoring high on Directiveness and low on Affiliation is denoted Producer, and one scoring high on Affiliation and low on Directiveness is denoted Relator. This picture is further fine-graded by dividing each of the four fields into another four fields which makes up 16 subcategories. So, a person with a (very) low score on both Affiliation and Directiveness is denoted A-4 (“Analytical Processor)” and a person having (very) high scores on both these dimensions is denoted D-1 (“Expressive Motivator”). Actually, the raw scores are not directly used for this categorization but rather the raw scores transformed to normed scores expressed as percentile intervals (1-12) based on data from the last years´ IDI testing and using the appropriate norm group (for Sweden, Norway, United Kingdom, United States, Brazil, etc.). These categories are called behavioral styles or IDI styles. The target person’s position and the position of the external persons’ combined ratings are then represented by two dots in the graphical representation so one can see how others perceive the target person and how he or she perceives him- or herself. The external raters’ combined rating is based on the mean expressed in a percentile interval (1-12) for the relevant norm group. If the disagreement among external raters is high (SD > 0.8), this is noted in a text. The third dimension of IDI, Adaptability, is not combined with the other two dimensions above, and the score on this dimension is communicated through a percentile score (1-100) based on the data from the last years’ testing).
The IDI data
Demographic description
The IDI data analyzed in this report were collected in Swedish and in Sweden by IDI Profiling between August 1, 2019 and August 31, 2020. Altogether, data from 42,615 persons were collected, 5,022 of which were delivered from a target person (self-ratings) and 37,593 from an external person (external ratings). Because the external ratings were made anonymously, there is no demographic information to present for this group whereas some demographic information for the target persons is available and presented in Table 1. Before that, the respondents were classified as to which norm they were evaluated against. 92,3% were evaluated against a Swedish norm, 1,5% according to a norm for United Kingdom, 1,5% with a norm for United States of America, 1,3% with a norm for Brazil, 1,2% with a norm for France, and 1,1% with a norm for the Netherlands. I have chosen to pick out those evaluated with a Swedish norm in Table 1 (N=3,596) but the results showed to be very much the same if all participants had been analyzed together.
Table 1. Description of the target persons (n = 3,596) in the total sample evaluated with a Swedish norm.
| Variable | Categories | Percent (%) |
| Gender | Female | 49.5 |
| Male | 50.5 | |
| Age | < 29 | 11.7 |
| 30-39 | 26.4 | |
| 40-49 | 33.4 | |
| 50-59 | 24.6 | |
| 60+ | 4.0 | |
| Education | Primary or Secondary school | 27.1 |
| Post-Secondary without degree | 10.4 | |
| University with degree | 62.5 | |
| Sector | Private | 53.7 |
| Public | 39.0 | |
| Non-Profit | 2.4 | |
| Other
|
4.9
|
|
| Subordinates | Yes | 50.8 |
| No | 49.2 |
Table 1 shows that the target persons were equally divided between men and women, a large majority was between 30 and 59 years of age, around 60% had a university degree, a majority worked in private business, and a slight majority of the target persons had subordinates in their jobs.
Descriptive statistics
Because each of the three scales is built on 11 items the total means are meaningful to compare. Thus, Table 2 shows the total means and standard deviations of self-ratings and external ratings, respectively, for each of the three factors.
Table 2. Total mean (M) and standard deviation (SD) of each IDI factor for self-ratings (N = 5,022) and external ratings (N = 37,593), respectively.
| Factor | Self-ratings | External ratings | ||
| M | SD | M | SD | |
| Directiveness | 47,37 | 10,12 | 48.09 | 9.92 |
| Affiliation | 41,09 | 10.30 | 39.90 | 10.70 |
| Adaptability | 59,85 | 7.39 | 58.12 | 10.47 |
As can be seen in Table 2, the self-rating means are higher in 2 of 3 comparisons, and the standard deviations lower in 2 of 3 comparisons, than corresponding external ratings. The difference is most pronounced for Adaptability. Comparing the three factors, the Adaptability means are the highest, and the Affiliation means the lowest, with the Directiveness means falling in between. That is the same picture as shown in the first and second technical reports (Ekehammar, 2013, 2017).
The product-moment correlations among the total scores for the three theoretical IDI factors are depicted in Table 3. As shown there, the magnitude of the correlation coefficients is low. Thus, it seems as if the IDI factors is almost independent of each other and contributes with unique information to the IDI profile. The same picture was shown in the two previous technical reports (Ekehammar, 2013, 2017).
Table 3. Product-moment correlations among the total scores of the theoretical IDI factors with external ratings (N = 37,593) above the main diagonal and self-ratings (N = 5,022) below.
| Factor | Directiveness | Affiliation | Adaptability |
| Directiveness | — | .086 | -.024 |
| Affiliation | .128 | — | .156 |
| Adaptability | .106 | .136 | — |
Reliability
Perhaps the first question to ask in a psychometric assessment of an instrument is whether the reliability is satisfactory or not. In the present case, it was possible to assess two types of reliability, internal consistency and inter-rater reliability.
Internal consistency
The internal consistency was measured through Cronbach’s Alpha (see e.g., Crocker & Algina, 1986) on the total sample of self-reports and external ratings, respectively. Cronbach’s Alpha is to a large extent dependent on the number of items in a test or factor (the more items the higher the index) but this is not a problem here because all three scales has the same number of items. However, as a complementary index of item homogeneity, the average inter-item correlation was also computed. Whereas Cronbach’s Alpha denotes the reliability of the total score, built on the sum of 11 items, the average inter-item correlation coefficient provides an estimate of a single item’s reliability. Table 4 presents the internal consistency reliability data assessed as described above.
Table 4. Internal consistency reliabilities (Cronbach’s alpha) and average inter-item correlations (within parentheses) of each IDI factor for self-ratings (N = 5,022) and external ratings (N = 37,593), respectively.
| Factor | Self-ratings | External ratings |
| Directiveness | 0.900 (0.450) | 0.891 (0.426) |
| Affiliation | 0.881 (0.421) | 0.855 (0.353) |
| Adaptability | 0.850 (0.282) | 0.918 (0.504) |
Table 4 shows that Directiveness (self-ratings) and Adaptability (external ratings) had a higher internal consistency than for the other factors. According to Kline (1999) and others, a Cronbach’s Alpha above .90 can be labeled “excellent” so the internal consistency of Directiveness (self-ratings) and Adaptability (external ratings) was thus excellent in the present case. By the same authors, a Cronbach’s Alpha between .80 and .90 can be labeled “good” so in the present case the internal consistency of Affiliation (self-ratings and external ratings), Adaptability (self-ratings) and Directiveness (external ratings) was good. For Adaptability the situation is complicated by the fact that the self-ratings have the lowest internal consistency whereas the figure for the external ratings is quite satisfactory. The same pattern was found in the first technical report (Ekehammar, 2013)
Inter-rater reliability
A final reliability issue to report concerns inter-rater (inter-judge) reliability, that is, the agreement among external raters when rating the target person. Thus, each target person is in the present case rated by six (35%), seven (22%), eight 15%), nine 11%) or 10+ (17%) persons chosen by the target person, which means that the external raters differ from target person to target person. This is not an optimal situation for estimating inter-rater reliabilities because some variance sources that can be isolated in a fully crossed design (all target persons are rated by the same raters) cannot be isolated here. However, using analysis of variance (ANOVA) and computing intraclass correlation coefficients (ICCs) a rough estimate of inter-rater reliability can be obtained (see e. g., Shrout & Fleiss, 1979). As the original IDI data file was not suited for these calculations, a new data file was constructed by taking a random sample of 50 target persons and forming a 50 x 8 Target person by External rater matrix for each of the three IDI dimensions. The median number of raters was 8 and that is why 8 raters were chosen for the present analysis. Applying a random effects one-way ANOVA to these data, the inter-rater reliability could be estimated as an ICC (see Shrout & Fleiss, 1979, Case 1) for each IDI dimension. The ICC reliability can be computed for a single rater or for the raters’ combined score. The last-mentioned aspect is of primary importance here (see p. 5). Thus, Table 6 presents the inter-rater reliabilities for the external raters’ combined ratings.
Table 6. Inter-rater reliabilities expressed as intraclass correlation coefficients for the external ratings of 50 randomly selected target persons rated by 8 external persons, respectively.
| Factor | External ratings (n = 400) |
| Directiveness | .86 |
| Affiliation | .83 |
| Adaptability | .86 |
Table 6 discloses that the inter-rater reliability coefficients for all IDI dimensions are of high and quite satisfactory magnitudes. The two previous technical reports showed that Directiveness and Affiliation had high and satisfactory inter-rater reliabilities whereas Adaptability had a low and unsatisfactory inter-rater reliability in the first report and a consistently lower magnitude than the others in the second report. However, in the present case, the inter-rater reliabilities were satisfactorily high and of the same magnitude for all three IDI dimensions.
Construct validity
Factor analysis
A central question is of course if the three theoretical IDI dimensions can be revealed when making an empirical factor analysis of the 33 items in the IDI questionnaire. To answer this question, an exploratory factor analysis (EFA) was carried out using principal factors with a Varimax rotation of the number of factors that according to a scree test (Cattell, 1966) should be retained for rotation. The scree test indicated clearly three factors for both self-ratings and external ratings. The eigenvalues and the proportion explained variance of each factor and totally are shown in Table 7. As this table shows, with the exception of Factor 2, the factors explain each more variance for external data than for self-rating data. Thus, for external ratings the three factors explained together 53.3% of the variance in IDI ratings whereas for self-ratings these factors explained 48.5%. The results are very close to the figures presented in the two previous technical reports on IDI (Ekehammar, 2013, 2017)
Table 7. Initial eigenvalues and explained variances for self-ratings (N = 5,022) and external ratings (N = 37,593), respectively.
| Factor | Self-ratings | External ratings | ||
| Initial Eigenvalue | Explained Variance | Initial
Eigenvalue
|
Explained Variance | |
| 1 | 6.37 | 19.9% | 7.08 | 22.1% |
| 2 | 5.39 | 16.9% | 5.10 | 15.9% |
| 3 | 3.76 | 11.7% | 4.90 | 15.3% |
| Total | 48.5% | 53.3% | ||
As a further analysis, the factor scores of the three factors were computed and correlated with the total scores for Directiveness, Affiliation, and Adaptability. The results are presented in Table 8.
Table 8. Product-moment correlations between factor scores for Factor 1, 2, and 3 obtained through the factor analysis and the total scores on Directiveness, Affiliation, and Adaptability for self-ratings (N = 5,022) and external ratings (N = 37,593), respectively).
| Factor | Self-ratings | External ratings | ||||||
| Factor 1 | Factor 2 | Factor 3 | Factor 1
|
Factor 2 | Factor 3 | |||
| Directiveness | .989 | .989 | ||||||
| Affiliation | .991 | .993 | ||||||
| Adaptability | .986 | .991 | ||||||
Note. Only the strongest correlation for each factor is displayed.
The results in Table 8 shows that for both self-ratings and external ratings Factor 1 is strongly correlated with Directiveness, Factor 2 is strongly correlated with Affiliation whereas Factor 3 is strongly correlated with Adaptability. The results are very similar to those presented in the two previous technical reports (Ekehammar, 2013, 2017), that is, for both self-ratings and external ratings, there is one factor score that correlates strongly (around 0.99) with the total score for Directiveness, Affiliation, and Adaptability, respectively.
This picture is further developed in Table 9, which displays the whole factor matrix showing each item’s highest factor loading on each factor. The factor matrix shows a simple structure as all items’ highest loading is on one factor only. For both self-ratings and external ratings, all IDI Directiveness items load strongest on Factor 1, all Affiliation items load strongest on Factor 2 whereas all Adaptability items load strongest on Factor 3.
Table 9. Exploratory factor analysis of IDI items using principal axis factoring and Varimax rotation for external ratings ((N = 37,593) and self-ratings (within parentheses, N = 5,022), respectively. Only factor loadings above .4 are presented.
|
Item no x, expected to cover |
Factor 1 | Factor 2 | Factor 3 |
| 5, Directiveness | .61 (.62) | ||
| 8. Directiveness | .56 (.59) | ||
| 10. Directiveness | .68 (.70) | ||
| 14, Directiveness | .75 (.76) | ||
| 17. Directiveness | .60 (.62) | ||
| 20. Directiveness | .62 (.61) | ||
| 26, Directiveness | .67 (.79) | ||
| 45. Directiveness | .77 (.66) | ||
| 47. Directiveness | .65 (.66) | ||
| 64. Directiveness | .65 (.66) | ||
| 72. Directiveness | .68 (.70) | ||
| 4. Affiliation | .72 (.74) | ||
| 9 Affiliation | .55 (.53) | ||
| 16. Affiliation | .67 (.68) | ||
| 21. Affiliation | .71 (.68) | ||
| 23. Affiliation | .49 (.49) | ||
| 25. Affiliation | .61 (.60) | ||
| 32. Affiliation | .62 (.63) | ||
| 42. Affiliation | .65 (.63) | ||
| 52. Affiliation | .71 (.69) | ||
| 56. Affiliation | .62 (.58) | ||
| 67. Affiliation | .75 (.60) | ||
| 18. Adaptability | .73 (.61) | ||
| 36, Adaptability | .70 (.63) | ||
| 37. Adaptability | .71 (.58) | ||
| 38. Adaptability | .71 (.55) | ||
| 43. Adaptability | .67 (.54) | ||
| 44. Adaptability | .71 (.51) | ||
| 49. Adaptability | .70 (.52) | ||
| 57 Adaptability | .76 (.65) | ||
| 59. Adaptability | .74 (.63) | ||
| 74. Adaptability | .70 (.61) | ||
| 76. Adaptability | .67 (.51) |
The picture obtained from the exploratory factor analysis appears very clear, and corresponds very well with expectations and with the two previous technical reports on IDI (Ekehammar, 2013, 2017), although the item number is not the same or some items might have been replaced. Because of that it did not seem necessary to carry through a confirmatory factor analysis, which in my first technical report confirmed the three factor structure of the IDI data.
Conclusions
This is the third technical report on the IDI instrument. In contrast to the two previous reports, this one analyzed the IDI data based on a random presentation of items (word-pairs) for each participant, whereas the two previous reports analyzed IDI data based on the same presentation order of word-pairs for all participants. On every point in the analysis, I have compared the result with those of the two previous reports. On every point the results are very similar and the general conclusion is that the random presentation of items (word-pairs) has not affected the psychometric properties of the IDI instrument. The only miss-match in results was that Adaptability in the present study showed a higher inter-rater reliability than in the two previous. A working hypothesis is that this change has to do with the move from trait to state, the new instructions and method for data-collection to a specific context rather than to a general view of the target person.
References
Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 27, 509-540.
Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Harcourt Brace Jovanovich.
Ekehammar, B. (2013). Technical report on the Interpersonal Dynamics Inventory (IDI).
Ekehammar, B. (2017). Technical report on the Interpersonal Dynamics Inventory (IDI).
Grant, J., & Larsson, T. (2020). IDI 2.0 – Bortom personlighet. Stockholm: IDI Profiling.
Kline, P. (1999). The handbook of psychological testing (2nd ed.). London: Routledge.
Larsson, T. (2007). Utvecklingen av Interpersonal Dynamics Inventory (IDI). Täby, Sweden: Effectiveness Consultants.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability.
Psychological Bulletin, 86, 420-428.
Zackrison, R. E. (1977). Adult distributive education instructor behavior and effectiveness as perceived by students. Minneapolis, MN: University of Minnesota.
Bo Ekehammar
Stockholm 2020-12-16