Research Update

I hope you're ready, because this is a big one! It has been a while since we summarized recent stduies of relevance to test publishers. In this post, we clear the backlog by summarizing articles (from various journals) on the following topics:

• Validity of police-selection measures and differences between ethnic groups
  
• Reducing faking on situational judgment tests
  
• Consequences of frequent applicants in adverse impact analyses
  
• The influence of organizational culture on work-related personality requirement ratings: A multilevel analysis
  
• The relationship between financial history and counterproductive work behavior
  
• Measuring compensation satisfaction
  
• Does age of transformational leaders matter?
• Applicant–employee similarity and attraction to an employer
  
• Do you think KSAOs can be improved?
  
• Can coaching improve performance on structured interviews?
  
• Work values: Some demographic and cultural correlates
  
• How organizational climate affects discrimination claims
  

Criterion-related validity of Dutch police-selection measures and differences between ethnic groups

Meijer, Born, Terlouw, & van der Molen (2008) compared the validity of cognitive ability and non-cognitive ability measures (e.g., Big-5 personality, assessment center, structured interview) in predicting police training performance. Validities differed depending on Ethnic group. Participants were 2365 majority applicants and 682 minority applicants (minority applicants were primarily Antillean, Moroccan, Surinamese, or Turkish) training to become Dutch police officers. The authors cite recent evidence that general mental ability may be less predictive in police officer jobs than in most other occupations (r = .24; Salgado et al., 2003). In accordance with – though more extreme than – that evidence, the authors concluded that generally mental ability had low validity in predicting training performance for police officers, especially for the ethnic majority group (r = .14 for ethnic minority group; r = .04 for ethnic majority group). No big 5 personality dimension predicted performance for either ethnic group. Assessment centers and employment interviews were more predictive for the ethnic majority group. The authors discuss the potential impact of (1) bias in ratings by raters who were all members of the majority-group, (2) small variance in the training performance criterion, (3) degree to which training performance is likely to agree with subsequent on-the-job performance.

Full link to article: http://dx.doi.org/10.1111/j.1468-2389.2008.00438.x

On the consequences of frequent applicants in adverse impact analyses: A demonstration study

Automated application processes make applying to jobs easier, allowing candidates to apply to multiple positions within an organization or the same position multiple times. These repeat candidates can influence adverse impact analyses. Dunleavy et al. (2008) demonstrate the effects using simulations. The authors show how a single applicant applying multiple times can cause the appearance of adverse impact when none exists. Conversely, in cases where there is adverse impact, repeat applicants can hide the adverse impact. These effects are demonstrated in various selection scenarios using simulations. The authors then discuss methods of identifying repeat applicants and present options for handling data from them.

Full link to article: http://dx.doi.org/10.1111/j.1468-2389.2008.00439.x

The influence of organizational culture on work-related personality requirement ratings: A multilevel analysis

If organizational culture affects selection, the same job in different organizations should have slightly different requirements. Li et al. (2008) found just that: personality importance ratings differed between organizations. These importance ratings were related to organizational culture. Participants were 270 Chinese customer service representatives working for 37 mobile phone service companies owned by the same parent company. Participants rated their organizations culture and subsequently used a translated version of O*Net’s Work Styles instrument to rate the importance of several personality traits for their job. There was agreement within organizations and differences between organizations on two work-related personality requirements: achievement-orientation and conscientiousness. Participants were more likely to rate achievement-orientation as important when their organizational culture was highly achievement-oriented. Conscientiousness was rated as important in integrity-oriented cultures. Ratings of achievement and conscientiousness were particularly influenced by (higher in) team-oriented organizational cultures. The authors discuss implications for the transportability of results from job analyses.

Full link to article: http://dx.doi.org/10.1111/j.1468-2389.2008.00442.x

The relationship between financial history and counterproductive work behavior

Oppler et al. (2008) found that an employee’s financial history predicted their likelihood of being caught engaging in counterproductive work behaviors. Participants were 2519 employees of US government agency. Employees completed a financial history questionnaire upon applying for the job, and every 5 years after hire. The financial history questionnaire asked two questions: (1) “In the last 7 years, have you, or a company over which you exercised some control, filed for bankruptcy, been declared bankrupt, been subject to a tax lien, or had legal judgment rendered against you for a debt?” and (2) “Are you now over 180 days delinquent on any loan or financial obligation?” Counterproductive workplace behaviors were incidents identified and recorded by the organization (e.g., theft, accepting anything prohibited by law). The authors found that employees with some previous financial difficulty were significantly more likely than those without a record of financial difficulty to be identified in an incident of counterproductive workplace behavior (31.3% and 18.1% respectively). The effect size (phi = .13) was similar to those found in meta-analyses (e.g., Ones et al., 1993) investigating overt integrity measures with objective theft criterions. The authors suggest that future research should investigate the incremental validity of financial history over other integrity measures, and that future studies should examine the adverse impact of using financial history.

Full link to article: http://dx.doi.org/10.1111/j.1468-2389.2008.00445.x

Measuring compensation satisfaction

Noting how existing measures are deficient, Williams et al. (2008) describe the development of a model and measure for compensation satisfaction. They authors capture two forms of compensation: pay versus benefits. For each form of compensation, the authors look at two areas of reaction: outcomes versus procedures. The resulting model consists of four dimensions for pay satisfaction (level, structure, raises, and variable pay procedures satisfaction) and three dimensions for benefits (level, determination, and administration satisfaction). The corresponding measure consists of 29 items and is supported by exploratory and confirmatory factor analysis, as well as evidence that the scale relates to other variables (e.g., affective commitment) as predicted.

Full link to article: http://bpsoc.publisher.ingentaconnect.com/content/bpsoc/joop/2008/00000081/00000004/art00004

Age differences between leader and followers as a moderator of the relationship between transformational leadership and team performance

According to Kearney (2008), transformational leadership behaviors may only be effective when leaders are older than their subordinates. The study involved 289 people in 49 research and development teams employed at a pharmaceutical company. When the leader’s age was older than the average age of the team members, transformational leadership behaviors (measured by the MLQ-5x Short) positively related to team performance (supervisor ratings of efficiency, quality of innovations, productivity, mission fulfillment, and overall performance). However, when the leader was the same age as or younger than the average team member, transformational leadership behaviors were unrelated to team performance. The authors suggest the reason for the difference is that age is often resource allocation (e.g., leadership responsibilities typically fall on the older members of the group). When a leader lacks this justification, the team may attempt to alleviate any inequality in resources by curtailing the leader’s influence on the team (e.g., not adopting the leader’s values).

Full link to article: http://bpsoc.publisher.ingentaconnect.com/content/bpsoc/joop/2008/00000081/00000004/art00011

Applicant–employee similarity and attraction to an employer

Labelled “person-to-person fit,” Devendorf & Highhouse (2008) found that people are attracted to an organization in part based on their similarity (perceived or prototypical) to employees who currently work at the organization. The authors categorized retail clothing stores as either conservative/conventional, sporty, or unconventional using multidimensional scaling. Grad students categorized numerous adjectives into those same categories based on their prototypes for typical retail employees at each store. Finally, 296 participants (1) rated how similar they were to employees at each organization (perceived similarity), (2) rated themselves on the adjectives categorized by graduate students (used to calculate similarity to the organizational prototypes), and (3) indicated each store’s attractiveness as an employer. Results were that both perceived similarity and similarity to the organization’s prototype predicted perceptions of organizational attractiveness. Perceived similarity was the stronger predictor, but prototype match showed promise in discriminating who would find which employers attractive.

Full link to article: http://bpsoc.publisher.ingentaconnect.com/content/bpsoc/joop/2008/00000081/00000004/art00002

Do you think KSAOs can be improved?

Your answer to the question likely depends on your occupation. Compared to people with Ph.D.s in applied psychology, Maurer & Lippstreu (2008) found that people in the general workforce were more likely to believe that someone can apply effort to improve their score on most knowledge, skills, abilities and other job requirements (KSAOs).

In 524 working adults, belief in the improvability of KSAOs was also related to positive views of selection tests and performance appraisal, degree to which KSAOs were needed at job entry, and learning goal orientation. Experts with Ph.D.s were more likely to differentiate on which KSAOs could be improved upon, and gave lower improvability ratings in general. The authors suggest that views on KSAO improvability are in interesting addition to understanding aspects of organizational psychology.

Full link to article: http://bpsoc.publisher.ingentaconnect.com/content/bpsoc/joop/2008/00000081/00000004/art00012

Can coaching improve performance on structured interviews?

The short answer is yes. Tross and Maurer (2008) found that the level of coaching a person received affected his/her scores on a structured interview. 144 undergraduate students received one of three levels of training: low (introduction-only training), medium (introduction plus awareness training), and high (introduction plus awareness plus skill training). Those given higher levels of training did better on a subsequent structured interview. The authors also investigated mediators of this effect. Knowledge was a partial mediator, but interview efficacy and anxiety were not, suggesting that the reason that training worked was due to the increased knowledge of highly-trained participants.

Full link to article: http://bpsoc.publisher.ingentaconnect.com/content/bpsoc/joop/2008/00000081/00000004/art00001

Work values: Some demographic and cultural correlates

In 33 European countries, Warr (2008) explored relationships between demographic variables (sex, age, employment status, education level, culture) with various work values (e.g., importance of high pay, pleasant coworkers, job security, flexible hours). The authors replicate and extend several important findings, such as that women tend to place greater emphasis than men do on flexible working hours, pleasant coworkers, and opportunities to meet people, adding that these gender differences are particularly pronounced among part-time workers; young adults just starting to earn their wealth tend to place more importance on high pay than did older adults; people with higher levels of education placed greater emphasis on self-actualizing job content, whereas less educated respondents highlighted extrinsic features such as pay and conditions; people in communist countries placed more emphasis on extrinsic features compared to people in traditionally protestant countries, who more highly valued intrinsic job features.

Full link to article: http://bpsoc.publisher.ingentaconnect.com/content/bpsoc/joop/2008/00000081/00000004/art00009

How organizational climate affects discrimination claims

One might think that organizations with climates with poor diversity records would be more susceptible to discrimination claims. In actuality, the situation is slightly more complex. In both field and lab settings, Leslie and Gelfand (2008) found that the likelihood of internal discrimination claims (the kind that can be dealt with within the organization rather than legally) depended both on a person’s gender identity and the organizational diversity climate. People who don’t strongly identify with their gender are more likely to file internal discrimination claims when the diversity climate is bad than when it is good. However, women who strongly identify with their gender show the opposite: they are more likely to file claims when the diversity climate is good rather than bad. The authors propose that poor diversity climates constrain the behavior of women with strong gender identifications because those women are more susceptible to fearing retaliation or stigmatization.

Full link to article: http://dx.doi.org/10.1016/j.obhdp.2008.02.004