A number of working groups were formed as part of the 2008 ATP Test Security Summit. The IP Theft Collaborative Working Group has recently published its work product. If you don't have a password to access the wiki, there is information on the page on how to obtain one. Also, information on the 2009 Security Summit can be found on the ATP Conference website.
Read more!
ATP Intellectual Property Theft Collaborative Working Group
Journal of Applied Testing Technology
The Association of Test Publishers publishes a peer-reviewed online journal, Journal of Applied Testing Technology edited by Chad W. Buckendahl and William G. Harris. The most recent article is a piece by Nathan Thompson, entitled "A Proposed Framework for Test Administration Methods." The article compares and contrasts a number of test administration methodologies including CBT, CAT, CCT, and LOFT.
Read more!
I-O Division Elections
Congratulations to Julie Carswell who has been elected Secretary of the I-O Division for 2009! She will join the 2009 I-O Division leadership team of Ryan Ross (Chair) and John Weiner (Vice Chair). Julie's contributions to the blog over the past year are much appreciated, and her continued involvement with be a boon to the Division and ATP.
Read more!
Interesting Article on APA's Position on Torture
In September 2008, the American Psychological Association banned its members from participating in interrogations that use torture. (This contrasts with other organizations like American Medical Association and the American Psychiatric Association which have had such bans in place since 2006.) In an interesting article, Stanley Fish outlines some of the factors that he believes account for this differential response. One factor that he sees as being primary is the fact that in contrast to organizations that are comprised exclusively of health care providers, psychologists may work in roles that extend beyond the concept of "healer."
Read more!
Some "Must Read" Articles
The September 2008 edition of Industrial and Organizational Psychology: Perspectives on Science and Practice (table of contents) has two series of articles that are likely of great relevance to anyone interested in personnel selection. The first set of articles focuses what the authors see as some of the key questions facing personality researchers working on practice-related issues. The second looks at some of the factors which contribute to the resistance of hiring mangers to use empirically supported selection methods in favor of other methods, primarily unstructured interviews, which tend to have very limited predictive validity.
Read more!
ATP Conference Registration Now Open
Registration for 2009 ATP Conference is now open. Innovations in Testing 2009 will be held February 22-25 2009 in Palm Springs, CA. See the Conference website for full information.
Read more!
Call for Papers: 11th European Congress of Psychology
The European Congress of Psychology (ECP2009) to be held in Oslo 7-10 July 2009 will have a special track on developments in testing and test use. Chaired by Dave Bartram, this track will be organized in close cooperation with the ITC and with the EFPA Standing Committee on Tests and Testing. The track will consist of keynote addresses, state of the art lectures, symposia and a number of parallel paper and poster sessions which will run throughout the Congress. If interested, please send an e-mail here.
Read more!
Group differences in personality – Meta-analyses comparing five U.S. racial groups
Foldes, Duehr, and Ones conducted a large-scale meta-analysis to examine the magnitude of racial group differences on measures of personality and whether these differences are likely to result in adverse impact.
Due to the paucity of analyses and research in this area, this study provides an important contribution to the literature. The current research extends beyond existing research in many ways. First, researchers included understudied racial groups such as Asian Americans and American Indians and used a reliable estimate of the magnitude of racial group differences (d-values). The authors also examined differences at both the broad factor and narrow facet levels. Data from 44 different personality assessments was used and more than 700 effect sizes contributed to the database. Results suggested that in general, racial group differences were negligible and unlikely to result in adverse impact. However, there is some concern for adverse impact for certain groups and traits but this is dependent on characteristics of the selection scenario such as the trait being measured, the effect size, the composition of the applicant pool, and the selection ratio. The authors present a summary of potential trait-group combinations that may result in adverse impact. Specifically, adverse impact could be a concern for Blacks when Emotional Stability, Anxiety, Extraversion and Sociability are measured. There is some concern for Asians when Emotional Stability, Even Tempered, Extraversion, Dominance, Sociability, and Conscientiousness are measured, and for Hispanics when Sociability is measured. For American Indians, there is some concern when Emotional Stability and Extraversion are measured and for Whites there is some concern for Conscientiousness and its facets (Achievement, Cautiousness, Order), Extraversion, and Self-Esteem. This research has implications for the use of personality assessments for selection, such that practitioners should carefully consider the job-related traits being measured, the composition of their applicant pools, and their selection ratios.
View full abstract/get the article at:
http://www.ingentaconnect.com/search/article?title=group+differences+in+personality&title_type=tka&year_from=1998&year_to=2008&database=1&pageSize=20&index=4
Read more!
Submission Deadline for EAWOP Conference Approaches
October 3, 2008 is the proposal submission deadline for the 14th European Congress of Work and Organizational Psychology to be held in Santiago de Compostela, Spain on May 13-16, 2009. Organized under the auspices of the European Association of Work and Organizational Psychology (EAWOP), the conference has the theme "Developing People in 21st Century Organizations: Global and Local Perspectives."
Read more!
Article on Emotional Intelligence
The lead article (abstract) of the September 2008 edition of the American Psychologist is a paper by John Mayer, Peter Salovey, and David Caruso reviewing the concept and subsequent controversy and confusion that have emerged in regard to the construct of emotional intelligence. The first to propose the idea of emotional intelligence, the authors explore some of the complexities which have emerged in relation to this concept since their original work in 1990 and make suggestions for future work in the area.
Read more!
Theory of Geographic Differences in Distribution of Personality Traits
In a recent article (here's the abstract) published in Perspectives on Psychological Science, Peter Rentfrow and colleagues outline their theory about how geographic differences in personality are developed and maintained. The author's website also has links to a wide variety of papers likely to be of interest which explore the relationship between personality and topics ranging from voting behavior, music preference, to matchmaking.
Read more!
International Test Commission Presentations
Many of the presentations from the International Test Commission Conference, held in July 2008 in Liverpool UK, are now available online.
Read more!
Research Update
The September issue of the International Journal of Selection and Assessment (IJSA) contained a number of interesting articles. We provide a brief summary of the following articles:
- Validity and/or Reducing Adverse Impact – Still a Complex Balancing Act
- A New Source of Bias in the Performance Appraisal Process?
- Knowledge and Beliefs about Assessment Tools and Techniques: UK Practitioners Perspective
- Personality Plays an Important Role in How to Handle Retail Customers
- Are Nonverbal Measures of Cognitive Ability more Likely to Produce Fair Employment Decisions?
- CAT and Personality Testing: A Cautionary Note
- Important Questions on the Construct Validation of Assessment Center Dimensions
Validity and/or Reducing Adverse Impact – Still a Complex Balancing Act
This issue contains an interesting and thoughtful exchange on the concept of Pareto-optimal predictor composites as a potential approach to alleviating (not to be confused with resolving) the validity diversity dilemma. Pareto-optimal composites refer to a specific approach for assigning weights to individual predictors in a selection composite. This approach, which is primarily contrasted with regression weighting, suggests there is value in considering alternative weighting schemes that may provide a more balanced trade-off between validity and adverse impact (AI). De Corte, Lievens, and Sackett essentially argue that the function of regression-based composites is to maximize validity. At the opposite end of the spectrum, one could adopt a weighting approach that results in the highest possible adverse impact ratio (AIR). Pareto-optimal weights reflect a more balanced tradeoff between these two approaches that allows organizations to ask “what reduction in validity are we willing to accept in order to increase diversity?” This paper is an extension of De Corte’s and colleagues 2007 work that introduced the concept of Pareto-optimal tradeoffs. The authors attempt to demonstrate that, in contrast to findings reported by Potosky, Bobko, & Roth (2005), adding non-cognitive measures to a cognitive ability test can result in more than and a modest decrease in adverse impact - if one is willing to consider alternatives to the regression weighting method. In two separate commentaries Potosky and her colleagues, and Kehoe respond to De Corte et al.’s description of the Pareto-optimal concept, their findings, and the implications and interpretability of these findings for organizations.
View full abstract/Get the article at:
http://www3.interscience.wiley.com/journal/121382614/abstract
A New Source of Bias in the Performance Appraisal Process?
The Influence of a Manager’s Own Performance Appraisal on the Evaluation of Others
Latham, Budworth, Yanar, and Whyte examine the influence of managers’ most recent performance appraisal on their subsequent appraisal of others. They hypothesized that a manager who receives a favorable performance appraisal will subsequently evaluate another person more positively than a manager who receives an unfavorable appraisal. Four separate studies were conducted to test the hypothesis – a case study, a lab experiment and two field studies. In the lab setting, managers received hypothetical feedback then were asked to rate an individual’s videotaped performance. The field studies were conducted with managers in a manufacturing plant in Canada and in a retail organization in Turkey to determine whether the lab study results would generalize to these settings. Findings from the field supported the results of the case study, the lab study, and the hypothesis that performance evaluations received by managers predicted their performance evaluations of their employees. These findings support the anchoring and adjustment hypothesis, which states that people look for a guidepost or anchor when making estimates of value under uncertainty, and these anchors can often be based on irrelevant information. This research supports the importance of rater training and the use of highly structured behavioral appraisal instruments as potential strategies for reducing bias in performance ratings.
View full abstract/get the article at:
http://www3.interscience.wiley.com/journal/121382612/abstract
Knowledge and Beliefs about Assessment Tools and Techniques: UK Practitioners Perspective
HR Professionals’ Beliefs About, and Knowledge of, Assessment Techniques and Psychometric
Tests
Furnham examines the knowledge and beliefs about various assessment tests and techniques among 255 HR and or related professionals in the UK. Respondents provided information on a number of assessment methods including interviews, references, assessment centers and “personal hunch.” They were also asked to share information about their knowledge of 21 personality tests and 19 cognitive ability tests. Practitioners considered Assessment Centers, Cognitive Ability Tests, and Work Samples to be the most valid techniques. The most widely known and used personality tests were the 16PF and the MBTI. The Belbin Team Role Inventory and the FIRO-B were also widely recognized and used. The NEO-PI-R and the 16PF were rated as most useful for selection, while the MBTI and FIRO-B were considered most useful from a development perspective. The most widely known and used Cognitive Ability tests were the Graduate and Managerial Assessment and the Watson-Glaser Critical Thinking Appraisal. Furnham considers the implication of these results for educating and informing practitioners.
View full abstract/get the article at:
http://www3.interscience.wiley.com/journal/121382621/abstract
Personality Plays an Important Role in How to Handle Retail Customers
Effects of Personality Characteristics on Knowledge, Skill, and Performance in Servicing Retail Customers
Motowidlo, Brownlee, and Schmit investigated knowledge, skill, and personality characteristics associated with effective customer service. Following from research indicating that ability affects performance primarily through its effects on technical knowledge and skill, the authors of the current study tested whether the effect of extraversion, agreeableness, and neuroticism on customer service performance is mediated through interpersonal knowledge and skill. The authors administered a situational interview to retail store associates to measure customer service knowledge, a role-play simulation to measure customer service skill, the NEO-FFI, and the Wonderlic Personnel Test were used to measure personality and cognitive ability respectively. Job performance was measured using supervisor ratings. Results from 140 associates indicated that extraversion, agreeableness, and neuroticism explain incremental variance in customer service knowledge beyond that accounted for by ability, experience, and conscientiousness. They also found that the effects of personality, ability, and experience on customer service performance all funnel through customer service knowledge and skill. There was also a moderating effect for conscientiousness such that customer service knowledge predicts performance best for those who are highly conscientious. Results suggest a causal model that could be tested in future research. These results have implications for hiring practices, lending support for matching knowledge content with interpersonally oriented personality traits.
View full abstract/get the article at:
http://www3.interscience.wiley.com/journal/121382617/abstract
Are Nonverbal Measures of Cognitive Ability more Likely to Produce Fair Employment Decisions?
Comparing the Performance of Native North Americans and Predominantly White Military Recruits on Verbal and NonVerbal Measures of Cognitive Ability
Vanderpool and Catano compared the cognitive ability of Native North Americans (Canadian First Nations Members) and predominately white Canadian Forces (CF) recruits on both verbal and non-verbal measures of cognitive ability. Previous research conducted by Lynn and Vanhanen (2006) has suggested that American Indians perform better on visual and spatial aspects of cognitive ability than they do on verbal measures. Based on these results, the authors of the current study explored whether the use of a nonverbal test could reduce the effect size between Native Americans and White applicants to <1> 1 SD. The nonverbal measures showed much smaller differences (CFAT- SA, d=.30, and SPM, d=.38.) between First Nations members and predominantly white CF recruits. Results indicated that making employment decisions solely on verbal cognitive ability scores is likely to produce adverse impact against First Nation members when verbal ability has not been identified as a bona fide occupational requirement. The authors recommend caution when making employment decisions based on total scores on verbal cognitive measures and suggest that nonverbal measures of cognitive ability are more likely to produce fair employment decisions without sacrificing the gain in utility of cognitive ability tests.
View full abstract/get the article at:
http://www3.interscience.wiley.com/journal/121382613/abstract
CAT and Personality Testing: A Cautionary Note
Effects of Changed Item Order: A cautionary note to practitioners on jumping to computerized adaptive testing for personality assessment
Ortner investigated the effects of item order on CAT delivery of the Eysenck Personality Profiler (EPP). CAT is a method for administering assessments that is tailored to each test taker in that questions are successively selected based on previous responses. The test taker’s individual trait level can be iteratively estimated during the testing process. The authors hypothesized that a questionnaire beginning with an extremely high trait level may cause test takers to agree less frequently with subsequent items as compared with questionnaires that begin with an item measuring a low trait level. They also hypothesized that participants would have longer response latencies when presented with an extreme or high trait level at the beginning of the questionnaire. The Eysenck Personality Profiler (EPP) was administered to four groups using four different test versions. Group 1 was presented the conventional item order, group 2 was presented an adaptive form with the first item representing a medium trait level, group 3 was presented an adaptive version with the first item representing a low trait level, and group 4 was presented an adaptive form with the first item representing a high or extreme trait level. Results found significant effects for mean person parameters gained for three scales: Manipulative, Hypochondriac, and Expressive. In other words, items representing a high trait level at the beginning of a questionnaire altered test takers’ performance on three of the seven personality scales. Results also indicated that longer response latencies might also be caused by effects of item order. This research has implications for practitioners and researchers who should consider the effects of item order when administering CAT for personality assessment.
View full abstract/get the article at:
http://www3.interscience.wiley.com/journal/121382609/abstract
Important Questions on the Construct Validation of Assessment Center Dimensions
Assessor Constructs in Use as the Missing Component in Validation of Assessment Center Dimensions: A critique and directions for research
Jones and Born critiqued the use of multitrait-multimethod (MTMM) approaches to measure assessment center (AC) dimension ratings and offered further recommendations for the construct validation of AC dimensions. The objective of the current research was to highlight the importance of construct definition and a priori theoretical descriptions of expected relations among rating dimensions, exercises, and dimension-exercise interactions. The authors attempted to answer the following questions (1) What are ‘traits’ and ‘constructs,’ how do we define them, and how do they relate to AC dimensions (2) What is the ‘method’ in ACs and (3) What does a correlation indicate about trait/method relationships? The results of the current research argue for clear articulation of AC dimensions in terms of trait permanence, specificity of constructs, and their interrelationships. Furthermore, the authors argue that exercises are not the only ‘method’ in ACs - other important aspects of AC methodology include raters, observations vs evaluation, and the rating process. Finally, the authors argue that application of MTMM analysis to ACs has limited value since it confounds methods with the object of study directly, by relying on a single methodological process (within-exercise ratings) to evaluate two methodological processes (within-exercise and behavioral report ratings). The authors offer several suggestions on how assessors or subject matter experts can improve their ratings of AC dimensions. This research has implications for practitioners and how they arrive at construct judgments through AC exercises.
View full abstract/get the article at:
http://www3.interscience.wiley.com/journal/121382622/abstract
Read more!
Reliability and Validity of a 3 Item Job Satisfaction Scale
A meta-analytic examination of the construct validity of the Michigan Organizational Assessment Questionnaire Job Satisfaction Subscale.
Another article in the most recent issue of JVB investigates the reliability and construct validity of a measure of Job Satisfaction. Bowling and Hammond use meta-analyses to examine the psychometric properties of the Michigan Organizational Assessment Questionnaire Job Satisfaction Subscale (MOAQ-JSS). The MOAQ-JSS is a brief, face-valid measure of global job satisfaction that was developed to overcome some of the limitations of other job satisfaction measures. The advantages of the MOAQ-JSS include its length (3 items) and its focus on the affective component of job satisfaction. Researchers present a nomological network of the hypothesized causes, correlates, and consequences of job satisfaction. Hypothesized causes include job characteristics, work stressors, social and organizational support, and person-environment fit. Hypothesized correlates include job attitudes, organizational justice, psychological and physical strains, and life satisfaction. Hypothesized consequences include in-role performance, extra-role performance, and withdrawal behavior. The authors also compiled normative data for the MOAQ-JSS. Results indicated that the MOAQ-JSS demonstrates acceptable levels of reliability and provide evidence of construct validity based on a pattern of relationships consistent with that predicted by the nomological network. Consistent with past research, continuance commitment was the only job attitude found to be weakly related to the MOAQ-JSS. This research supports the use of the MOAQ-JSS as a construct- and face- valid measure of global, affective job satisfaction, especially in circumstances where questionnaire length is a concern.
View full abstract/get the article at:
http://www.sciencedirect.com/science/journal/00018791
Read more!
More test items in the public domain
Development and initial validation of public domain Basic Interest Markers
In the most recent version of the Journal of Vocational Behavior (JVB), Liao, Armstrong, and Rounds developed and validated a pool of public domain vocational interest items. This research was inspired by Goldberg’s International Personality Item Pool (IPIP) – a widely-used public domain personality item pool developed to encourage personality research. Liao and colleagues contend that creating a pool of public-domain interest items would advance the science of interest measurement by making items instantly accessible at no cost for graduate students and others with limited financial means. They organized interest items into Basic Interest Scales called Basic Interest Markers (BIMs) based on research supporting the utility of this type of organizing structure. Basic Interest Scales group items according to homogeneous content areas that capture various related job titles. The study details the methods by which the 343 items and 31 BIM scales were generated and present validity evidence based on correlations with the Strong Interest Inventory (SII), and discriminant validity evidence based on the ability of BIMs to differentiate between 12 major fields of education and training programs. This work and the corresponding website:
https://netfiles.uiuc.edu/jrounds/IIP/home.htm
should serve to stimulate research in the career interest domain. However, it also has potential negative implications for test publishers who sell career interest measures. Specifically, some argue that these types of item pools have the potential to compromise the utility and integrity of published, commercially available tests.
View full abstract/get the article at:
https://netfiles.uiuc.edu/hliao/web/Liao_JVB_08inpress.pdf
Read more!
New EEOC Compliance Manual on Religious Discrimination
The EEOC recently released a new compliance manual section on religious discrimination in the workplace. This is accompanied by a best practices booklet and a Q&A fact sheet.
Read more!
IPMAAC Presentations Posted
Many of the presentations from the Thirty-Second Annual IPMAAC Conference on Personnel Assessment, held June 2008 in Oakland, are now posted.
Read more!
Smoke-Free Hiring Policies
It appears that Escambria County in Florida now includes a drug test to screen for tobacco use in its hiring process.The use of drug testing to ascertain smoke-free status appears to be a growing trend in those states that do not prohibit the use of smoking status as a criterion for hiring. It may prove interesting to monitor public reaction if this trend continues.
Read more!
Research Update
Summarizes and provides full links for the following research of interest:
- Insight on Enhancing Acceptance of Developmental Feedback
- What is Positive Psychological Capital (PsyCap)?
- More Evidence of the Predictive Value of Narrow Traits?
- Cultural Values Matter in Multi-Source Feedback Systems
Insight on Enhancing Acceptance of Developmental Feedback
Feedback acceptance in developmental assessment centers: The role of feedback message, participant personality, and affective response to the feedback session
In the most recent issue of the Journal of Organizational Behavior (JOB), Bell and Arthur explore factors influencing people’s acceptance of feedback in a development context. Feedback acceptance is defined as the recipient’s belief that the feedback accurately describes his or her performance. The authors’ predictions are based on self-enhancement theory (people are motivated to behave and think in ways that enhance their feelings of self-worth), self verification theory (people are motivated to behave and think in ways that are consistent with their self-view), and the affective infusion model (AIM; affect influences how we process and respond to information). Consistent with predictions based on self-enhancement theory and AIM, higher ratings from assessors resulted in higher feedback acceptance. This relationship was partially mediated by the positive affective reaction to the feedback session. In other words, higher ratings lead to higher acceptance, in part, because of the positive feelings the feedback session evoked among participants. Finally, both extraversion and agreeableness had an impact on the relation between assessor ratings and feedback acceptance, through their influence on affective reactions. This research has implications for the delivery of assessment results - suggesting that evoking a positive affective reaction in participants during the feedback session may enhance feedback acceptance.
View full abstract/Get the article at:
http://www3.interscience.wiley.com/journal/118823480/abstract
What is PsyCap?
A Call for Longitudinal Research in Positive Organizational Behavior
Also in this issue of JOB, Avey, Luthans, and Mhatre discuss the construct of positive psychological capital (PsyCap). PsyCap is defined as an individual’s positive psychological state of development characterized by self-efficacy (having confidence to take on and put in the necessary effort to succeed at challenging tasks), optimism (making a positive attribution about succeeding now and in the future), hope (persevering toward goals and when necessary redirecting paths to goals in order to succeed), and resilience (when beset by problems and adversity, sustaining and bouncing back and even beyond to attain success). According to the authors a distinguishing feature of PsyCap is its state-like, open to development nature. The authors acknowledge that much of the confusion surrounding PsyCap stems from each of the composite dimensions having been described in positive organizational behavior (POB) literature as both a state and a trait. They further suggest that PsyCap and other traits in the domain of POB may be best viewed in terms of a continuum or their degree of stability. The authors discuss the importance of longitudinal research designs in furthering an understanding of the stability of the construct over time.
View full abstract/Get the article at:
http://www3.interscience.wiley.com/journal/117908793/abstract
More evidence of the predictive value of narrow traits?
Beneath the Surface: Uncovering the Relationship Between Extraversion and Organizational Citizenship Behavior Through a Facet Approach
In the June issue of the International Journal of Selection and Assessment (IJSA), Moon, Hollenbeck, Marinova, and Humphrey propose that the lack of a relation between extraversion and OCB documented in previous research, is due to the measurement of extraversion at the factor level. In two studies the authors build support for the contention that the facets of extraversion defined as surgency, sociability, and positive emotions, predict citizenship behavior in opposing ways and, as a result, mask the predictive ability of the broad factor of extraversion. Consistent with this position, their results showed that each of the facets of extraversion were differentially predictive of OCB; sociability was unrelated to OCB, positive emotion was positively related to OCB, and surgency was negatively related to OCB. The opposing nature of the relations between surgency, positive emotions, and OCB, results in a suppression of the relationship between OCB and the broad factor of extraversion. In terms of practical applications, the authors point to the typical employment interview and caution of the danger in assuming that just because an individual appears extraverted we can assume a consistent set of behaviors; in their words, “One extravert is not necessarily the same as another.”
View full abstract/Get the article at:
http://www3.interscience.wiley.com/journal/120087714/abstract
Cultural Values matter in Multi-Source Feedback Systems
Do cross-cultural values affect multi-source feedback dynamics? The case of high power distance and collectivism in two Latin American countries
Also in the June issue of IJSA, Varela and Premeaux examine the role of two cultural values (High Power Distance and Collectivism) on MSF ratings of a group of managers from Venezuela and Columbia; two High Power Distance (HPD) and collectivist cultures. The authors contend that in HPD cultures, power differences separate superiors from subordinates by reinforcing perceptions of leaders’ superiority and restricting subordinate involvement in decision making. As a result, it was predicted that in HPD cultures, subordinates would provide the highest evaluations among feedback sources. This is contrary to typical findings in American samples (characterized by LPD), where self-ratings produce the highest scores among MSF sources. The second cultural variable examined was collectivism. Collectivism fosters cooperation and loyalty aimed at benefitting the social group. In contrast, Individualism fosters egocentric behaviors such as competition and self-promotion. The authors predicted that collectivism would impact MSF dynamics by resulting in people-oriented behaviors receiving the highest score in self assessments. Results confirmed the hypotheses. Unlike results of studies conducted in Individualistic, LPD cultures, subordinates provided the highest evaluations across feedback sources, peers provided the least discrepant source of information, and employees tend to overestimate their degree of concern for others and the effort they invest in nurturing relationships. The authors suggest that in HPD and collectivist settings, peer scores should receive higher weightings when generating a composite score based on MSF ratings. They also emphasize the importance of stressing anonymity in HPD settings in order to reduce the anxiety and uneasiness that subordinates may feel about rating their supervisors.
View full abstract/Get the article at:
http://www3.interscience.wiley.com/journal/120087713/abstract
Read more!
2009 SIOP Call for Papers
The call for papers for the 2009 SIOP Conference (April 2-4 in New Orleans) is now available. The deadline is September 5, 2008 @ 5PM EDT.
Read more!
2009 ATP Conference Call for Papers
Submissions are now being accepted for the 2009 Innovations in Assessment Conference to be held February 22-29, 2009 in Palm Springs, CA. Submission deadline is August 15, 2008.
Read more!
Update on eBay Sales of Protected Test Content
As an update to an earlier post, the Summer 2008 issue of the Test Publisher, the ATP newsletter, has the following: eBay Removes intellectual property from auction site in response to ATP member complaints "Six auctions down and a myriad to go," is how ATP's Legislative Counsel Alan Thiemann recently described the ongoing battle with eBay over test materials being sold online.
In response to growing concerns by its members over the presence of copyrighted tests being sold by various Internet sites, ATP has started working with the Software & Information Industry Association (SIIA) to expand the SIIA's existing enforcement for taking down unauthorized sales of copyrighted content.
Thiemann reported that "acting on information obtained from an ATP member, ATP/SIIA contacted eBay and within 24 hours six different auctions had been taken down." He added that, "as often occurs, one of the auctioneers challenged that action." Again working with the publisher, ATP/SIIA provided specific information demonstrating that the test at issue was a copyrighted product used commercially and that the attempted sale was neither licensed nor authorized and therefore was infringement.
Based on this successful "proof of concept" ATP is working with SIIA on how to best move forward in promoting a robust enforcement effort for the handling of piracy involving tests and related materials on the Internet.
"Although the success of the trial run proved promising, rolling out a full-blown, 24/7 enforcement program is going to be a significant undertaking," Thiemann cautioned. He added that as part of efforts so far, a standard email response template has been created which can be used to explain to sellers the legal arguments (copyright/trademark infringement, license agreement violations, etc.) that form the basis of enforcement actions against their auctions.
If any ATP member has an interest in this type of program and the potential savings for your company of joining an Association-lead effort to deal with Internet piracy, ATP would like to hear from you. Please contact ATP CEO Dr. William G. Harris at wgh.atp@att.net.
Read more!
Research Update
Summarizes and provides full links for the following research of interest:
- New SIOP journal - IO Psychology: Perspectives on Science and Practice
- Perceptual Speed Tests Account for Incremental Validity over GMA
- Using O*NET to Predict Occupational Literacy Scores
- Meta-analysis Exploring the Role of Personality in Turnover Decisions
- Comparing Personality Test Formats and Warnings
New SIOP Journal
If you are not already aware, the Society of Industrial and Organizational Psychology (SIOP) has a new journal available: Industrial and Organizational Psychology: Perspectives on Science and Practice. The first issue of this quarterly publication debuted in March 2008, produced by Wiley-Blackwell Publishing.
SIOP members receive the publication as part of their membership. For non-SIOP members articles from the first 2 issues are currently available for free at:
http://www.blackwell-synergy.com/loi/iops
The journal takes a focal article–peer commentary–response format. The March issue focuses on employee engagement, while the focal article in the June publication deals with issues surrounding the measurement of job performance. Definitely worth checking out.
Perceptual Speed Tests Account for Incremental Validity over GMA
In the spring issue of Personnel Psychology, Mount, Oh, and Burns examine the incremental validity of the specific aptitude, perceptual speed, over GMA in predicting job performance (task, citizenship behavior, and counterproductive behavior) in warehouse workers. Perceptual speed was measured as both the number correct (NC) - measuring the speed of processing, and the number wrong (NW) - measuring attention to detail/carefulness. They also examined whether conscientiousness accounted for incremental validity over GMA and perceptual speed. After controlling for demographics and GMA, NC on the perceptual speed test accounted for a 15% increase in the prediction of task performance. Of interest, NW accounted for substantial incremental validity (76% increase) in the prediction of counterproductive work behavior (defined as compliance with rules, policies, and regulations). Conscientiousness also predicted task performance and rule compliance after controlling for demographic characteristics, GMA and perceptual speed (both NC and NW). A primary practical implication of the results? Measures of perceptual speed, measured in terms of NC and NW (and conscientiousness – reinforcing what we already know) may add substantial utility to selection batteries for low complexity jobs.
View full abstract/Get the article at:
http://www.blackwell-synergy.com/doi/abs/10.1111/j.1744-6570.2008.00107.x
Using O*NET to Predict Occupational Literacy Scores
In the summer issue of Personnel Psychology, Lapolice and colleagues use O*NET job analysis information to effectively predict National Adult Literacy Survey (NALS) scores using job component validity. Results provide evidence for the construct validity of O*NET. The authors discuss their accumulated “database” of prose, document, and quantitative literacy requirements for all O*NET-SOC occupations and the potential utility of the information for career guidance or counseling, educational and policy settings.
This research references Scherbaum’s (2005) article that is worth a read for those interested in the concept of synthetic validity (past, present, and future!) and JCV.
View full abstract/Get the article at:
http://www.blackwell-synergy.com/doi/abs/10.1111/j.1744-6570.2008.00118.x
Meta-analysis Exploring the Role of Personality in Turnover Decisions
Also in the summer issue, Ryan Zimmerman explores the role of personality traits on turnover decisions in a meta-analysis of over 80 empirical studies. Results indicate that personality traits do have an impact on an individual’s turnover intentions and behavior. Emotional Stability showed the strongest (negative) relation with intention to quit, while agreeableness and conscientiousness had the strongest (negative) relations to actual turnover decisions. Zimmerman discusses the role of personality in the context of a dispositional path model of turnover. A test of the model showed that personality had direct effects on intentions to quit that were not explained through job satisfaction or job performance. For instance, individuals who are low on Emotional Stability may intend to quit for reasons other than dissatisfaction with their jobs or poor job performance. In terms of practical implications, the results suggest that personality tests can be used as part of the selection process to hire individuals who are less likely to leave the organization.
View full abstract/Get the article at:
http://www.blackwell-synergy.com/doi/abs/10.1111/j.1744-6570.2008.00115.x
Comparing Personality Test Formats and Warnings
In the June issue of the International Journal of Selection and Assessment (IJSA), Converse and colleagues examine how two promising approaches to reducing motivated distortion (forced choice response formats and explicit warnings against faking) influence criterion-related validity and test-taker reactions (cognitive and affective reactions). Validity and reactions were compared among students who took the Caliper Profile in either a forced choice or Likert-type format, and who received either an explicit warning against faking or no warning. Both FC and Likert scales showed incremental validity beyond cognitive ability in both warning and no warning conditions. No clear differences in validity emerged across the two response formats; however there was evidence indicating that FC formats and warnings may produce negative test-taker reactions (e.g., increased anxiety). Although the study uses a student sample, some interesting results emerge that may generalize to the selection context.
View full abstract/Get the article at:
http://www.blackwell-synergy.com/doi/abs/10.1111/j.1468-2389.2008.00420.x
Read more!
Summer Reading
Blogging has become an popular means of communication for both groups as well as individuals. A couple of other blogs which focus on issues related to I-O psychology and personnel selection are HR Matters and Selection Matters. Enjoy!
Read more!
5th Annual Survey on Online Screening and Assessment
Charles Handler of Rocket-Hire recently published his 5th Annual Survey on usage of online screening and assessment tools. Some of the highlights include the finding that there is continued growth in the use of a variety of screening and pre-employment assessments and that the implemention of applicant tracking systems is now quite common.
Read more!
Two Summer Conferences
The programs for the International Test Commission and IPMAAC are now available. Both meetings are likely to be of interest to I-O Division members and are still accepting registrations--ITC and IPMAAC.
Read more!
Article on Fairness and Validity of Cognitive Assessments
The May-June 2008 edition of American Psychologist has a great lead article by Paul Sackett, Matthew Borneman, and Brian Connelly entitled “High stakes testing in higher education and employment: Appraising the evidence for validity and fairness.” As the Abstract states: The authors review criticisms commonly leveled against cognitively loaded tests used for employment and higher education admissions decisions, with a focus on large-scale databases and meta-analytic evidence. They conclude that (a) tests of developed abilities are generally valid for their intended uses in predicting a wide variety of aspects of short-term and long-term academic and job performance, (b) validity is not an artifact of socioeconomic status, (c) coaching is not a major determinant of test performance, (d) tests do not generally exhibit bias by underpredicting the performance of minority group members, and (e) test-taking motivational mechanisms are not major determinants of test performance in these high-stakes settings.
Read more!
The Validity-Diversity Dilemma
A set of articles addressing the “Diversity-Validity Dilemma” was recently published in Personnel Psychology (see Spring Issue, Vol. 61). The articles summarize recent research and best practice designed to help organizations simultaneously enhance diversity, validity, and legal compliance in staffing.
Polyhart and Holtz present a number of selection strategies hypothesized to minimize group differences and compare and contrast their effectiveness.
Kravitz discusses the use of affirmative action to enhance representation of underrepresented groups. He cautions against the use of preferential forms and recommending nonpreferential approaches. Examples of strategies that are “most common or most likely to be effective” for increasing representation are also presented.
Pyburn, Polyhart, and Kravitz, provide an overview of the legal context that motivates and constrains the approaches discussed in the previous articles. They also summarize some key court decisions related to affirmative action preferences in the employment context.
The journal issue containing these articles is currently being promoted as a free sample. While this promotion period lasts, the articles are available for free at http://www.blackwell-synergy.com/toc/peps/61/1.
Read more!
Input Needed on "Model Guidelines" Revision
The Model Guidelines for Pre-employment Integrity Testing are in the process of revision. Initial ATP member input is needed by June 15, 2008. The following is from Jack Jones: To: ATP’s Industrial-Organizational Division Members
If you are an ATP member and need a copy of the Guidelines, please contact the I-O Division blog.
From: Jack Jones, Ph.D., Vangent, Inc.
Date: May 13, 2008
RE: “Model Guidelines” Update
I have been appointed by William G Harris, ATP’s Chief Executive Officer, to serve as the Chair of the Model Guidelines Revision Committee. I am being assisted by Dr. David W. Arnold who will serve as the Committee’s Chief Legal Advisor. Dave and I will be selecting a core Task Force to oversee this project, but we intend to provide all ATP Industrial-Organizational Division members the opportunity to participate in this revision process.
As you might know, the “Model Guidelines for Pre-employment Integrity Testing” were last updated in 1996. They were originally published in 1991. My ambitious goal is to have a copy of the revised guidelines ready for release at ATP’s Annual Conference in 2009, if not sooner. The key steps to this project include:
1. Phase 1: Needs Analysis - Surveying ATP members and special external contributors to assess their perspectives on what new sections and topics need to be addressed in the 2009 version of the Model Guidelines based on new types of integrity tests, technologies, market trends, laws, etc. that have evolved over the past decade. (In fact your first assignment, if interested, is to review the attached Model Guidelines and either send me a fax or email that provides your perspective on those topics/areas that need to be addressed in the updated version of the Model Guidelines. Please try to get me this information by June 15th at the latest and then Dave Arnold and I will compile a summary report for your review.)
2. Phase 2: Strategic Focus - The Task Force will focus on the most important topics that need to be addressed in the revised edition of the Model Guidelines. “Importance” will be defined as those business, professional, and legal topics that need to be effectively addressed to ensure the ongoing viability of pre-employment integrity testing as both a science and a practice. The results from the Phase 1 survey will be used to assist in this determination, as will consultations with leading experts in the field of pre-employment integrity testing.
3. Phase 3: Task Force Specialization – The Task Force members will be assigned a key section of the Model Guidelines and will be responsible for updating and revising the targeted section. Dave Arnold and I will then review and compile their work to establish a first draft of the revised guidelines. This draft will then be sent out to all key contributors for their feedback and then a 2nd draft will be quickly prepared.
4. Phase 4: Broader External Review – Special interest groups and associations will then be invited to provide feedback on the revised guidelines and will hopefully provide us with an endorsement along with approval to use their organization’s name in the document.
5. Phase 5: ATP Board Approval – The revised Model Guidelines will be submitted to ATP’s Board for final approval. Any final edits will be made at this time.
6. Phase 6: Marketing and Public Relations – Lauren Scheib will work with the Task Force to prepare press releases announcing both the purpose and the availability of these new guidelines.
I hope you embrace this project and contribute to the Phase 1 effort immediately. Remember we are looking for broader themes and macro-level topics in the Phase 1 survey. We will drill down to specific bullet points once we clearly understand what needs to be added/updated. Both my fax number and my e-mail are listed below. Please include all contact information when send a fax or email. Thanks in advance for your contributions!
Regards -
John W. “Jack” Jones, Ph.D.
Vice President & Chief Scientist
Human Capital Division
Vangent, Inc.
One North Dearborn St., Suite 1600
Chicago, IL 60602
O: 312-242-4422
M: 312-636-6912
F: 312-242-4210
http://www.vangent-hcm.com/
Jack.Jones@Vangent.com
Read more!
Economic Study of Impact of Pre-Employment Assessment
A recent paper (press release here) in the Quarterly Journal of Economics provides a very interesting analysis of the implementation of a pre-employment assessment program. Highlights include findings that the use of formal assessment in a selection process led to increases in productivity and job tenure with no change in the selection ratio between majority-minority job candidates. Some of the analyses are rather technical but the piece provides an enlightening alternative perspective on how economists analyze data related to selection.
Hat tip to Dave Arnold for the reference!
Read more!
In Defense of Validity Generalization...
Since Frank Schmidt and John Hunter published their classic paper in 1977, the concept of validity generalization has become almost universally accepted within the field of I-O psychology. However, there have been reports over the past several months that OFCCP has become increasingly resistant to validity arguments based upon meta-analytic validity generalization and has suggested that local validation studies be carried out when cognitive ability tests are used. In part, OFCCP's concerns seem to center around the fact that the notion of meta-analytic validity generalization does not appear in the Uniform Guidelines, which were drafted in 1978.
In an attempt to make the ATP membership and others in the I-O community aware of this issue, Jim Sharf of Employment Risk Advisors was kind enough to share two documents. The first is a letter that Jim wrote to the EEOC where he argues that the sections of the Uniform Guidelines which address "situational specificity" and "single group validity" are obsolete. The second is an open letter to OFCCP authored by David Copus, a well-known employment law attorney, who provides a very extensive review of the history of validity generalization and argues how the reliance of the Uniform Guidelines on the concept of "situational specificity" is dated and renders them inconsistent with current thinking in the field.
Read more!
EEOC Uniform Guidelines to Stay As Is
The EEOC has opted to leave the Uniform Guidelines as they currently are for at least the next three years. Specifically, the EEOC gave notice of its "intent to submit the Uniform Guidelines on Employee Selection Procedures (UGESP or Uniform Guidelines) without change, to the Office of Management and Budget (OMB) for a three-year approval under the Paperwork Reduction Act of 1995 (PRA)."
Additionally, the EEOC is not going to provide additional on the questions which were outlined in 2004 in relation to the definition of an "applicant." Specifically,EEOC does not intend to finalize the five additional Questions and Answers that include clarification of the definition of ‘‘applicant.’’ However, employers still must ensure that they are complying with the requirements of UGESP.
Read more!
Civil Service Testing in China
Here is an interesting article outlining the history of the civil service examination system in China. The author argues that this may be one of China's most important contributions. Unfortunately, but not surprisingly, it appears that disgruntled test takers and cheating were of concern at least 1000 years ago just as they are today.
Read more!
Dilbert the Inquisitor
The New York Times takes a humorous look at pre-employment testing and assessment...
Read more!
EEOC Proposed Rulemaking on ADEA
In an update courtesy of David Arnold, ATP General Counsel, the Equal Employment Opportunity Commission (EEOC) has issued a notice of proposed rulemaking to address issues related to the United States Supreme Court's decision in Smith v. City of Jackson. The Court ruled that disparate impact claims are cognizable under the Age Discrimination in Employment Act (ADEA) but that liability is precluded when the impact is attributable to a reasonable factor other than age. Current EEOC regulations interpret the ADEA as prohibiting an employment practice that has a disparate impact on individuals within the protected age group unless it is justified as a business necessity. EEOC is proposing a change in rules to accommodate this Court ruling.
Read more!
On the International Front...
Those interested in global issues related to testing may want to take a look at the International Testing Commission website. Information about the organization and its goals are outlined as well as is information about ITC 2008 International Conference to be held July 14-16, 2008 in Liverpool, UK.
Also, Eric Shepherd, immediate Past Chair of the European Division of ATP (E-ATP) has created a wiki with information about the division and its goals. (For those not familiar, a wiki is a collaborative workspace that enables members to both comment and update a web site's content.)
Read more!
On Faking...
The issue of job candidates attempting to present themselves in a favorable light has long been of concern to those working in the area of personnel selection. The issue is quite complex from both a theoretical and practical perspective.
For example, a recent study in Human Performance looked at the efficacy of warning applicants about faking on a personality inventory. 464 participants completed personality instruments in either a “warned” or “unwarned” condition. While subjects in the warned condition had lower mean scores on some scales, it did not increase the convergent validity between self and other ratings.
In thinking about job candidates “faking good,” it is also important to remember that job seekers will may attempt to present themselves favorably in selection tasks beyond psychometric assessment. For example, Julia Levashina and Michael Campion have been exploring the role of faking in employment interviews. In a recent study, they report that over 90% of undergraduate job candidates admit to attempting to present themselves in a way that enhances their attractiveness to employers.
Another interesting finding is that follow-up questioning increases faking. They write:We hypothesized the opposite effect by assuming that probing would be a response verification mechanism that would inhibit faking. Informal debriefing with participants revealed that follow-up questioning was perceived not as response verification but rather as a cue signaling what types of answers were important and critical.
Read more!
Thanks to All for a Successful Conference
Many thanks to all who attended the 2008 Conference in Dallas with special appreciation to those who presented as well as those who found time in their schedules to take part in the I-O Division Meeting. Conference presentations are now available on the Innovations in Testing website. Also, the 2009 ATP Conference will be held in Palm Springs, CA on February 22-25, 2009. The venue will be the Westin Mission Hills, location of the 2007 meeting.
Read more!
Interesting Read on Assessing Leaders and Leadership
Rob Kaiser, Bob Hogan and Craig Bartholomew have just published a provocative article entitled "Leadership and the fate of organizations" in the February-March 2008 issue of American Psychologist. One of their main arguments is that while the quality of leadership is a critical factor in shaping the success of an organization, most of the literature focuses upon the trajectory of a leader's career as opposed to the sucesss that he or she may experience in actually leading individuals, teams, and organizations.
Read more!
Steven Hunt Responds to Morgeson et al.
As might be expected, the original Morgeson et al. article has stirred quite a bit of interest. Due to limited space, Personnel Psychology was only able to print a limited number of responses. Steve Hunt of SuccessFactors also wrote an interesting reaction entitled Commentary on “Reconsidering the Use of Personality Tests in Personnel Selection Contexts”: Thoughts from a Practitioner Perspective that he was kind enough to permit us to post.
As space permits, we would also be interested in posting papers written by others on this as well as other topics that would be of interest to either the I-O Division or the larger I-O community. If you have something that you would like to share, please send it our way.
Read more!
I-O Division Input on Revised Testing Standards
As all are likely aware, the Standards for Educational and Psychological Testing are currently being revised. The ATP I-O Division has created a Task Force of volunteer members to provide input from both a publisher and professional perspective. The Task Force, chaired by Joseph Orban, will present on its findings at the upcoming ATP Conference during a session scheduled for Tuesday, March 4 @ 1:30 to 2:30 PM in the Ft. Worth 5 Room) Additionally, SIOP has posted the comments of its Task Force.
Read more!
Pearson Acquisition of Harcourt Assessment to be Approved
It appears that the Department of Justice will approve Pearson's acquisition of Harcourt Assessment contingent upon the divestiture of parts of its combined test catalog. Specifically, Harcourt's Adaptive Behavior Assessment System, Harcourt's Emotional Assessment System (currently in development) and either Pearson’s Comprehensive Assessment of Spoken Language and the Oral and Written Language Scales or Harcourt’s Clinical Evaluation of Language Fundamentals must be sold to a buyer approved by DOJ.
Read more!
eBay and Sales of Restricted Test Materials
The sale of restricted test materials on eBay has become the focus of increasing concern in the testing community for both publishers as well as clinicians. This is particularly acute in the case of clinically-oriented assessments because they often use printed materials (such as Rorschach plates and TAT cards) and physical stimuli (in the case of IQ kits) which can be sold easily. eBay has expressed recent resistance to prohibiting the sale of restricted test materials and will now only remove them from auction if it can be shown that their sale is harmful to the public. Aurelio Prifitera, Ph.D., President of The Psychological Corporation, posts his concerns and reactions to this.
Read more!
Outlook on Assessments for 2008
Charles Handler, an I-O psychologist and student of the employee selection arena, has posted his annual observations of industry trends. Amongst his observations are continued interest in the use of assessments and the integration of testing into the hiring process. A review of the comments which follow the article provide an interesting perspective on the objections that are often heard from individuals who reject the use of pre-employment assessments as well as some thoughtful observations and suggestions from David Arnold, ATP General Counsel.
Read more!
Using Personality Tests for Personnel Selection
Based upon a presentation at the SIOP Conference in 2004, Personnel Psychology (Autumn 2007 and Winter 2007) published a provocative series of articles on use of the personality assessments for personnel selection. As an initial salvo, Fred Morgeson and colleagues wrote an article quite critical of using personality-based assessments for selection, citing concerns with the impact of faking on self-report inventories and the low criterion-related validity estimates that are often found. As might be expected, this proved to be quite provocative and was picked up by a number of media outlets. Both this critique and the press reaction have also been of significant interest to ATP members.
The following issue of Personnel Psychology (Winter 2007) contained two replies disputing a number of assertions made in the paper (one by Robert Tett & Neil Christensen and another by Deniz Ones and others) plus a rejoinder from the original authors. This series of articles is a “must read” for anyone interested in the use of personality testing for selection purposes and is likely to be discussed and cited for many years to come.
Read more!
Previsor Acquires British Test Publisher
It was announced today that Previsor has acquired ASE, leading UK-based business test publisher.
Read more!
EEOC Announces Settlement with Ford
Ford Motor Co., along with two related companies and a national union, will pay $1.6 million and provide other remedial relief to a class of nearly 700 African Americans to settle a race discrimination lawsuit brought by the EEOC. The EEOC had charged that the written test used by Ford, Visteon and Automotive Components Holdings (ACH) to select employees an apprenticeship program had a disproportionately negative impact on African Americans.
Read more!
EEOC Posts Fact Sheet on Assessment and Selection
The EEOC recently posted a handy fact sheet that covers some of the common issues related to the federal anti-discrimination laws and the use of tests and other selection procedures in the employment process.
Read more!
Subscribe to:
Posts (Atom)