PDA Development and Validation

PDA is Validated by the American Institute of Business Psychology. The basis of the Personal Development Analysis (PDA) was initially developed in 1942 by William M. Marston. For years this assessment tool has undergone several revisions. Since the PDA has been, and is used, as a tool for human resources, this article summarizes the theories that underlie the development of the instrument and the research conducted to evaluate the PDA as a tool that adds value in the management of human talent.

INTRODUCTION

PDA is defined as a tool that assesses the behavioral style of people in working environments. The PDA form consists of a list of adjectives for open response. It has been developed based on an extensive study of the selection of "core words" from an initial list of over 3,000 words. Since its first release, this form has been revised and improved and there have been constant and diverse studies to validate this instrument which currently evaluates more than 25,000 people per year.

PDA is a solid and reliable tool, it is based on a structure of underlying theories and was developed for years based on studies which form its foundation, we understand that the main value of the instrument is a high degree of success through successful applications in recruitment and talent management, which strengthens the consistency and bases its "empirical validity". As is common and usually happens with other tools used in industry, PDA is usually judged by its body of theory, however at PDA International we believe that, while the body and theoretical basis are important, the real success of the PDA is and will remain its excellent and practical applicability for end user.

Theory

To develop the instrument Marston based it on the principles of perception, in his four-factor model described in his book “Emotions of Normal People,” (1928) and the theories of Self and Self- Consistency, by Prescott Lecky. In his studies he defines that under “normal” circumstance, a person has a predisposition to respond or behave in a certain way depending on how they perceive the nature of the situation, whether favorable or unfavorable, and the tendency of the individual to take action or retreat. These trends are what define and frame the four-factor model, described in Table 1. The four factors of the Marston’s model are: Dominance, Influence, Submission and Induction. Based on these 4 factors, PDA International developed its own model of four factors in order to update terminology, facilitate understanding and improve the applicability of the instrument. The new names defined by PDA International for these four factors are: Risk, Extroversion, Patience, and Norms. The definitions of these axes are described in Table 2.

A key element in the development of the model of personality by Marston was the theory of self, stated by Prescott Lecky (1945) in which he describes the concepts of social self and ideal self.Lecky said that during the first 15 to 18 years of life, and as a result of their experiences, people develop a relatively stable perception of themselves. Additionally, he also said that people develop an expectation of what the environment demands and requires of them. On this basis, Marston determined that individual behavior is determined in part by the interaction between perceptions of self and by adjustments to the demands of the environment.

Marston stated that people are born with a certain amount of energy, of activity, which provides the individual with the motivational stimulus of their behavior. This activity is similar to the Freudian concept of "psychic energy": it represents the force that powers simultaneously the physical and intellectual activity. People have different levels of activity that are relatively constant throughout their lives.

The structure of Marston’s Personality Model was built on three fundamental propositions. The first is that people perceive situations of the environment as favorable or unfavorable. The second is that the individual's response to any environmental stimulus will either be to confront or avoid. The third is that people have a certain amount of energy to act. Using these three basic variables, Marston developed a simple model to classify human behavior (Table 1). This model is based on the idea that people perceive any situation as favorable (non-threatening) or as unfavorable (threatening). It also assumes that people will confront or avoid situations, not remain neutral. So Marston identified four quadrants in this matrix. Each quadrant includes an independent set of behavioral tendencies. Marston stated that in normal situations people have a predisposition to behave in a certain way, depending on how they perceive the situation (favorable or unfavorable) and the tendency of the person to take action or retreat. These trends define the model of four quadrants.

In summary, Marston’s model of personality consists, intially, in a structure composed of four independent axes. We can represent or understand this structure as a geometrical sphere. The center of the sphere represents the individual at a level of "zero energy". The four areas emanating from the center of the sphere; the trend of behavior represented by the four axes that originate at the center of this area and go outward; each axis represents each one of the behavioral trends described in the model. Each of these axes is conceptually and statistically independent of one another. The length of the sum of these lines represents the ratio of activity of the individual.

Years later, based on experience gained through the application of the instrument, Marston identified the need to add a variable to the model he considered important. He defined the need to include the fifth axis, emotional self, interpreted as the level of self-discipline, emotional self-control and sense of social responsibility. (See Table 2). This fifth axis affects and influences the other four axes.

As was defined above, based on the application by Marston’s Theory of Self, and by Prescott Lecky, the observed behavior of a person is the result of: (1) the perception that the individual has of the environment and (2) the natural predisposition to behave according to certain patterns. Thus, in practice, the evaluation consisted of exposing an individual to review, on two occasions, a single list of 86 adjectives, one from the perspective of "their own perception of themself" and the other "according to how they understand their perceived environment. "

Measurement: PDA Analysis Form

Based on the afore-mentioned it is proposed, to evaluate the natural predisposition of individuals to respond to situations and others (as determined in the four-axis model), the format of self- assessment based on a list of adjectives. Thus, the instrument developed here is based on self- assessment, of free response, which may be applied to paper and pencil as well as "online", today known as Personal Development Analysis (PDA). The form currently consists of 4 simple steps described below:

1st) Personal Information
2nd and 3rd) Identical lists of 86 descriptive adjectives
4th) A space to provide the individual the possibility of sharing an additional self-description.

While the form is composed of these 4 stages, the core of the PDA is in the 2nd and 3rd stages. In the 2nd stage the individual must read the list of adjectives and mark those adjectives, according to their understanding, on how they believe they perceive their environment. The instructions are: "Please read the following list and tick each word that others would use to describe you. Remember to mark all of the words with which you understand others would describe you. People say I am a person ..." In the 3rd revision, as a self-evaluation, the individual evaluated shall select and mark those adjectives that they believe describe themselves. The instructions are: "Now, please read the following list and tick each word that you think that describes you. Remember to mark all the words that describe you understand describe you. I'm actually a person ... " The first list of responses relate information as the individual understands how they are perceived by others, the second list concerns information as to how the person perceives and sees themself.

The Scales of PDA

Initially, the results were processed on the basis of extensive and complex forms, however nowadays the results of the forms completed by the individuals are processed by the computer system developed by PDA International. Each of the 86 words on the list is assigned to one of the 4 axes. Thus the system, according to the words selected by the individual, identifies trends and distributes power to each of the axes. The system performs this equation for both the Self Perception Profile (Natural) and also for the environment (Adapted Profile). In the score, the number of words marked for each axis is calculated for the Natural profile and the Adapted Profile. Hence arise the "raw values," which are then converted into "C-values (converted)." C-values have been developed from a normative sample obtained from several hundred participants evaluated in the standardization study. So the C-values are calculated independently of each of the ten Axes (A-1 to A-5 in the Natural and Adapted). The activity, now known as Energy Level, is computed based on the sum of the words selected for Axes 1 to 4 (does not include the words selected for Axis 5). To get the score of an individual, PDA computes (1) the "raw values", (2) "C-values" and (3) the "ipsative line" (* 1). The system delivers the result as a PDA Graphic , shown in Figure 1.

(*1): Ipsative scores refer to the individual's score compared with their own average score and not a standard or external score.

The PDA Graphic gives us a characteristic pattern or "behavioral profile" that represents and reflects the individual's score on each of the four axes. This PDA Graphic is interpreted by analysts who have been trained and certified in the proper use of the PDA instrument. Distributed there in the PDA Graphic (Figure 1) are other scales and indicators observed to achieve fine and detailed interpretations. Some of these scales and their definitions are explained in Table 3, and other variables and scales are explained in detail in the PDA Analyst Certification Course.

The Development of the PDA Analysis and its Revisions

The first version of the PDA Form was developed in 2003. It was built on a base of 110 adjectives. With the first PDA form, known as "Form A", while it was effective in several respects, it did have some problems. One was that every word could score on one or more of the four axes, generating some confusion. Additionally, the initial word list contained some words that could have been misinterpreted as disparaging. Finally, further research revealed that some of the words on Form A were poorly mapped, so they were reassigned to other axes. Throughout this process the main obstacle was to find the words that were not ambiguously related with axes 3 and 4. Once these problems were corrected the current Form B was put into operation. Already in this final version each word is assigned to a single axis, the words that could be perceived as derogatory or disparaging were replaced and finally, those words whose relationship to an axis might be questioned were deleted or associated with another axis. The adjectives were selected so that the four primary axes were mutually independent and each word is directly related to only one of these four primary axes. Thus, the number of words to assess each axis was standardized between 16 and 18 words. (Table 4)

Another significant change between Form A and Form B was the development of a set of adjectives exclusively assigned to the 5th axis of Self-Control. In the first version the adjectives related to this 5th axis were also affecting the other axes. In this latest version, as with the other four axes, a separate list is defined for the fifth axis.

There was also a change in the scoring of the Energy Level. In the first version the Energy Level score came from the sum of all the words marked by the individual assessed. We understood that this process, defined in this way, gave redundant information, "in the theoretical model of human behavior and motivation behind the PDA, the energy level is understood as the totality of the available power or energy used (for an individual) in the manifestation of behaviors represented by the four primary axes." The fifth axis is a parallel axis, resulting, for which both theory and experience suggest, that it should not enter the power level measurement. Thus, in the latest version, Form B, the energy level is only measured by those items that are used to score the four primary axes.

STUDIES ON RELIABILITY

We refer to reliability as the consistency or stability of the scores, if the reliability analysis provides estimates of the stability of scores over time or other variables. Tables 1 and 2 summarize the reliability of PDA in its final version (Form B). Table 5 shows that the basic scales of PDA are relatively stable over a period of at least three months, and also describes the consistency of the profile patterns of individuals in the testing and re-testing of PDA. This estimate was determined by correlating the profile patterns formed by the four basic axes in both the first and second administration of the PDA natural profile and the adapted profile. Obtaining results over 77% confirm that the PDA measurements are stable over time.

As well, the "Alpha Coefficient" was tested to verify the internal consistency of PDA’s scales. The results of this study are shown in Table 2. This study was conducted including all five axes of PDA for both the Natural and Adapted profiles. These estimates give an average of 75%, indicating that the words assigned to each axis tend to measure the same construct, thus confirming that the internal scales are consistent.

STUDIES ON VALIDITY

Validity refers to the extent that a test measures what it claims to measure. Validity is a concept of "all or nothing." It is understood as a matter of relative strength and focuses on the credibility of the interpretations of test scores. The interpretation of a test can be valid for one use but not for another. Assessing the validity requires driving both field studies and laboratory studies. To date, research on PDA can be divided into several categories: studies of structure validity, studies of criterion validity as well as other studies.

Construct Validity

Construct validity is a broad term; it is the demonstration that the test itself measures the construct it claims to measure. It must be consistent with the psychological theory that serves as the basis of the test. There are two levels of construct validity. The first, internal validity is an assessment of the extent to which scores from the test are consistent with the fundamental model on which the test was developed. The second level, external validity is an assessment of the extent to which the behaviors predicted by test scores are consistent with the behavior in real life. Of all the categories of validation, external validation is the most powerful and important, as it measures how well the test predicts present and future behavior or results compared with other measurements. Because PDA is most often used for staff appraisals and evaluations of the behavior of individuals in real-life situations, we understand that it is vital that the results of each of its scales have a high degree of external validity.

Internal Validity

Although the demonstration of an evidence-based foundation may be sufficient to support the PDA, research has been conducted to develop the PDA so its current form is consistent with the PDA model. One of the key features of this model is that the four primary axes are statistically independent. We assume that all four are one-dimensional axes with a common origin: the center of the sphere. Visualize the four axes emanating from the center of the sphere. This means that the Axis-1 score should be independent of the scores of the other 3 axes. It is based on the assumption that the four axes are independent of each other and specific words were defined for each axis. From this change the representation of this relationship in terms of correlation showed statistical independence of the four axes.

Other features of the PDA were studied to assess the internal validity of the instrument. We studied the differences between the application of a form with open response and a forced response. This study demonstrated that those being evaluated widely preferred the open response format. The results of this study showed that the forced response formats used with lists of adjectives generated discomfort with those being evaluated and delivered very high percentages of invalid results.

One of the key assumptions of PDA is the concept that the social self of individuals differs from their ideal self. The studies showed that people can make systematic distinctions in describing others. This is an important item to keep in mind for the basis of the PDA as the PDA requires people to designate words that "have been used to describe them" and words that they "honestly believe describe them." If people cannot make this distinction the system itself may be questionable. In short, there is strong support for the internal validity of PDA with respect to the theoretical model. First, the relationships between the four axes closely resemble the relationships of the underlying theory. Second, the format of the list of adjectives for open response is received and accepted by those being evaluated. Finally, we conclude that PDA can distinguish between social self and ideal self. These three preconditions must be met for the PDA to be consistent with the theory behind it.

External Validity

External validity studies have investigated the relationship between behavioral descriptions derived from other tests, PDA scores and measures of behavior in real life. We performed a Student T-test to compare each of the 5 axes between managers and workers and the results are summarized in Table 3. The result was that for each of the axes where the value of T was less than alpha (0.05), the null hypothesis is therefore rejected, and this means that there are significant differences between the scores of managers and workers. As well, the self-descriptions made by the managers differed markedly from those made by the workers. We conclude from this study that the list of adjectives on the PDA is able to identify between the self-descriptions of individuals who occupy positions which differ substantially from one another.

Results of subsequent studies also showed that there is a significant relationship between PDA axes scores and comparable constructs measured by other instruments. Both PDA and DiSC were administered. We selected a panel of three judges and 108 people were evaluated with both PDA and DiSC. The judges surveyed additional information obtained through group and individual interviews with these 108 people. CVs and performance data were observed. With all this information the panel had a thorough knowledge of these people. This information was then compared for these people between the information obtained through PDA and the info obtained through DiSC. The results of this study are shown in Table 4.

The result of this study was an average of 71% for PDA and an average of 56% for DiSC. Additionally, a T-Test was performed between the two groups, obtaining a t <0.01, and therefore we conclude that although we find that there is a significant relationship between the scores of both tools (more than 50%) the PDA is a more precise tool for measuring behavioral profile than DiSC.

We also considered the degree to which the descriptions of the personality test that arose from the interpretations of the PDA profiles fit the descriptions of a person being evaluated by non-test criteria. On several occasions we invited people to complete the PDA form and, additionally, without receiving the results yet, we invited them to make a self-description and narrative summary of about a page answering the question "Who am I?" From there, other consultants trained and certified in PDA, who had not seen or had contact with people evaluated, carefully read these self descriptions and rated the estimated profile pattern describing the combination of the four primary axes for each self description . Later the forms were then processed and compared to the scores. The average correlation between the actual patterns and profiles was estimated at =. 84. From this we conclude that there was a high degree of congruence between the personality descriptions "estimated" by PDA and those performed by the standard process. For years there were several such studies continuing to prove the profile patterns of PDA are descriptively valid and are not linked solely to the verification of self-descriptive adjectives.

Criterion Validity

Criterion validity is the demonstration of the extent to which a test instrument (an instrument of prediction or forecasting) is related to performance in the position (criterion). It is particularly important in business environments, as it demonstrates the "relationship to the position" and the efficiency of a predictive assessment tool that is used as part of a selection process or the development of human capital. There are basically two methods to conduct a study of criterion validity. First, the concurrent method: Here we test a group of people who have been in one position for a period of time. It also measures performance, efficiency, effectiveness and results, both results (PDA Profile and the measure of their performance) are compared and correlated.

The second process to conduct a criterion validity study is the predictive validation method. In this trial, the PDA was administered to all applicants for a position and all those who completed the test were hired without taking into account the results of the test. These results were saved and were not seen by anyone. The subsequent performance of each of the individuals in the sample is measured by one or more independent measures of success. This measure of success is then correlated with the initial results of the completed PDAs.

Concurrent Validation Studies

There have been some concurrent validation studies observing the relationship between PDA and performance at work. We conducted previous studies and defined a PDA profile based on the characteristics that we considered typical of entrepreneurs and presidents. Then we conducted the PDA with 154 entrepreneurs who were already in their own companies and found that the profile patterns of these entrepreneurs greatly resembled the default PDA profile for entrepreneurs. The president entrepreneurs had a personality profile highly fitted to the hypothetically defined profile (100-75-0-25). This pattern proved to be ideal for entrepreneurs. Thereafter we concluded that the successful entrepreneurs in this study possess, as a group, mostly those behavioral characteristics that theoretically (from the PDA Model) are ideal for this position.

Another study showed statistically that PDA is significantly correlated with turnover rates among workers who work at tollbooths. Of the 122 workers hired, only 61 were hired as a condition of their high level of correlation. The other 61 remaining workers were hired to cover other requirements of the position, not taking into account the low level of correlation with predefined behavioral requirements for the position. A 6-month rotation was measured in both groups. The rotation of the first group, for which the high correlation had been an exclusive requirement, measured 17%, the second group measured 53%. With these results we observed a statistically significant correlation predicting the rotation applying PDA in the selection process.

In another study, involving staff from insurance companies, PDA was administered to 19 Account Managers. Supervisors who had no knowledge of the results of the PDA revealed performance data of these Account Managers. Then, once all the information was revealed, cross correlations were performed on both data (PDA's and Performance Results). The results of this study are expressed in Table 5. We performed a Student T-Test and obtained a t <0.01, therefore the null hypothesis is rejected, meaning there are significant differences between the correlations of the Account Managers with greater performance and lesser performance. It was also noted that the average correlation between each person and the job was 70% for higher performance as opposed to 36% for low performance, concluding that the Account Managers with the highest percentage of correlation with the position, on average, get better performance in sales. Results are summarized in Table 5.

Predictive Validation

In one of the most recent validation studies on the predictive profiles of PDA we compared the profiles of life insurance agents based on their success after 3 years of working in their position. All participants involved in this study were recruited using the PDA as an evaluation tool. Three years after joining the company each of these agents were assigned to one of two groups, successful or unsuccessful. The approach was to have achieved sales targets, have advanced to positions of supervision or management or have left the company to become successful managers, agents or other companies. Everyone who did not meet this criteria was classified as "unsuccessful". Deviation scores of PDA were computed in both groups and by averaging the scores of each of the four primary axes. Then an average was calculated individually for each axis. Table 6 describes the results of comparisons of the PDA axis of the successful vs. unsuccessful. We concluded that this study shows that successful agents in this study have significantly higher scores on Axes 1 and 2 and significantly lower scores on Axes 3 and 4. This differential pattern is consistent with the hypothetical "best" profile for life insurance agent salesmen.

A Study of restaurant managers that were learning their jobs reported an analysis of compatibility coefficients for successful and unsuccessful restaurant managers. Compatibility coefficients is the correlation between an individual’s PDA pattern shape an the profile that is considered to be ideal for the job as determined independently by a job analysis. In this analysis, and without reference to the PDA profiles of any individuals, management personnel determined the ideal job profile through focus Group discussions led by a trained job analyst. Based on these discussions, an PDA profile for the ideal restaurant manager was developed.

The results shown by this study indicated that those managers whose performance was satisfactory at the time they stopped working (voluntarily) had, at time of hire, half the level of compatibility coefficient of + .53 with the ideal profile of a restaurant manager, while those who were involuntarily separated from the position had at time of hire, an average compatibility coefficient of - . 17. The statistical test of the difference between the average of these two coefficients of compatibility of these two groups was p <.05. The study report concludes that "as a group, managers whose profiles are more compatible (with the ideal profile for the position) have done better and earned better wages, despite having held the position for significantly less time on average, than those whose profiles are less compatible and have earned a lower annual wage. "

Cross-Validation Study

Cross-validation studies are, statistically, the most powerful demonstrations of the predictive potential of a test. Here an assessment tool, such as PDA, is validated on an initial sample through a predictive validity study. The findings of this study are used to make predictions about a new and completely independent sample. In this second sample, the predictions of the PDA are later compared with the results of the positions. Applying this approach, an independent sample of 52 life insurance agents was studied to cross-validate a selection profile that originally had been validated with a sample of 55 workers drawn from the same company. The prediction set included the PDA and a set of five variables of personal history. The classifying criterion was the measure of a successful or unsuccessful outcome for a period of three years. In the results, PDA was significantly related to the results of the sample and successfully predicted which agents would be successful and which would be unsuccessful.

Demographic Studies

Since PDA is a test that uses a list of adjectives for self-assessment as its main element, there was some concern that the predominantly high verbal instrument can affect the average values and differ across different groups depending on the race, age or gender. There were several studies regarding this aspect. 108 participants were recruited. The breakdown by ethnicity is shown in Table 8.

Race

Table 7 summarizes the data obtained with Form B and shows the means on the basis of ethnic groups. We performed the ANOVA summary, which is a statistic that indicates whether or not groups belong to the same population. This statistic was performed for each of the axes of the Natural profile and as a result, in all cases P> 0.05 and F < F critical, therefore we accept the null hypothesis indicating they do not identify significant differences between the different ethnic groups for any of the axes.

Age

The same form has been tested on any differences in relation to age. Table 8 summarizes the data from Form B and shows the means for people of different ages. The ranges studied were under 20, between 20 and 40, between 40 and 60 and over 60. ANOVA summary was performed, which is a statistic that indicates whether or not groups belong to the same population. This statistic was performed for each of the axes of the Natural profile, and as a result, in all cases P> 0.05 and F < F critical, therefore we accept the null hypothesis indicating no significant differences between the different ages for any of the axes.

Gender

The results of studies based on Form B of the PDA are shown in Table 9. Table 9 summarizes the data obtained from Form B and shows the means on the basis of various genders. We performed the T-Student test which is a statistic that indicates whether or not groups belong to the same population. This statistic was performed for each of the axes of the Natural profile and as a result, in all cases T> 0.05 and we therefore accept the null hypothesis indicating no significant differences between the different genders for any of the axes.

SUMMARY AND CONCLUSIONS

Studies related to the validity of the PDA began during its early development stage and updates have continued through till the latest version (Form B). We understand that in this review we have summarized the most relevant selections of the core of our research on the reliability and validity of the PDA. Taken as a whole, there is strong evidence to support the implementation of this tool in business and industrial processes for selecting and managing people.