Factorial Invariance of the Generic Job Satisfaction Scale in Argentina and Brazil
Invarianza factorial de la escala de satisfacción laboral genérica en Argentina y Brasil
Invariância fatorial da escala de satisfação laboral genérica na Argentina e no Brasil
Solana Salessi, Alexsandro Luiz de Andrade, Alicia Omar
Factorial Invariance of the Generic Job Satisfaction Scale in Argentina and Brazil
Avances en Psicología Latinoamericana, vol. 38, no. 2, 2020
Universidad del Rosario
Solana Salessi * firstname.lastname@example.org
Centro de Investigación y Transferencia de Doble Dependencia Universidad Nacional de Rafaela y CONICET, Argentina
Alexsandro Luiz de Andrade **
Universidade Federal do Espírito Santo, Brasil
Alicia Omar ***
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), Argentina
Date received: 06 March 2019
Date accepted: 08 December 2019
Abstract: The aim of the present study was to analyse the factorial invariance of the Mac Donald and Mac Intyre’s Generic Job Satisfaction Scale in Argentina and Brazil. Researchers developed an instrumental cross-sectional study on a non-probabilistic sample of workers (nArgentina = 663, nBrazil = 672). The results indicate that the scale is invariant at configural (SBχ2/gl = 2.06, GFI = .96, CFI = .97, RMSEA = .03), metric (δGFI = .009, δCFI = .008, δRMSEA = .006), and scalar levels (δGFI = .007, δCFI = .009, δRMSEA = .004); whereas, partially invariant at a strict level. In turn, the comparison of latent and observed means shows that Brazilian participants are more satisfied with their jobs (MBrazil = 3.86, SDBrazil = .91, MArgentina = 3.52, SDArgentina = .90, t = 8.54, p < .000; CR = 11.062, p <.000), although it is in small effects (d = .37; r = .18). The results obtained indicate that the scale is adjusted to the model of strict factorial invariance, except for three of its items that only reached strong invariance.
Keywords: Job satisfaction, scale, multi-group factor analysis, factorial invariance, cross-cultural study.
Resumen: El objetivo del presente estudio fue analizar la invarianza factorial de la Escala de Satisfacción Laboral Genérica de Mac Donald y Mac Intyre, en Argentina y Brasil. Se desarrolló un estudio instrumental de corte transversal sobre una muestra no probabilística de trabajadores (nArgentina= 663; nBrasil= 672). Los resultados indican que la escala es invariante a nivel configural (SBχ2/gl= 2.06, GFI= .96, CFI= .97, RMSEA= .03), métrico (δGFI=.009, δCFI= .008, δRMSEA= .006) y escalar (δGFI=.007, δCFI= .009, δRMSEA= .004), mientras que es parcialmente invariante, a nivel estricto. A su vez, la comparación de las medias latentes y observadas muestra que los participantes brasileros se encuentran más satisfechos con su trabajo (MBrasil= 3.86, DEBrasil = .91; MArgentina = 3.52, DEArgentina = .90; t= 8.54, p < .000; CR= 11. 062, p < .000), aunque se trata de efectos pequeños (d= .37; r= .18). Los resultados obtenidos indican que la escala se ajusta al modelo de invariancia factorial estricta para todos los ítems, a excepción de tres de ellos que solo alcanzaron invariancia fuerte.
Palabras clave: satisfacción laboral, escala, análisis factorial multi-grupo, invarianza factorial, estudio intercultural.
Resumo: O objetivo do presente estudo foi analisar a invariância fatorial da Escala de Satisfação no Trabalho Genérica de Mac Donald e Mac Intyre, na Argentina e no Brasil. Se desenvolveu um estudo instrumental de corte transversal sobre uma amostra não probabilística de trabalhadores (nArgentina = 663; nBrasil= 672). Os resultados indicam que a escala é invariante nível a configurar (SBχ2/gl= 2.06, GFI= .96, CFI= .97, RMSEA= .03), métrico (δGFI=.009, δCFI= .008, δRMSEA= .006) e escalar (δGFI=.007, δCFI=.009, δRMSEA= .004); enquanto, parcialmente invariante a nível rigoroso. Ao mesmo tempo, a comparação das médias latentes e observadas mostra que os participantes brasileiros se encontram mais satisfeitos com seu trabalho (MBrasil= 3.86, DEBrasil = .91; MArgentina = 3.52, DEArgentina= .90; t= 8.54, p < .000; CR= 11. 062, p < .000), ainda que se trata de efeitos pequenos (d= .37; r= .18). Os resultados obtidos indicam que a escala se ajusta ao modelo de invariância fatorial rigorosa para todos os itens; com exceção de três deles que só alcançam invariância forte.
Palavras-chave: satisfação no trabalho, escala, análise fatorial multigrupo, invariância fatorial, estudo intercultural.
Since the beginning of the twentieth century, the study of job satisfaction (JS) has become a central theme of organizational psychology (Judge, Weiss, Kammeyer-Mueller, & Hulin, 2017). The interest from academics and researchers to unravel the nature, determinants, and consequences of JS has its counterpart in the concern of managers and organizational leaders to know its impact on performance and business success. The current trend is to consider JS as an attitude towards work experiences, a conceptualization that has consensus among specialists, after having been considered, alternatively, as a logical and rational evaluation, or as an emotional response to work experiences (Salessi, 2014).
JS recognizes among its antecedents both dispositional and contextual variables. The analysis of the dispositional sources of JS has a rich and varied history. These studies have shown that traits such as self-esteem, locus of control, general self-efficacy, and emotional stability are conceptual and empirically relevant for JS (Judge et al., 2017). The literature review also shows links between JS and some sociodemographic variables such as age, sex, marital status, and tenure. The evidence indicates that the elderly, women, married workers, and those who report more tenure are the most satisfied with their work (Salessi, 2014). Among the situational variables, the empirical evidence indicates as main antecedents (Keller, & Semmer, 2013) the variables related to the design of the position (autonomy, meaning of the task, diversity of competencies required for its execution, etc.), as well as organizational practices (for example, work flexibility, support for work-family balance, etc.).
The importance of JS lies in its beneficial effects for both the employee and the organization. That is, just as the organization can influence employee satisfaction through fair wage policies, training to increase productivity, rewards for outstanding executions, and the like; satisfied employees can contribute to reliable, responsible, and quality work, which reduces the internal costs of any production process (Sanín-Posada & Salanova-Soria, 2014). It has been emphasized (Zito et al., 2018) that JS increases psychological well-being and reduces intentions to change jobs, as well as psychosomatic illnesses, stress, and anxiety. Likewise, it has been pointed out (Dou, Nie, Wang, & Liu, 2016) that JS has a positive influence on satisfaction with life in general, and that personal satisfaction makes people work more efficiently and effectively.
Parallelly, in recent years, differences in JS between countries have begun to be analysed (Bentley, Coates, Dobson, Goedegebuure, & Meek, 2012). In this regard, it has been hypothesized that the perceptions of JS would be impregnated both by the tendency of the members of a society to value positive affect and have a positive vision of life, as well as by the cultural patterns of socialization in accepting and identifying the positive affect from the experiences lived. Within this new line of research, researchers have begun to analyse the invariance of the instruments to measure JS among distinct cultural groups. That is, the current concern is to have instruments that guarantee that the construct being evaluated (in this case, JS) has the same meaning for the groups compared.
The measurement of JS
Among the instruments available to measure JS, the Generic Job Satisfaction Scale (Mac Donald & Mac Intyre, 1997) stands out because it is a tool that can be administered in the same workplace and in a wide range of occupations. Unlike the scales that measure satisfaction as the average of the scores obtained in multiple aspects of the work, each one independent of the others (multidimensional approach), this scale explores overall attitude towards work, providing a generic assessment of the satisfaction of the worker (uni-dimensional approach). Both, the uni and the multidimensional approach have characterized the development of instruments to measure attitudes (Judge & Kammeyer-Mueller, 2012), although in the specific case of JS the multidimensional scales have been strongly criticized because they presuppose that the facets of JS are combined in a linear and additive way (Dalal & Credé, 2013) and, what is worse, because there is still no consensus regarding which are the relevant facets that make up the construct (Salessi, 2014).
Unlike the multidimensional scales that measure JS through 50 or more items, the Generic Job Satisfaction Scale, developed in the framework of the inu-dimensional perspective, provides valid and reliable scores with only ten of them. Initial validation of this instrument (Mac Donald & Mac Intyre, 1997) was conducted on a sample of 885 Canadian workers, representatives of numerous occupations and organizations, in the city of Ontario. Exploratory factor analyses indicated retain 10 of the 44 items elaborated, which showed acceptable internal consistency (α = .77). Correlational analyses, meanwhile, confirmed the construct validity based on the significant associations found with variables such as work stress and life satisfaction. So far, the instrument has been adapted for use in some European and Asian countries (Bai, Kwok, Chan, & Ho, 2013; Van Saane, Sluiter, Verbeek, & Frings-Dresen, 2003). In Latin America, there are already adaptations made in Argentina (Salessi & Omar, 2016) and in Brazil (De Andrade, Omar, & Salessi, In Press).
The Argentine validation (Salessi & Omar, 2016) was carried out with a multi-occupational sample of 328 workers. Exploratory and confirmatory factor analyses showed a uni-dimensional solution with good fit indices. The model’s integrations consist of only seven items. Three items were removed for presenting low factor loadings, namely: “I get along well with my bosses and supervisors”; “The company cares about me”, and “I think working is good for my health”. The reliability of the instrument reached satisfactory values, demonstrating that the seven items contribute significantly to measuring the construct.
The Brazilian validation (De Andrade, Omar, & Salessi, In Press), meanwhile, was carried out with a sample of 681 workers from the southeast of Brazil. The exploratory factor analysis indicated the adequacy of the scale to a uni-dimensional structure, which was confirmed by confirmatory factor analysis. This model was also composed of seven items since three had to be eliminated due to low factorial loads or redundancy due to the overlapping contents. Coincidentally, the items eliminated were the same from the Argentine validation. The reliability of the instrument was within the values recommended by international standards.
Factorial Invariance Analysis
Factorial invariance refers to the process by which it is verified that the psychometric properties of the instruments (or their items) are independent of the characteristics of the groups under study, but not of the construct that the instrument measures (Byrne, 2008). It is the mechanism, which guarantees that the construct evaluated (in this case, JS) has the same meaning in the groups compared. In other words, the factorial invariance ensures that the evaluated construct presents a similar meaning for the compared groups (Caycho, 2017).
Demonstrating the invariance of an instrument is crucial to compare the scores in samples from different countries. In practice, a researcher can compare the results of a test applied in different contexts when it shows that the values of the observed variables depend on the values obtained in the latent variable, independently of the groups studied (Wu, Li, & Zumbo, 2007). That is, an instrument is invariable when there are no biases in the scores due to the existence of different patterns of relations between its items and the latent variables (Elosua, 2005).
According to the recommendations of the specialists, the factorial invariance of an instrument is progressively evaluated (Elousa, 2005) through the verification of different levels, namely: configurational invariance, weak (or metric) invariance, strong (or scalar) invariance, and strict invariance. By configurational invariance, it is understood that people of different compared groups use the same conceptual framework to respond to the items, a mechanism that contributes for each item to saturate on the same factor (Cheung & Rensvold, 2002). The configurational invariance requires that the latent variables are specified by the same manifest variables in the compared groups. By weak invariance, it is understood that the factor loadings of the items in each of the latent variables are equal in the groups compared. That is, in each group under study, the change of a unit in an item produces the same change in the factor with which it is associated (Wu et al., 2007). The strong invariance refers to the fact that both the factorial loading and the intercepts are the same in the groups compared. Strict invariance is the last step in the progressive analysis of factorial invariance. This type of invariance requires not only that factorial loads and intercepts be equal in the samples compared, but also the variances and covariances of the errors. The strict invariance shows that the instrument measures the construct in the same way in the groups considered.
The aim of this study was to analyze whether both Argentine and Brazilian workers interpret and respond similarly to items of the JS scale. That is, if the factor structure underlying the JS construct (as it has been operationalized on the scale developed by Mac Donald and Mac Intyre) remains invariant in the two groups considered or if the attributed cultural differences between the two countries have a negative impact on the invariance of the instrument. In this regard, it should be remembered that previous cross-cultural studies (Omar et al., 2007) have highlighted some idiosyncratic notes of each country, contrasting the individualism and fatalism of Argentines to a more collectivist and optimistic view of life by Brazilians. Asymmetries and contrasts that could be explained by their different cultural roots and socialization processes. In this sense, for example, it has been reported (Omar, Uribe, Assmar, Ferreira, & Terrones, 2000) that, in the face of the causes of success/failure, Brazilians make more external attributions, and explain success as a result of stability of the conditions of the environment, the mood, and the degree of difficulty of the task. These are self-protective mechanisms, own of a socialization characterized by expectations of control, influence, and power. Argentines, meanwhile, assign a greater role to internal causes and consider that success depends on the effort, dedication, and, to a lesser extent, the help of others more powerful. That is, mechanisms that rest in the process of socialization grounded on needs of achievement, self-realization, and affiliation.
In the face of such inter-country differences, it was considered appropriate to verify the factor invariance of the instrument. Having this type of information would allow implementing programs and making more effective organizational decisions since, despite the geographical proximity and the increase of economic collaboration between Brazil and Argentina, there are very few studies that have compared the perceptions of JS in both countries. In this sense, the review of the bibliography shows that in a cross-cultural study, focusing career dimensions and satisfaction with life, stronger relationships were observed between life and career satisfaction for Argentines than Brazilians (Punnett, Greenidge, & Ramsey, 2007). While in other cross-cultural research on stress and JS in academic professionals, it was observed that the Brazilian and Argentine participants reported less stress and less JS compared with professionals from Japan, Canada, Netherlands, Finland, and Korea (Shin & Jung, 2014).
The present work belongs to the category of instrumental study because its orientation was to the analysis of the psychometric properties of a measuring instrument (Ato, López, & Benavente, 2013).
Participants were 1350 Argentine and Brazilian workers, selected in a non-probabilistic manner. The description of each national sample is below.
Argentine sample: It was integrated (by availability) by 669 workers (398 women and 271 men) from different public and private organizations located in Rosario and Gran Rosario. The average age was 34.48 years (SD = 10.57), while the average tenure was 8.50 years (SD = 8.50). Fifty-two percent of the participants were married or living with a partner, and 61% had completed higher education (tertiary or university). The 25.57% of the sample occupied medium-high positions in their respective organizations, while 75.43% of the sample worked in organizations belonging to the private sector. The distribution by activity branches was 31.72% commerce and services; 25.89% industry; 22.04% education, and 20.35% health. On average, participants worked 42 hours per week (SD= 5.40).
Brazilian sample: From a sampling by availabil ity, the sample was integrated of 681 workers (429 men and 252 women) from different organizations based in Vitoria, and Rio de Janeiro. The average age was 34.20 years (SD= 8.50), and the average tenure was 9.02 years (SD = 9.30). Fifty-eight percent of the participants were married or living with their partner. Forty-two percent had completed higher education (tertiary or university), and 57% worked in private sector organizations. The distribution by activity categories was as follows: 39% commerce and services; 27% industry; 24% health, and 10% education. On average, participants worked 43 hours per week (SD= 6.50).
During the year 2017, several Brazilian and Argentine organizations were contacted and invited to participate in the study. In order to collect data, researchers agreed on days, schedules, and other conditions with the organizations that consent to participate in the investigation. The participation of workers was not compulsive, and no incentives of any kind were granted. The volunteers answered the questionnaire in their respective workplaces, in small groups. The instructions were given to answer it, and all the questions were personally clarified with each participant. The research was conducted in full accordance with the ethical standards established by the 1964 Declaration of Helsinki and its later amendments, and by the American Psychological Association Ethical Principles of Psychologists and Code of Conduct (2017). Conicet’s ethical recommendations for research in the social and human sciences (Resolution 2827/06) were also considered in the implementation of the study.
The Generic Job Satisfaction Scale (Mac Donald, & Mac Intyre, 1997) was used, in its adapted version for Argentina (Salessi & Omar, 2016) and Brazil (De Andrade, Omar, & Salessi, In Press), respectively. Both versions are integrated by seven items: “In my job I can apply my abilities” (item 1); “In my job I receive recognition for my good performance” (item 2); “I feel good working for this company” (item 3); “I feel comfortable with my coworkers” (item 4); “My job gives me security” (item 5); “My salary is adequate” (item 6), and “In general terms I have a good job” (item 7) with a five-point Likert scale (1 = Totally disagree; 5 = Totally agree).
The results of exploratory and confirmatory factor analysis communicated by the respective validation studies (Salessi & Omar, 2016; De Andrade, Omar, & Salessi, In Press) demonstrated the relevance of an unifactorial measurement model with seven indicators as observable variables and their respective measurement errors [Argentina: SBχ2 = 1.97; GFI = .99; CFI = .99; RMSEA = .04 and Brazil: SBχ2 = 2.14; GFI = .99; CFI = .98; RMSEA = .04]. Both versions have adequate convergent-discriminant validity [Argentina: ave = .76; Brazil: AVE = .72], as well as internal consistency and composite reliability [Argentina: ᾠ = .90, CR = .87; Brazil: ᾠ = .77, CR = .70]. Sociodemographic questionnaire: Information about sex, age, seniority, marital status, educational level, type and organization sector, position, etc. was asked with descriptive purposes.
First, the presence of missing values and atypical scores was analysed. Those observations that were more than 3.5 SD from the mean were considered univariate atypical, whereas those presenting distances from Mahalanobis to square (D2) with a probability equal to or less than .001 (Hair, Black, Babin, Anderson, & Tatham, 2010) were classified as multivariate atypical. Secondly, the distribution of the items from a univariate perspective was analysed from the calculation of means and standard deviations, coefficients of asymmetry and kurtosis, Kolmogorov-Smirnov tests with the Lilliefors correction, and complementary qq plots, while the distribution of the items from a multivariate perspective was analysed with the Mardia’s normalized coefficient of multivariate kurtosis. Asymmetry and kurtosis values of less than +/- 1.60, p-values not significant associated to the Kolmogorov-Smirnov test, and multivariate kurtosis coefficients in the range -/+ 3 were considered adequate (Bentler, 2006; Hair et al., 2010; Tabachnick, & Fidell, 2013).
To determine if the scale was invariant in both countries, a confirmatory factorial analysis of multiple groups was carried out (Brown, 2015). The robust maximum likelihood method was used, given the breach of the assumption of normality in the distribution of the items. The analyses were made on the matrix of polychoric correlations, which estimate the continuous variables underlying the items of ordinal nature (Hoffmann, Stover, de la Iglesia, & Fernández Liporace, 2013). A progressive evaluation was carried out (Elousa, 2005; Timmons, 2010), based on a sequential study that begins with the estimation of a free model and continues imposing restrictions until the model is rejected.
In this sense, we analysed: (a) the configural invariance, relative to the structure of the scale; (b) the metric or weak invariance, consisting of the equivalence of the factorial loads between the groups, in all the items; (c) the scalar or strong invariance, related to invariance in factorial loads and intercepts, and (d) the strict invariance, related to the invariance not only in the previous levels but also in the residual variances.
To evaluate the configurational invariance, the base model suggested by the empirical evidence (De Andrade, Omar, & Salessi, In Press; Salessi & Omar, 2016 ) was estimated free and simultaneously in both samples, analysing that: (a) the index SBχ2 on the degrees of freedom (SBχ2/ ) was less than 3; (b) that the Goodness of Fit Index (GFI) and the Comparative Fit Index (CFI) reached values equal to or greater than .90, and (c) that the Root Mean Square Error of Approximation (RMSEA) was less than .05 (Hair et al., 2010). For the evaluation of the metric, scalar, and strict invariance, the different nested models were analysed, that is, the model without restrictions and the models with restrictions on the various parameters. In this regard, following the criteria proposed by Cheung and Rensvold (2002), the difference between fit indices was considered assuming that, if the value of the difference between GFI and CFI between two nested models is higher than .010, or higher than .015 in RMSEA, we should reject the model with more restrictions. This criterion was chosen instead of the chi-square difference test, taking into account the sensitivity of this estimator to the sample size (Barrera-Barrera, Navarro-García, & Peris-Ortiz, 2015). Data processing and analysis were done with the following software: Factor version 8 (descriptive analysis of items, intercorrelations, and univariate normality test), eqs version 6.2 (progressive invariance analysis and kurtosis multi-normality analysis), and spss version 23 (means comparison analysis, estimation of missing values, and univariate outliers).
The percentage of missing values for each sample did not exceed 5%; they were replaced by the estimation-maximization method (Tabachnick & Fidell, 2013). In the Argentine sample, six cases were detected that fulfilled the condition to be considered multivariate atypical; whereas, in the Brazilian sample, nine were identified that presented D2 with a probability equal to or lower than .001. Such observations were eliminated from the data matrix. Table 1 presents the inter-correlations between items, descriptive statistics, and Kolmogorov-Smirnov tests.
Although the values of asymmetry and univariate kurtosis corresponding to the Argentine sample were below the tolerable limit, in the Brazilian sample, observations showed that items 3 and 7 presented values of kurtosis higher than 1.60. The Kolmogorov-Smirnov (KS) test led to reject the null hypothesis of normality for all the items in both samples. At the same conclusion arrived the authors after analysing de qq plots. In turn, the coefficient of normalized multivariate kurtosis was located outside the recommended range, both in the Argentine sample (MK = 19.23) and Brazilian sample (MK= 29.36). These results demonstrated the non-compliance of the normality assumption, justifying the use of robust estimators for the invariance analysis.
Table 2 shows the fit indices corresponding to the nested models.
First, the base model (M0) suggested by the theory and the empirical evidence (De Andrade, Omar, & Salessi, In Press; Salessi & Omar, 2016) was estimated free and simultaneously obtaining an adequate fit for both samples. Such results provided evidence of configural invariance and allowed to continue with the next level of nesting: the weak invariance. In this sense, the model (M1) was re-specified, forcing the factorial saturation matrix of the Argentine sample to be equivalent to the factorial saturation matrix of the Brazilian one. The differences observed between the GFI, CFI, and RMSEA indices were within the ranges recommended by the specialized literature, demonstrating the metric invariance between the two samples.
Next, we proceeded to evaluate the strong factorial invariance, imposing the equivalence between intercepts. The indices obtained show an optimal adjustment of this model, both independently evaluated and analysed with respect to its nesting with the metric invariance model (M2). Once the strong invariance was confirmed, the analysis was continued by imposing the equivalence between the error variances in order to examine the strict invariance. On this occasion, the difference between the statisticians exceeded the tolerable value, leading to reject this model (M3). Following the suggestions of the specialized literature (Boomsma, 2000; Lei, & Wu, 2012), the statistically significant restrictions (p < .05) were identified from the Lagrange Multiplier. This indicator suggested that releasing the restrictions imposed on the error variances of items 5, 6, and 7 could lead to an improvement in the adjustment. Based on this information, we proceeded to relax the equality criterion in a progressive way for each of these items (Elousa, 2005). After eliminating the restriction in the residual variance of item 5, an improvement in the adjustment was observed with respect to the strict invariance model (M3a). However, the difference between the indices continued to be higher than the cut-off point set, so it was decided to relax also the restriction that weighed on the error variance of item 6. Once again, an improvement in the adjustment was obtained, but the differences continued exceeding the tolerable values (M3b). Therefore, it was decided to release the parameter corresponding to item 7, obtaining an appropriate adjustment for the partial strict invariance model (M3c).
Based on these results, it can be concluded that the Generic Job Satisfaction Scale is configural, metric, and scalar invariant between Brazil and Argentina; whereas, it conforms to strict factor invariance model only partially. That is to say, except for the error variances corresponding to item 5 (“My job gives me security”), item 6 (“My salary is appropriate”), and item 7 (“Considering it in general terms I have a good job”).
Differences in JS in Brazilian and Argentine workers
Once the factorial invariance was verified, possible differences between the observed and latent means were analysed to check if Brazil and Argentina differed significantly in the structure of means. The statistical significance associated with the differences between the observed ones was determined on the basis of the Student’s .-statistic. The possible differences between the latent means of the construct, meanwhile, were analysed from the model of complete scalar invariance (M2). In line with the procedure recommended by the literature (Bentler, 2006, Dimitrov, 2010), the value of the mean for the Argentine sample was fixed at 0, and the one for the Brazilian sample was freely estimated. The statistical significance of the differences was determined based on the computation of the critical ratio (CR). A CR value greater than 1.96 indicates statistically significant differences. If this value is positive, it implies that the contrast group (in this case, Brazil) has a higher latent mean compared to the reference group (in this case, Argentina). Conversely, if the CR is negative, the reference group has a higher latent mean than the contrast group.
The results indicated that the observed mean of JS was significantly higher in the Brazilian sample (MBrasil= 3.86, SDBrasil= .91; MArgentina= 3.52, SDArgentina= .90; t = 8.54, p < .000). Differences in latent mean corroborated this finding. In this regard, the CR obtained (CR= 11.062, p < .000) indicated that the Brazilian sample had a latent mean in JS higher than the Argentine one. However, the effect size, using Cohen’s d statistic (Lakens, 2013), showed that it was a small difference (d = .37; r = .18).
The current trend in instrumental studies, aimed at the validation and psychometric analysis of psychological measures, indicates the relevance of providing evidence to support that the measurement model is independent of the cultural and sociodemographic characteristics of the participants. Only if the factorial invariance of an instrument is verified empirically, it is possible to establish if the distribution of the scores observed in the answers depends solely on the space of the latent dimension evaluated (Byrne, 2008; Caycho, 2017; Cheung, & Rensvold, 2002; Dimitrov, 2010; Elousa, 2005; Wu et al., 2007).
Regarding the Generic Job Satisfaction Scale of Mac Donald and Mac Intyre (1997), although several instrumental studies have been developed in Europe, Asia, and Latin America (Bai et al., 2013, Salessi & Omar, 2016; van Saane et al., 2003), to date there is no evidence of its factorial invariance in the framework of cross-cultural research. In order to cover this empirical gap, the objective of this study was to analyse whether the scale, in its adapted and validated version for Argentina (Salessi & Omar, 2016) and Brazil (De Andrade, Omar, & Salessi, In Press), constitutes an instrument invariant at the configural, metric, scalar, and strict levels, respectively.
The progressive evaluation developed allowed to conclude that the instrument analysed is, in the first place, configural invariant. It is worth mentioning that JS, as a latent variable, is specified by the same manifest variables (items) in the two compared samples (Bentler, 2006; Brown, 2015). This result implies the substantive equivalence of the construct job satisfaction, as it is operationalized by the Generic Job Satisfaction Scale, between the two cultures.
Second, the scale has been shown to be invariant at the metric level, since each of its items presented equivalent saturations in the latent factor. In parallel, the results also indicate the scalar invariance, since the loads and intercepts between the groups are equivalent for all items. Such findings show that the instrument measures the same construct and that the items are psychometrically equivalent, presenting the same capacity for discrimination. So that, among a set of possible answers, they evoke with the same level of aptitude the same specific response between the subjects (Elousa, 2005).
With respect to strict invariance, the analysis showed that this was only partially achieved since the residual variances corresponding to three items showed not to be equivalent. The strict invariance ensures that the error variance is the same in the groups compared. It is worth saying that, under strict invariance, group differences in the variances of the observed variables are attributable only to group differences in variances of latent variables. In this sense, one could hypothesize that cultural differences could be behind these results. Considering that JS is an attitude related to work and interpersonal relationships that are interwoven in the workplace (supervisors, bosses, co-workers, clients), one would think that both the axiological profile and the motivational profile would impact its development. In other words, one might think, for example, the greater tendency towards collectivism by Brazilians, as well as their more optimistic vision of life, would lead them to experience greater JS, so that such characteristics would contribute to the development of feelings of greater collective well-being and interpersonal cooperation, as well as more positive emotional ties with the organization; perhaps because they perceive it as their main reference group and as the basis of their personal identity.
Based on the results obtained, it is possible to conclude that the Argentine and Brazilian adaptation of the Generic Job Satisfaction Scale had a factorial structure that adjusts to the strict factorial invariance model for all the items, except for the last three, which are equivalent in their configuration, load, and intercepts, but not in the residual variances. The determination of strong invariance allows comparing latent means since it offers guarantees to assume that the differences between the groups are due to the factors (Lei, & Wu, 2012; Wu et al., 2007).
In relation to this question, the analysis of the latent and observed means of JS indicates significant differences between the compared samples. Indeed, the Brazilian sample presents higher means in JS, which allows inferring that Brazilian workers are more satisfied with their work situation. However, the calculation of the effect size shows that these are small differences. These results coincide with the findings reported by other studies (Bentley et al., 2012; Ritter, & Anker, 2002), as well as with the conclusions reached by some well-known human resources consultants (Ranstad, 2016).
Like all research work, this study presents strengths and limitations. Regarding the latter, we must mention the representativeness of the sample, which was selected in a non-probabilistic manner. Although this circumstance limits the generalization of the results, as a palliative, the construction of the sample included workers from a wide range of activities. Nevertheless, it is recommended to continue analysing the properties of this scale in larger and heterogeneous samples. It would also be important for future research to examine whether the items that were not equivalent present a differential functioning.
Despite these limitations, this research constitutes a significant contribution to organizational literature, of a cross-cultural nature, in the Latin American context. In this regard, although research aimed at understanding the perceptions and motivations underlying the behavior of workers from different national groups is gaining importance in recent years, comparative studies between two South American countries are scarce, because most of the comparisons are made between North American, European, or Asian workers, where the socio-economic conditions are very different from Latin American ones.
In our region, the same as in the rest of the world, globalization gradually reduces the boundaries between workplaces geographically distant. For this reason, having a tool with proven psychometric properties, such as the one presented here, has important practical implications for researchers, managers, organizational psychologists, and occupational health professionals. Therefore, the use of this instrument is recommended for several reasons. First, for allowing to measure and permanently monitor the degree of JS of employees, regardless of their country of origin. Second, offering information to create organizational environments that favor more experiences of JS. Finally, for favoring the implementation of practices and interventions that result in higher levels of JS, productivity, and organizational health.
The main strength of this research lies, possibly, on the fact that it is the first study focused on analysing the invariance of a JS measurement instrument. In this way, in addition to providing additional evidence on the psychometric properties of the Generic Job Satisfaction Scale (Bai et al., 2013, Salessi & Omar, 2016, van Saane et al., 2003), it allows concluding that the differences observed in the levels of satisfaction between Brazilian and Argentine workers are, in fact, due to real differences in the variable. In this sense, it is noteworthy that the absence of factorial invariance can generate erroneous or biased interpretations when comparing different groups, so there is no certainty whether they are due to real differences in the construct or to the differential performance of the indicators (Byrne, 2008; Cheung & Rensvold, 2002).
The generalization of psychological constructs entails the need to standardize measurement instruments beyond specific cultures or languages. The comparison in the framework of transcultural research is an acceptable practice only if its formal and substantive equivalence is justified empirically. For this, it is essential to have empirical evidence to ensure that the construct has the same meaning in the groups since it does not make sense to compare scores in a construct between groups that do not have the same representation of it (Caycho, 2017). Paraphrasing Elousa (2005), it is possible to affirm that only the results of an invariance analysis can justify the comparisons between different groups.
Ato, M., López, J. J., & Benavente, A. (2013). Un sistema de clasificación de los diseños de investigación en psicología [A classification system of research designs in psychology]. Anales de Psicología, 29(3), 1038-1059. Doi: https://doi.org/10.6018/analesps.29.3.178511
Bai, X., Kwok, T., Chan, N., & Ho, F. (2013). Determinants of job satisfaction in Hong Kong. Health & Social Care in the Community, 21(5), 472-479. Doi: https://doi.org/10.1111/hsc.12029
Barrera-Barrera, R., Navarro-García, A., & Peris-Ortiz, M. (2015). El papel de la invarianza factorial en la validación del constructo calidad de servicio electrónico [The role of factorial invariance in the validation of the construct electronic service quality]. Revista Europea de Dirección y Economía de la Empresa, 24(1), 13-24. Doi: https://doi.org/10.1016/j.redee.2014.07.001
Bentler, P. (2006). eqs 6 structural equations program manual. Los Angeles, CA: Multivariate Software Inc.
Bentley, P., Coates, H., Dobson, I., Goedegebuure, L., & Meek, V. (2013). Job satisfaction around the academic world. Bridgewater, NY: Springer Verlag.
Boomsma, A. (2000). Reporting analyses of covariance structures. Structural Equation Modeling, 7(3), 461-483. Doi: https://doi.org/10.1207/s15328007sem0703_6
Brown, T. A. (2015). Confirmatory factor analysis for applied research. New York, NY: Guilford Publications.
Byrne B.M. (2008). Testing for multigroup equivalence of a measuring instrument: A walk through the process. Psicothema, 2, 872-882.
Caycho, T. (2017). Importancia del análisis de invarianza factorial en estudios comparativos en ciencias de la salud [Importance of factorial invariance analysis in comparative studies in health sciences]. Educación Médica Superior, 31(2), 1-3.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indeces for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255. Doi: https://doi.org/10.1207/S15328007SEM0902
Dalal, R. S., & Credé, M. (2013). Job satisfaction and other job attitudes. In K. F. Geisinger et al. (Eds.), apa handbook of testing and assessment in psychology (pp. 675-691). Washington, DC: American Psychological Association. Doi: https://doi.org/10.5860/CHOICE.50-7042
De Andrade, A., Omar, A. & Salessi, S. (en prensa). Generic Work Satisfaction Scale: Psychometric Qualities of the Brazilian Version. Avaliação Psicológica.
Dimitrov, D. M. (2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counselling and Development, 43(2), 121-149. Doi: https://doi.org/10.1177/0748175610373459
Dou, K., Nie, Y., Wang, Y., & Liu, Y. (2016). The relationship between self-control, job satisfaction and life satisfaction in Chinese employees: A preliminary study. Work, 55(4), 797-803. Doi: https://doi.org/10.3233/WOR-162447
Elosua, P. (2005). Evaluación progresiva de la invarianza factorial entre las versiones original y adaptada de una escala de autoconcepto [Progressive evaluation of factorial invariance between the original and adapted versions of a self-concept scale]. Psicothema, 17(2), 356-362.
Hair, J. E., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2010). Multivariate data analysis. Upper Saddle River, NJ: Pearson-Prentice Hall.
Hoffmann, A., Stover, J., De la Iglesia, G., & Fernández-Liporace, M. (2013). Correlaciones policóricas y tetracóricas en estudios factoriales exploratorios y confirmatorios [Polychoric and tetrachoric correlations in exploratory and confirmatory factorial studies]. Ciencias Psicológicas, 7, 151-164.
Judge, T. A., & Kammeyer-Mueller, J. D. (2012). Job attitudes. Annual Review of Psychology, 63, 341-367. Doi: https://doi.org/10.1146/annurev-psych-120710-100511
Judge, T., Weiss, H., Kammeyer-Mueller, J., & Hulin, C. (2017). Job attitudes, job satisfaction, and job affect: A century of continuity and of change. Journal of Applied Psychology, 102(3), 356-374. Doi: https://doi.org/10.1037/apl0000181
Keller, A. C., & Semmer, N. K. (2013). Changes in situational and dispositional factors as predictors of job satisfaction. Journal of Vocational Behaviour, 83(1), 88-99. Doi: https://doi.org/10.1016/j.jvb.2013.03.004
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and anovas [Electronic version]. Frontiers in Psychology, 4, 863-867. Doi: https://doi.org/10.3389/fpsyg.2013.00863
Lei, P., & Wu, Q. (2012). Estimation in structural equation modeling. In Hoyle, R. H. (Ed.), Handbook of structural equation modeling (pp. 164-179). New York, NY: Guildford Press.
Macdonald, S., & MacIntyre, P. (1997). The Generic Job Satisfaction Scale: Scale development and its correlates. Employee Assistance Quarterly, 13, 1-16. Doi: https://doi.org/10.1300/j022v13n02_01
Omar, A., Uribe, H., Assmar, E., Ferreira, M. & Terrones, A. (2000). Atribución transcultural del rendimiento académico. Un estudio entre Argentina, Brasil y México. Revista Mexicana de Psicología, 17(2), 163-170.
Omar, A.; Uribe, H., Ferreira, C., Assmar, E., Terrones, A. & Galaz, M. (2007). Colectivismo, justicia y ciudadanía organizacional en empresas argentinas, mexicanas y brasileras. Revista Mexicana de Psicología, 24(1), 101-116.
Punnett, B. J., Greenidge, D., & Ramsey, J. (2007). Job attitudes and absenteeism: A study in the English-speaking Caribbean. Journal of World Business, 42(2), 214-227. Doi: https://doi.org/10.1016/j.jwb.2007.02.006
Ranstad (2016). Randstad Workmonitor [Electronic version]. Retrieved from https://www.randstad.es/tendencias360/?s=workmonitor
Ritter, J. A., & Anker, R. (2002). Good jobs, bad jobs: Workers’ evaluations in five countries. International Labour Review, 141(4), 331-358. Doi: https://doi.org/10.1111/j.1564-913x.2002.tb00244.x
Salessi, S. (2014). Satisfacción laboral: acerca de su conceptualización, medición y estado actual del arte. Revista de Psicología (UCA), 10(18), 718.
Salessi, S. & Omar, A. (2016). Satisfacción laboral genérica. Propiedades psicométricas de una escala para medirla. Alternativas en Psicología, 34(1), 93-108.
Sanín, J. A., & Salanova, M. (2014). Satisfacción laboral: el camino entre el crecimiento psicológico y el desempeño laboral en empresas colombianas industriales y de servicios [Job satisfaction: The path between psychological growth and job performance in Colombian industrial and service companies]. Universitas Psychologica 13(1), 95-107. Doi: https://doi.org/10.11144/Javeriana.UPSY13-1.slcpUniversitas
Shin, J., & Jung, J. (2014). Academics job satisfaction and job stress across countries in the changing academic environments. Higher Education, 67(5), 603-620. Doi: https://doi.org/10.1007/s10734-013-9668-y
Tabachnick, B. G., & Fidell, L. S (2013). Using multivariate statistics. Boston, MA: Pearson.
Timmons, A.C. (2010). Establishing factorial invariance for multigroup confirmatory factor analysis [Electronic version]. KUant Guide, 22... Retrieved from http://crmda.dept.ku.edu/resources/kuantguides/22.Factorial_Invariance_Guide.pdf
Wu, A.D., Li, Z., & Zumbo, B. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with data [Electronic version]. Practical Assessment Research & Evaluation, 12, 1-26. Retrieved from https://www.semanticscholar.org/paper/Decoding-the-Meaning-of-Factorial-Invariance-and-of-Wu-Li/085103499f57ed09d9cd12ececb3fe245e219db2
Zito, M., Molino, F., Cortese, M., Giovanni, C., Ghislieri, C., & Colombo, L. (2018). Turnover intentions in a call center: The role of emotional dissonance, job resources, and job satisfaction [Electronic version]. PLoS One, 13(2). Doi: https://doi.org/10.1371/journal.pone.0192126
** Works at Departamento de Psicologia, Programa de Pós-graduação em Psicologia, Laboratório de Avaliação e Mensuração Psicológica, Vitória, Brasil.
*** Is now at Consejo Nacional de Investigaciones Científicas y Técnicas, Instituto de Investigaciones de la Facultad de Humanidades y Artes, Rosario, Argentina.
* Address correspondence to Solana Salessi: Consejo Nacional de Investigaciones Científicas y Técnicas, Universidad Nacional de Rafaela, Departamento de Educación, Cultura y Conocimiento, Rafaela, Argentina; Pontificia Universidad Católica Argentina, Facultad de Derecho y Ciencias Sociales del Rosario, Rosario, Argentina. E-mail: email@example.com ORCID ID https://orcid.org/0000-0001-9496-9493
How to cite this article: Salessi, S., de Andrade, A. L., & Omar, A. (2020). Factorial Invariance of the Generic Job Satisfaction Scale in Argentina and Brazil. Avances en Psicología Latinoamericana, 38(2), 1-14. https://doi.org/10.12804/revistas.urosario.edu.co/apl/a.7477
Solana Salessi * firstname.lastname@example.org
Centro de Investigación y Transferencia de Doble Dependencia Universidad Nacional de Rafaela y CONICET, Argentina
Alexsandro Luiz de Andrade **
Universidade Federal do Espírito Santo, Brasil
Alicia Omar ***
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), Argentina