Introduction

Recently, treatment of HIV infection has shown remarkable progress. In the 1990s, a combination of drugs exhibiting a high capacity to suppress viral loads was discovered. Reverse transcriptase inhibitors along with protease inhibitors comprise what is now called highly active antiretroviral therapy (haart). haart can reduce the viral count to undetectable levels, thus improving the functioning of the immune system, preventing transmissibility, and reducing morbidity and mortality (1–3). However, achieving these effects requires a treatment adherence level of close to 90%–95% (4); therefore, optimal level of treatment adherence is a required to achieve undetectable viral load, which was one of unaids’s 90-90-90 goal (5).

A meta-analysis reveal that approximately 60%–70% of patients undergoing treatment in different countries (6) show this level of treatment compliance, particularly those in Latin America and the Caribbean (7). Owing to the problems that may arise from poor treatment adherence, such adherence is one of the main challenges associated with the treatment of hiv infection today. Notably, inadequate adherence can lead to the development of strains that are resistant to the most effective treatment available to date (1). Furthermore, risk behaviors (e.g., sexual intercourse without a condom) expose uninfected individuals to these strains and those with hiv to re-infection, thus limiting treatment options and worsening the situation (1,3).

Different techniques have been proposed to address the lack of long-term treatment adherence, including the Information, Motivation, and Behavioral (imb) skills model (8). This concept differs from others as it was designed specifically for hiv-infected individuals. The imb model allows the identification of key intervening factors that can help evaluate and understand the problems associated with haart adherence as well as determine the target variables to design interventional strategies.

The imb model uses a simple and dynamic scheme that links different factors that influence haart adherence. It specifically states that if a person with hiv has enough information about items, including regime and side effects, and the necessary motivation to undergo successful treatment (favorable attitudes towards adherence and perception of social support), then they will put specific behavioral skills into practice to achieve treatment adherence, such as through acquiring the drugs and including the regimen in their daily routine. These skills will result in treatment adherence behavior (8).

The Information factor here refers to aspects including the dose, how and when to take the medication, side effects, degree of treatment adherence required, and interactions with other drugs (9). Different studies have highlighted the impact of this factor on adherence to antiretroviral treatment (10–15).

The individual’s attitude towards treatment adherence and perceived social support are included in the Motivation factor. In this sense, Wasti et al. state that not informing others about their diagnosis increases the probability of non-adherence to treatment (15). Several studies support this hypothesis for antiretroviral treatment as well as for treatment in general (10,13,14,16,17).

Other studies consider Information and Motivation as independent factors that play important roles in preventive behaviors that improve the health status (18), with Information being mandatory for treatment adherence (9).

Behavioral skills in the imb model include the individual’s specific abilities to be adherent and the self-efficacy for all actions related to treatment adherence (9). Several studies refer to the relationship between specific behavioral skills, perceived self-efficacy, and adherence to haart (10,14,19,20).

To assess the barriers to treatment adherence, an instrument was developed within the framework of the imb model called the “Life Windows – Information, Motivation, Behavioral Skills–Antiretroviral Treatment Adherence Questionnaire” (lw-imb-aaq) (21). This model has been used in different countries including India (22), Romania (23), South Africa (24), and Ethiopia (25). The use of the Motivation subscale has been reported in Argentina as well (26). However, many of these studies do not report the adaptation and validation of the instrument for local use. The original study underestimates the cultural differences that can affect not only the validity of the instrument but also that of the theoretical model; therefore, psychometric investigations of the model have been conducted in populations other than that of the original study (27).

One of the lw-imb-aaq adaptation studies was conducted by Peng et al. for application in Shanghai (4). The authors proposed a structure with three sections that corresponded to the three subscales of the original instrument based on the confirmatory factor analysis but with two subscales in each of these sections. Items 1 and 4 of the motivation subscale were eliminated for the model to yield acceptable fit indices. The reliability values (α) were .84 for Information, .81 for Motivation, and .91 for Behavioral Skills.

The psychometric properties of the lw-imb-aaq were assessed in another study (28). Exploratory factorial analysis was conducted with a varimax rotation where the original factorial structure was replicated. Further, validity in terms of other variables whose oblique factors were related to measures of treatment adherence was evaluated. Although it is important to acknowledge this effort, it should be noted that there are some limitations to the procedures. Confirmatory factor analysis is currently recommended when starting from a delimited structure (29). Furthermore, varimax rotation assumes that the factors are orthogonal, a condition that is not met in the lw-imb-aaq. Therefore, in this case, the use of oblique rotations is suggested as they are more in line with the representation of the constructs studied in psychology (30).

A literature review revealed that no instruments for measuring antiretroviral treatment adherence and/or factors related to it that have been developed, adapted, and/or validated in Argentina. A questionnaire to evaluate the possible drawbacks in adherence with the appropriate properties and technical requirements is needed to address the challenges posed by the current hiv epidemic (14,13,31). Therefore, this study aimed to validate the instrument for the application in the Argentinian population.

Materials and methods

The sample population comprised 190 people with hiv selected using the nonrandom accidental type who were undergoing antiretroviral treatment at the Córdoba province, Argentina. Ten people were excluded from the study because of missing data, thus exceeding the criteria suggested for the data imputation process (32). The final sample population included 180 people. The mean age of the participants was 40.61 (sd = 12.032) years and 82.8% were men. Regarding ongoing medication, 81% stated that they complied with the schedules but 60% stated that they sometimes forgot.

We included patients aged >18 years who were undergoing antiretroviral treatment and provided their informed consent in compliance with the ethical aspects listed in the Declaration of Helsinki (33). The project was endorsed by the Research Ethics Committee of the College of Psychologists of the Province of Córdoba (File No. 3678).

Two main instruments were used: a self-reported adherence questionnaire and the instrument to be adapted. Data collection was conducted using a questionnaire on sociodemographic variables and others topics related to the positive diagnosis of hiv as a control.

The questionnaire on sociodemographic and other variables was developed ad hoc and it included questions about gender, age, occupation, date of hiv diagnosis, the person(s) the patient lives with, level of education attained, how they found out about the diagnosis, how many people they told about their diagnosis, and their relationship with those individuals.

The Simplified Medication Adherence Questionnaire (smaq) (34) was used to asses treatment adherence. The structure of this test is unidimensional and it comprised six items, five with a dichotomous response option and one graded option. Questions were about compliance or noncompliance to drug intake according to the schedule determined for the patient during the past 3 months, the last weekend, and the last week. Forgetfulness, carelessness with the times of the shots, and differential behavior due to discomfort are also explored. This instrument was previously used in a study on an Argentinian population with HIV whose results reported that its validity and reliability were adequate (35).

The lw-imb-aaq test comprises 33 items grouped into three subscales: Information, Motivation,The lw-imb-aaq test comprises 33 items grouped into three subscales: Information, Motivation, and Behavioral skills. The first includes 9 items, the second includes 10 items, and the third includes 14. The psychometric properties are reported together with the publication of the original version of the instrument. The Information subscale had a low index (α = .59) of inter- Psychometric Properties of an Instrument to Assess the Barriers to Antiretroviral Treatment Adherence 6 Rev. Cienc. Salud. Bogotá, Colombia, vol. 21(1): 1-17, enero-abril de 2023 nal consistency, which was expected because the items were intended to consider aspects of the treatment that were not interrelated. Reliability indices (α) of .70 and .75 were found for the Motivation subscale. Finally, the Behavioral Skills subscale presented reliability indices of .90 in all except the first item (21). A Spanish version was provided by Amico (personal communication, November 16, 2016).

The methodological design of this study meets the criteria established by Montero and León for instrumental studies (36). Participants were invited to participate and informed about the attendance policy for patients regarding the medical appointments of the institution and private offices. Questionnaires were provided after they expressed their agreement and signed the informed consent form. The data were then systematized for the corresponding analyses and tests.

A confirmatory factor analysis was conducted using the mplus v.7 program, for the evidence of validity of the internal structure through the weighted least squares means and variance adjusted as a parameter estimation method (36,38). It is more precise when assessing ordinal variables with distributions that do not approach normality and with small samples (39-41); factor loads greater than .40 were considered acceptable (42). The adjustment of the measurement model was assessed following the recommendations of Hu and Bentler, Steiger and DiStefano, Liu, Jiang and Shi where cfi values ≥ .95, rmsea ≤ .07, tli ≥ .95, wrmr ≤ .90, are considered as indicators of acceptable fit (43-45). Transverse to this entire process, the modification indices were also analyzed where values >.20 of the Expected Unstandardized Parameter Change (epc) indicated specifications that had to be performed in the evaluated model (46,47).

The structural hypothesis initially suggested by the original authors of the instrument was analyzed. According to this model (M1), the 33 items can be grouped into three oblique factors (M1). New models (M2 and M3) were proposed based on the previous one, with the purpose of improving the fit of the measurement model.

The reliability of the scores was then estimated using the omega coefficient (ω´) and the correction for correlated errors (ω´), according to the recommendations of Gadermann et al., and Greco et al. (48-52) for interpretation.

We finally sought to provide evidence of validity regarding the other variables. Specifically, a concurrent validity analysis was conducted with the aim of analyzing the relationship between treatment adherence (total smaq score) and the three subscales of the lw-ibm-aaq using linear regression analysis. Statistically significant relationships can be expected based on previous studies (53).

Results

Descriptive analysis of the data was conducted first. As shown in Table 1, 27% of the items have asymmetry and kurtosis values >±2, suggesting that the data does not approximate a normal distribution (42,54).

Further, the fit of the measurement model was assessed. Unacceptable fit indices were obtained for the original model (M1), which were items with factor loads of <.40 and EPC values that suggested adding specifications between the items of the three oblique factors. Considering this, a review was performed based on the content of the items where potentially overlapping reagents were identified. Accordingly, items 3 and 5 (Information), 2, 4, 5, 6, 8, and 9 (Motivation), and 1, 3, and 4 (Behavioral Skills) were removed from this process.

Although the fit indices improve for M2, they do not optimally meet the criteria mentioned in the literature. Furthermore, there are still epc values >.20 that are related to the correlations between items that can be considered to be under parameterization. As a result, items 2 and 13 (Behavioral Skills) were removed as well.

Finally, for M3, a model of three oblique factors with acceptable fit indices was observed, with factor loads >.40 and without specifications to add or remove from the proposed model (epc < .20) (Table 2).

Third, the internal consistency was analyzed considering the different models. The reliability of the M1 scores were as follows: ω_F1 = .765/ω´_F1 = .757; ω_F2 = .781/ω´_F2 = .702 and ω_F3 = .889/ω´_F3 = .886; for model M2: ω_F1 = .832/ω´_F1 = .767; ω_F2 = .759 and ω_F3 = .917/ω´_F3 = .873; and for model M3: ω_F1 = .833; ω_F2 = .759 and ω_F3 = .888. In all cases, it can be inferred that the values are acceptable.

As a last step, evidence on the validity in terms of other variables was collected. As shown in Table 3, there is a statistically significant relationship between the scores of the Information and Behavior subdimensions of the lw-imb-aaq with treatment adherence. The evaluated model presents a value of R² = .134, which is considered mid-level (53). The Motivation subdimension did not provide statistically significant values.

Discussion

The objective of this study was to evaluate the psychometric properties of the lw-imb-aaq in a sample population including individuals diagnosed with hiv in Córdoba, Argentina. This study was needed in the absence of other studies, specifically those on the internal structure validity, relationship with other variables, and analysis of the reliability of the scores. All this was performed as per the recommendations of the Standards for Educational and Psychological Testing (55). The final version of the instrument included 20 items in the Information, Motivation, and Behavioral Skills subscales, with significant associations with treatment adherence and acceptable reliability results for application in the Argentinian population.

The results indicated that the original factorial structure is not replicated in terms of the evidence of validity of the internal structure. There were specifications in M1 that suggested adding relationship parameters between items, allowing us to infer a possible factorial complexity because of the poorly differentiated factorial loads of some items and because there were high interfactorial correlations between the Motivation and Behavioral Skills subscales (43,56). This evidence is supported by the high correlations between the same subscales by Santillán-Torres-Torrija et al. (28). For this reason, in view of the previous issues and the representation of the construct through the theoretically proposed components, we decided to remove items with redundant content and low factor loading so we could obtain a robust version of three oblique factors that continues to represent what was proposed by Fisher et al. (9).

The working hypothesis was that the subscales measured through the smaq are important predictors of treatment adherence for the evidence of validity regarding other variables of the final version of the lw-imb-aaq. The results of the linear regression analysis indicated that the factors of Information and Behavioral Skills are the most important predictors of the variance associated with taking medications. However, when analyzing the proportion of variance predicted by each factor, this is considered high for all subscales. This result is consistent with that reported by Fisher et al., who suggest that information is a necessary condition and the most important factor for engaging in behaviors involved in treatment adherence (9). Another reason for this is that there are people who have the information but low levels of motivation and vice versa (18). This result is consistent with those obtained by Botempi et al., Miller et al., Jones et al., Varela-Arévalo et al., Wasti et al., Simoni et al., Horvath et al., Ladero-Martín et al., and Fisher et al. (9-12,14-16,19,20).

The evaluated models can be considered acceptable according to the standards of Gadermann et al. and Greco et al. in terms of score reliability (51,52). However, there is a difference between the values of the omega coefficient and the correction for correlated errors as it has been reported that the under-parameterization of correlated errors in statistical models causes reliability to be a biased estimator (57). One of the causes is the presence of redundant items, as seen in the case of the lw-imb-aaq, and can lead false positives and false negatives that have a possible direct effect on decision making (58-60). These values suggest that the lw-imb-aaq can be applied for group description purposes and not for individual decision-making considering that values around .90 are needed in the three subscales.

In short, the findings of this study coincide with that of the original study in terms of factors as well as internal structure (21). The factorial structure confirms the presence of three main factors, coinciding with the Information, Motivation, and Behavioral Skills subscales, which was also reported by Santillán-Torres-Torrija et al. (28). After removing two items, Peng et al. (4) identified three sections that coincide with the abovementioned factors but with two subscales each. Moreover, the analysis of the internal consistency of the subscales also yielded acceptable values both in this study and in the research by Peng et al. (4). However, in the original study and in the one conducted by Santillán-Torres-Torrija et al. the Information subscale showed a α of .70 (21,28).

There are certain limitations to this study. The sample mainly comprised patients from the same institution, and the majority included patients who adhered to treatment; this could influence the results of the study. Therefore, future studies should include sample populations including participants from both public and private health institutions. Likewise, we recommend increasing the number of cases to obtain greater variability and to conduct more complex analyses, such as structural equations.

This is an initial study on wl-imb-aaq; therefore, we recommend evaluating the factor structure through confirmatory analysis of measurement invariance to corroborate its equivalence in different populations, thus contributing to the generalization of its use. This is especially considering that the removal of the items in the final structure of this version of the instrument should be confirmed with other samples for consolidation (61).

In conclusion, this research constitutes a significant theoretical and practical contribution to the approximation of adherence to antiretroviral treatment, because information was obtained on the best representation of the constitutive definition of the construct among the Argentinian population, and because this study can be a precedent for the wl-imb-aaq to be most widely supported, and to be used by health professionals as evidence for the design of specific intervention strategies.

Author contributions

Elena Lucía Colasanti was responsible for conceptualization, data curation, formal analysis (statistical techniques, data analysis), fundraising, the research process, design and development of the methodology, project administration (management and coordination), software (programming, design and implementation), drafting of the document, and its subsequent revision and edition. Jhonatan Navarro-Loli worked on data curation, formal analysis (statistical techniques, data analysis), design and development of the methodology, data visualization and presentation, document writing, and review and editing. Marcos Marino performed the conceptualization, research process, project administration (management and coordination), software (programming, design and implementation), and review and edition of the text. Leonardo Adrián Medrano worked on the conceptualization, data curation, formal analysis (statistical techniques, data analysis), the research process, design and development of the methodology, project administration (management and coordination), drafting of the document, and its revision and edition.

Funding

This investigation was conducted with the support of salud investiga “Dr. Abraam Sonis” 2017, individual category, granted by the Ministry of Health, through the Directorate of Health Research. These were not involved in the development of the research.

Conflicts of interest

None declared.

Acknowledgments

Participants, Rivet Amico, Jesús Maurino, Natalia Colazo, Miguel Díaz, Mario Belardi, Fernanda Molina, María Laura Calvo, Graciela Sadino, Norma Luna, Sofía Gauchat, Magdalena Sereno, Malena Parra, Marcelo Vaiman.

References

1. Altice F, Evuarherhe O, Shina S, Carter G, Beaubrun AC. Adherence to HIV treatment regimens: systematic literature review and meta-analysis. Patient Prefer Adherence. 2019;13:475-90. https://doi.org/10.2147/ppa.s192735

2. World Health Organization. hiv aids [Internet]. [Cited 2020 Jul 6]. Available from: https://www.who.int/es/news-room/fact-sheets/detail/hiv-aids

3. Villa IC, Vinaccia S. Therapeutic adherence and associated psychological variables in patients diagnosed with HIV-AIDS. Psychol Health. 2006;16(1):51-62.

4. Peng Z, Yu Y, Wei W, Hou Y, Sun Z, Wang Y, et al. Reliability and validity of the lifewindows information-motivation-behavioral skills antiretroviral therapy adherence questionnaire among HIV+ patients in Shanghai. Patient Prefer Adherence. 2020;14:507-15. https://doi.org/10.2147/ppa.s234041

5. Joint United Nations Program on hiv/aids [unaids]. Our action. 90-90-90: Treatment for all. [internet]. [Cited 2021 Jul 31]. Available from: https://www.unaids.org/es/resources/909090

6. Ortego C, Huedo-Medina TB, Llorca J, Sevilla L, Santos P, Rodríguez E, et al. Adherence to highly active antiretroviral therapy (haart): a meta-analysis. aids Behav. 2011;15(7):138196. https://doi.org/10.1007/s10461-011-9942-x

7. Mattos-Costa JD, Silva-Torres T, Esteves-Coelho L, Mendes-Luz P. Adherence to antiretroviral therapy for hiv/aids in Latin America and the Caribbean: systematic review and meta‐analysis. J Int AIDS Society. 2018;21(1):1-20. https://doi.org/10.1002/jia2.25066

8. Fisher JD, Amico KR, Fisher WA, Harman JJ. The information-motivation-behavioral skills model of antiretroviral adherence and its applications. Curr hiv/aids Rep. 2008;5(4):193203. https://doi.org/10.1007/s11904-008-0028-y

9. Fisher JD, Fisher WA, Amico KR, Harman JJ. An information-motivation-behavioral skills model of adherence to antiretroviral therapy. Health Psychol. 2006;25(4):462-73. https://doi.org/10.1037/0278-6133.25.4.462

10. Bontempi JMB, Burleson L, Lopez MH. hiv medication adherence programs: the importance of social support. J Community Health Nurs. 2004;21(2):111-22. https://doi.org/10.1207/s15327655jchn2102_05

11. Jones D, Cook R, Rodriguez A, Waldrop-Valverde D. Personal hiv knowledge, appointment adherence and hiv outcomes. AIDS Behav. 2013;17(1):242-49. https://doi.org/10.1007/s10461-012-0367-y

12. Miller LG, Liu H, Hays RD, Golin CE, Ye Z, Beck CK, et al. Knowledge of antiretroviral regimen dosing and adherence: a longitudinal study. Clin Infect Dis. 2003;36(4):514-18. https://doi.org/10.1086/367857

13. World Health Organization. Adherence to long-term treatments. Tests for action. Washington (DC): Pan American Health Organization (Noncommunicable Diseases Unit); 2004. Available from: https://www.paho.org/hq/dmdocuments/2012/WHO-Adherence-Long-Term-Therapies-Spa-2003.pdf

14. Varela-Arévalo MT, Salazar-Torres IC, Correa-Sánchez D. Adherencia al tratamiento en la infección por VIH/sida: consideraciones teóricas y metodológicas para su abordaje. Act Colom Psicol. 2008;11(2):101-13.

15. Wasti SP, Simkhada P, Randall J, Freeman JV, van Teijlingen E. Factors influencing adherence to antiretroviral treatment in Nepal: a mixed-methods study. PLoS One. 2012;7(5):1- 11. https://doi.org/10.1371/journal.pone.0035547

16. Simoni JM, Frick PA, Huang B. A longitudinal evaluation of a social support model of medication adherence among hiv-positive men and women on antiretroviral therapy. Health Psychol. 2006;25(1):74-81. https://doi.org/10.1037/0278-6133.25.1.74

17. DiMatteo MR. Social support and patient adherence to medical treatment: a meta-analysis. Health Psychol. 2004;23(2):207-18. https://doi.org/10.1037/0278-6133.23.2.207

18. DiClemente RJ, Crosby RA, Kegler MC. Emerging theories in health promotion practice and research. 2nd ed. San Francisco (CA): John Wiley & Sons; 2009.

19. Horvath KJ, Smolenski D, Amico KR. An empirical test of the information-motivation-behavioral skills model of art adherence in a sample of hiv-positive persons primarily in out-of-hiv-care settings. AIDS Care. 2014;26(2):142-51. https://doi.org/10.1080/09540121.2013.802283

20. Ladero-Martín L, Orejudo-Hernández S, Carrobles-Isabel JA. Self-efficacy, effort, and expectations of control in adherence to antiretroviral treatment in HIV+ patients in a methadone maintenance program. Health Drugs. 2008;8(1):73-91. https://doi.org/10.21134/haaj.v8i1.68

21. The LifeWindows Project Team. The Life Windows Information Motivation Behavioral Skills ART Adherence Questionnaire (LW-IMB-AAQ). Center for Health, Intervention, and Prevention. University of Connecticut; 2006.

22. Chitra L, Gnanadurai A. Anti-retroviral therapy adherence among women living with HIV in Coimbatore District of Tamil Nadu, India. Int J Innov Res Dev. 2015;4(1):15-8.

23. Dima AL, Schweitzer A-M, Amico KR, Wanless RS. The information-motivation-behavioral skills model of art adherence in Romanian young adults. J HIV AIDS Soc Serv. 2013;12(3-4):27493. https://doi.org/10.1080/15381501.2012.749819

24. Peltzer K, Friend-du Preez N, Ramlagan S, Anderson J. Antiretroviral treatment adherence among hiv patients in KwaZulu-Natal, South Africa. BMC Public Health. 2010;10(111):110. https://doi.org/10.1186/1471-2458-10-111

25. Zeleke AB. Evaluation of the impact of the information-motivation-behavioural skills model of adherence to antiretroviral therapy in Ethiopia [PhD thesis on internet]. Universidad de Sudáfrica; 2015. Available from: http://hdl.handle.net/10500/19649

26. Jones D, Cook R, Cecchini D, Sued O, Bofill L, Weiss S, et al. Examining adherence among challenging patients in public and private hiv care in Argentina. AIDS Behav. 2015;19(9):1619-29. https://doi.org/10.1007/s10461-015-1037-7

27. Casullo MM. La evaluación psicológica: modelos, técnicas y contextos. Rev Iberoam Diagn Eval Psicol. 2009;27(1):9-28.

28. Santillán-Torres-Torija C, Villagrán-Vázquez G, Robles-Montijo SS, Eguiluz-Romo L de L. The information and motivation and behavioral skills model of art adherence among hiv-positive adults in Mexico. J Int Assoc Provid AIDS Care. 2015;14(4):335-42. https://doi.org/10.1177/2325957415581903

29. Flora DB, Flake JK. The purpose and practice of exploratory and confirmatory factor analysis in psychological research: decision for scale development and validation. Can J Behav Sci. 2017;49(2):78-88. https://doi.org/10.1037/cbs0000069

30. Schmitt TA. Current methodological considerations in exploratory and confirmatory factor analysis. J Psychoeduc Assess. 2011;29(4):304-21. https://doi.org/10.1177/0734282911406653

31. Benetucci A, Kásparas G, Wainstein C. Adherencia al TARV. En the argentine society of infectious diseases, V Argentine consensus on antiretroviral therapy 2014-2015. Buenos Aires: Guadalupe Editions; 2015. p. 31-9.

32. Schlomer GL, Brauman S, Card NA. Best practices for missing data management in counseling psychology. J Couns Psychol. 2010;57(1):1-10. https://doi.org/10.1037/a0018082

33. World Medical Association. Declaration of helsinki ethical principle for medical research involving human subjects. jama. 2013;310(20):2191-4. https://doi.org/10.1001/jama.2013.281053

34. Knobel H, Alonso J, Casado JL, Collazos J, González J, Ruiz I, et al. Validation of a simplified medication adherence questionnaire in a large cohort of hiv-infected patients: the GEEMA study. aids. 2002;16(4):605-13. https://doi.org/10.1097/00002030-200203080-00012

35. Ahumada M, Escalante E, Santiago I. Estudio preliminar de las relaciones entre las estrategias de afrontamiento y el apoyo social con la adherencia al tratamiento de personas que viven con vih/sida. Subj Procesos Cogn. 2011;15(1):55-70.

36. Montero I, León OG. Guide to naming research studies in psychology. Int J Clin Health Psychol. 2007;7(3):847-62. Available from: http://www.aepc.es/ijchp/

37. Muthén LK, Muthén BO. Mplus user’s guide. Versión 7 [Software]. Los Angeles (CA): Muthén & Muthén; 2012.

38. Muthén BO. A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika. 1984;49(1).115-32. https://doi.org/10.1007/bf02294210

39. Li CH. Confirmatory factor analysis with ordinal data: comparing robust maximum likelihood and diagonally weighted least squares. Behav Res Methods. 2016;48(3):936-49. https://doi.org/10.3758/s13428-015-0619-7

40. DiStefano C, Morgan GB. A comparison of diagonal weighted least squares robust estimation techniques for ordinal data. Struct Equ Modeling. 2014;21(3):425-38. https://doi.org/10.1080/10705511.2014.915373

41. Bandalos DL. Is parceling really necessary? A comparison of results from item parceling and categorical variable methodology. Struct Equ Modeling. 2008;15(2):211-40. https://doi.org/10.1080/10705510801922340

42. Pérez E, Medrano L. Análisis factorial exploratorio: bases conceptuales y metodológicas. Rev Argent Cienc Comport. 2010;2(1):58-66.

43. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6(1):1-55. https://doi.org/10.1080/10705519909540118

44. Steiger JH. Understanding the limitations of global fit assessment in structural equation modeling. Pers Individ Differ. 2007;42(5):893-8. https://doi.org/10.1016/j.paid.2006.09.017

45. DiStefano C, Liu J, Jiang N, Shi D. Examination of the weighted root mean square residual: evidence for trusworthiness? Struct Equ Modeling. 2018;25(3):453-66. https://doi.org/10.1080/10705511.2017.1390394

46. Domínguez-Lara S, Merino-Soto C. Evaluación de las malas especificaciones en modelos de ecuaciones estructurales. Rev Argent Cienc Comport. 2018;10(2):19-24. https://doi.org/10.32348/1852.4206.v10.n2.19595

47. Whittaker TA. Using the modification index and standardized expected parameter change for model modification. J Exp Educ. 2012;80(1):26-44. https://doi.org/10.1080/00220973.2010.531299

48. Domínguez-Lara S. Errores correlacionados y estimación de la fiabilidad de estudios de validación: comentarios al trabajo de validación de la escala ehealth literacy (eheals) en población universitaria española. Rev Esp Salud Pública. 2016;90:e1-e2.

49. Raykov T. Bias in coefficient alpha for fixed congeneric measures with correlated errors. Appl Psychol Meas. 2001;25(1):69-76. https://doi.org/10.1177/01466216010251005

50. Raykov T. Point and interval estimation of reliability for multiple-component measuring instruments via linear constraint covariance structure modeling. Struct Equ Modeling. 2004;11(3):452-83. https://doi.org/10.1207/s15328007sem1103

51. Gadermann AM, Guhn M, Zumbo BD. Estimating ordinal reliability for Likert-type and ordinal item response data: a conceptual, empirical, and practical guide. Pract Assess Res Eval. 2012;17(3):1-13. https://doi.org/10.7275/n560-j767

52. Greco LM, O’Boyle EH, Cockburn BS, Yuan Z. Meta-analysis of coefficient alpha: a reliability generalization study. J Manag Stud. 2018;55(4):583-618. https://doi.org/10.1111/joms.12328

53. Ferguson CJ. An effect size primer: a guide for clinicians and researchers. Prof Psychol: Res Pract. 2009;40(5):532-38. https://doi.org/10.1037/a0015808

54. Gravetter F, Wallnau L. Essentials of statistics for the behavioral science. 8th ed. Belmont (CA): Wadsworth; 2014.

55. American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington (dc): American Educational Research Association; 2014.

56. Ferrando PJ, Lorenzo-Seva U. El análisis factorial exploratorio de los ítems: algunas consideraciones adicionales [Exploratory factor analysis of the items: some additional considerations]. An. Psicol. 2014;30(3),1170-5. https://doi.org/0.6018/analesps.30.3.199991

57. Gu F, Little TD, Kingston NM. Misestimation of reliability using coefficient alpha and structural equation modeling when assumptions of tau-equivalence and uncorrelated errors are violated. Methodol. 2013;9(1):30-40. https://doi.org/10.1027/1614-2241/a000052

58. Lloret-Segura S, Ferreres-Traver A, Hernández-Baeza A, Tomás-Marco I. El análisis factorial exploratorio de los ítems: una guía práctica, revisada y actualizada. An. Psicol. 2014;30(3):1151-69. https://doi.org/10.6018/analesps.30.3.199361

59. Charter RA, Feldt LS. Meaning of reliability in terms of correct and incorrect clinical decisions: the art of decision making is still alive. J Clin Exp Neuropsychol. 2001;23(4):530-7. https://doi.org/10.1076/jcen.23.4.530.1227

60. Glover TA, Albers CA. Considerations for evaluating universal screening assessments. J Sch Psychol. 2007;45(2):117-35. https://doi.org/10.1016/j.jsp.2006.05.005

61. Pérez, D. Revisión del concepto de causalidad en el marco del análisis factorial confirmatorio. Rev Iberoam Diagn Eval Psicol. 2020;54(1):103-17. https://doi.org/10.21865/ridep54.1.09

Author notes

* Corresponding author: colasanti.el@gmail.com