Neonatal intensive care parent satisfaction: a multicenter study translating and validating the Italian EMPATHIC-N questionnaire
Italian Journal of Pediatrics volume 44, Article number: 5 (2018)
In Neonatal Intensive Care Units (NICUs), parent satisfaction and their experiences are fundamental to assess clinical practice and improve the quality of care delivered to infants and parents. Recently, a specific instrument, the EMpowerment of PArents in THe Intensive Care-Neonatology (EMPATHIC-N), has been developed in the Netherlands. This instrument investigated different domains of care in NICUs from a family-centered care perspective. In Italy, no rigorous instruments are available to evaluate parent satisfaction and experiences in NICU with family-centered care. The aim of this study was to translate and validate the EMPATHIC-N instrument into Italian language measuring parent satisfaction.
A psychometric study was conducted in nine Italian NICUs. The hospitals were allocated across Italy: four in the North, four in Central region, one in the South. Parents whose infants were discharged from the Units were enrolled. Parents whose infants died were excluded.
Back-forward translation was conducted. Twelve parents reviewed the instrument to assess the cultural adaptation; none of the items fell below the cut-off of 80% agreement. A total of 186 parents of infants who were discharged from nine NICUs were invited to participate and 162 parents responded and returned the questionnaire (87%). The mean scores of the individual items varied between 4.3 and 5.9. Confirmatory factor analysis was performed and all factor loadings were statistically significant with the exception of item ‘Our cultural background was taken into account’. The items related to overall satisfaction showed a higher trend with mean values of 5.8 and 5.9. The Cronbach’s alpha’s (at domain level 0.73-0.92) and corrected item-total scale correlations revealed high reliability estimates.
The Italian EMPATHIC-N showed to be a valid and reliable instrument measuring parent satisfaction in NICUs from a family-centered care perspective. Indeed, it had good psychometric properties, validity, and reliability. Furthermore, this instrument is fundamental for further research and internationally benchmarking.
Patient satisfaction has become an important quality indicator in healthcare [1,2,3]. Patient opinions reflect their personal preferences, expectations and experience on the care received. Their perceptions contribute to measure the quality of the delivered care offering opportunities of improvement . In Neonatal Intensive Care Units (NICU), parent satisfaction and their experiences become fundamental to assess clinical practice and improve the care of infants and parents [5,6,7,8]. Furthermore, healthcare staff should deliver care recognizing the needs and the experiences of the family [9, 10].
Several neonatal parent satisfaction instruments have been published but the majority were not developed following the standards of validity, reliability or were not conducted with methodological rigor [7, 11]. Recently, a specific instrument for parent satisfaction in NICU has been developed and validated following psychometric measures in the Netherlands . The EMpowerment of PArents in THe Intensive Care-Neonatology (EMPATHIC-N) investigates different domains of NICU care from a family-centered care (FCC) perspective and measures the delivered care as perceived by parents. This instrument covers a wide range of care aspects; therefore, it could be used in every NICU, even in those where FCC is not completely applied.
Various definitions of FCC are available. Overall, FCC can be summarized as a clinical practice approach including the following principles: respect and understanding; provision of information and education to family; coordination of care attained by means of effective communication; physical and emotional support and involvement of parents in decision making and in care .
In Italy, no rigorous instruments are available to evaluate parent satisfaction and experiences in NICU with FCC. Furthermore, considering that FCC was ranked as the second research priority domain in NICUs across Europe , and was identified as a priority in pediatric critical care research by international experts , a validated instrument to measure outcomes and benchmark parent satisfaction is needed.
A validated parent satisfaction instrument offers the opportunity to compare and optimize FCC in NICUs from a broad perspective and might contribute to share FCC outcomes among NICUs nationally and internationally. Therefore, the aim of this study was to translate, cultural adapt, and validate the original Dutch EMPATHIC-N instrument into Italian language.
This multi-center study used a psychometric design with the rigorous approach to translate and culturally adapt the original Dutch EMPATHIC-N instrument.
The study was conducted in nine Italian level III NICUs. The NICUs were located in different types of hospitals; four academic children’s hospitals; one university hospital; four general hospitals. The hospitals were allocated across Italy: four in the North, four in Central region, one in the South. The number of beds in NICUs ranged between 6 and 10. In 2012, the infant discharge rate varied between 146 and 499 (mean 302.85; SD109.7) and the mean discharge rate of very-low-birthweight infants (birth weight < 1.5 kg) ranged between 22 and 154 (mean85.14; SD51.1).
Data were collected between November 2013 and August 2014.
Study participants were parents whose children were discharged from NICU or transferred to a high dependency neonatal ward. Only parents able to read and understand the Italian language were included. Parents with multiple births received only one Italian EMPATHIC-N instrument if all their infants were discharged. Parents whose infants died were excluded. Parents who had been already enrolled were excluded in case their infant was readmitted in NICU.
The medical ethical review board of the Bambino Gesù Children’s Hospital approved the study (protocol n. 604.13) and the other centers obtained similar ethical approval. Parents were informed regarding the study objectives and were asked to provide written informed consent.
Parents were enrolled and received the EMPATHIC-N instrument on the day of discharge or within the first 3 days after discharge. A demographic sheet was used by researchers to collect information regarding infants (e.g. gestational age, and birth weight) and parents (nationality, and education level). A study number was sequentially assigned to the enrolled parents to ensure anonymity. Parents who did not responded received a phone call after 2 weeks. Parents who completed the EMPATHIC-N could deliver the survey in a sealed envelope in a special box on the wards or returned it via mail.
The EMPATHIC-N instrument
The Dutch EMPATHIC-N is a parent satisfaction questionnaire composed of 57 items concerning neonatal intensive care and is divided into five domains: information, care and treatment, parental participation, organization, and professional attitude. The rating scale of the items is a 6-point scale; 1 “certainly no” to 6 “certainly yes”. The instrument measures also the overall satisfaction through four questions asking to parents if they would recommend the NICU to others, if they would come back to the unit if needed, and about the physicians’ and nurses’ overall performances (10-point rating scale). The instrument has a demographic section and a free space to allow parents to write their experiences . Congruent validity, reliability, internal consistency, non-differential validity were performed by the developers of the Dutch EMPATHIC-N and showed satisfactory results .
The translation of the Italian version of EMPATHIC-N followed a structured method consisting a 10-step process, including forward and backward translation . Two independent translators presented translations of the instrument in Italian, and backward in Dutch and the instrument developer assessed the faithfulness of the translations to the original version. Cognitive debriefing was performed with 12 parents (one non-native speaker) whose infants were hospitalized in two participating NICUs and they were asked to review the translated version of the EMPATHIC-N instrument. The final version was proofread by the authorsto check any spelling error, and by clinical nurses to assess its cultural consistence.
Participants’ socio-demographic characteristics were analyzed using descriptive statistics. As a preliminary analysis the items normality was examined computing the skewness and kurtosis indices, values of these indices higher than 1.0attested for the non-normality of the item. Confirmatory factor analysis (CFA) was conducted to examine the validity of the EMPATHIC-N. In line with the Dutch EMPATHIC-N instrument , five latent variables have been defined: information measured by items from Q1 to Q12, Care & Treatment from Q13 to Q29, Parental Participation from Q30 to Q37, Organization from Q38 to Q45, and Professional Attitude from Q46 to Q57. The goodness of the factor structure was evaluated considering the following fit indices: (a) chi square, (b) Comparative Fit Index  (CFI;), (c) Root Mean Square Error of Approximation (RMSEA), and (d) Standardized Root Mean Square Residual (SRMR) [15, 16]. According to a multi-faced evaluation of the fit a model has a good fit if CFI is higher than 0.95,RMSEA is lower than 0.06,and SRMR is below 0.08 [17,18,19]. Reliability of each factor was examined by internal consistency computing the Cronbach’s alpha coefficient. Congruent validity was examined by correlating the domains of the questionnaire with the four overall satisfaction indicators. The data were analyzed using IBM SPSS (version 15.0; Chicago, IL) and the statistical modelling program Mplus 7.11 . The level of significance was set at < 0.05.
Translation and cultural adaptation of EMPATHIC-N
The Italian translation of EMPATHIC-N was conducted as described previously. Few suggested modifications were needed: some verb tenses and some terms were changed to better adapt the instrument to the Italian syntax and vocabulary. Only 5 items did not reach complete consent by the 12 parents reviewing the instrument but none fell below the cut-off of 80% agreement and needed revision. The statement “the correct medication was always given on time” was modified in “the right drugs were always administrated on time” with the instrument developer approval, considering “drugs” as a synonym for medication.
Characteristics of parents and infants
During the study period, 186 parents of infants who were discharged from nine NICUs were invited to participate. A total of 162 parents responded and returned the questionnaire (87%). The instrument was completed by mothers (n = 70, 43.2%), fathers (n = 13. 8%) and by both (n = 79,48.8%). Characteristics of the infants and parents are presented in Table 1.
Regarding the gender of infants, there wasa slight majority of male infants (n = 85, 53.5%). A total of 49 infants (30.2%) required invasive ventilation, 48 infants (29.6%) underwent both invasive and non-invasive ventilation, 38 (23.5%) underwent non-invasive ventilation, and 24 (14.8%) infants have not been subjected to any mechanical ventilation technique.
The characteristics of the non-responders group were tested among the responders group on five variables of the infants (sex, length of stay, gestational age, birth weight, length of stay and day, and type of mechanical ventilation) and two variables of parents (nationality and education level).The non-responders group did not differ from the responders group on these variables (p > 0.05).
Validity and reliability of the Italian version of EMPATHIC-N
The mean scores of the individual items varied between a minimum of 4.3 (Q46) and a maximum of 5.9 (Q39). The items related to overall satisfaction (Q58 and Q59) showed a higher trend with mean values of 5.8 (Q58) and 5.9 (Q59) (Additional file 1: Table S1).
The analysis of the correlation between the items of the Italian Empathic-N version and the overall scores obtained from physicians and nurses showed that most of the items correlate with these two assessments overall (Additional file 1: Table S1).
Confirmatory factor analysis was performed using Mean and Variance-adjusted Maximum Likelihood (MVML) as method of estimation and the items were specified as categorical, since almost all items were not normally distributed. The model examined fits the data well χ2 (1529) = 1937.38; p < .001; RMSEA = .041 (CI: 0.035 0.046) p = 1.000; CFI = .97; WRMR = 1.057. All the factor loadings were statistically significant with the only exception of item Q55 (‘Our cultural background was taken into account’). Since all factors were highly correlated (Additional file 2: Table S2), the model was re-specified by defining a second order factor measured by the five domains of the questionnaire (Information, Care & Treatment, Parental Participation, Organization, Professional Attitude). Even in this case the model fits very well the data χ2(1534) = 1956.83; p < .001; RMSEA = .041 (CI: 0.036 0.047) p = 1.000; CFI = .97; WRMR = 1.073. Table 2 presents the factor loadings per item.
The examination of the Cronbach’s alphas and the corrected item-total scale correlations confirmed the reliability of all factors measured by the Italian version of EMPATHIC-N. Specifically, the Cronbach’s alpha estimates ranged between 0.73-0.92 (Table 3).
Congruent validity was obtained by correlating the domains of the questionnaire with the four overall satisfaction indicators. All domains significantly correlated with each of the four overall satisfaction indicators (Table 4).
The non-differential validity of the Italian version of EMPATHIC-N questionnaire was assessed by calculating the standardized mean difference, Cohen’s d, between the domains and four population variables (Table 5).
Results showed that no differences were statistically significant with the only exception of parents of infants with mechanical ventilation who had significantly lower mean values in the domain “professional attitude”.
The present study translated and adapted the Dutch version of the EMPATHIC-N instrument into Italian. This instrument aims to assess NICU parent satisfaction from a FCC perspective. This study showed that the Italian version has good psychometric properties, validity, and reliability. The internal consistency of all the domains showed a Cronbach’s alpha > 0.7 demonstrating the instrument as sufficiently reliable. The congruent validity of scales and correlations among factors showed adequate estimates. The mean scores of the last three items in information domain (communication, clarity, and information sharing) are all over 5. However, a low factor weight (less than 0.1) was observed in the confirmatory factor analysis. These results could suggest that knowledge sharing is not a significant factor in the information domain. This could be considered a weakness in the Italian EMPATHIC-N and would need further testing with a larger response group.
Our study demonstrated a high satisfaction rate on physicians and nurses attitude. We speculate that this evaluation may be independent by the unit organization and environment. The professional behavior of the staff does not depend directly by the NICU’s layout or by lack of service-oriented organizational culture. Even though the environment plays an important role for the parent satisfaction, the behavior of individual staff and the quality of parent-provider relationship still influence parent’s experience .
Our study was conducted in different hospitals located across Italy to recruit a representative sample of the country. The NICUs involved in the validation study had different organization and delivery of care. All NICUs practice a certain degree of FCC such as opening of unit to the parents, their involvement in the decisions and practical care or parental support in case of emergency. Although FCC was practiced at various levels in the participating NICUs, we did not consider the different FCC practice levels as a bias for the validation of the instrument. In fact the instrument represents a broad range of items related to clinical practice including principles of FCC. After all, we aim to validate the EMPATHIC-N in order to have a validated instrument to benchmark clinical practice in Italian NICUs including FCC practice levels. In Italy, the North is as prosperous as central and northern Europe, but the South is much poorer economically [22, 23]. Italy has been a country characterized by internal cultural differences mostly varying from region to region. Furthermore, in the last years, a vast influx of migrants has increased the cultural and ethnic diversity. Perhaps the mix of both aspects explains the fact that the only item not statistically significant was ‘Our cultural background was taken into account’. Indeed, it might not be well understood by parents, and therefore it was changed, with the instrument developer approval, in ‘Our cultural background (both national and local) was taken into account’. The Brazilian adaptation of the EMPATHIC-N instrument had a similar issue. However the authors of the study initially decided to exclude this statement .Instead, we believe that the cultural aspect is fundamental to meet all patient needs in every context. Culture may be a barrier influencing the levels of patient satisfaction and might influence the level of benchmarking satisfaction outcomes [25, 26].
Our instrument was delivered to parents at NICU discharge day or in the following 3 days. Parents could return the completed questionnaire either the same day or after taking the instrument home and return by post. This timing differed by Latour and colleagues’ study in which the instrument was mailed to the parents 3-4 weeks after the discharge . We opted for a different timing and a face-to-face recruitment to achieve the highest possible response rate considering Italian organizational issues and culture. Indeed, our study obtained a high response rate (87%). Likewise, the authors of the study that validated the Italian version of EMPATHIC instrument for Pediatric Intensive Care Units (PICU) distributed the instrument to parents at PICU discharge and obtained also a high response rate (79%) compared with postal recruitment [27, 28]. In another Italian study, parent satisfaction was evaluated during hospital staying .
The NICU nurses were involved in the distribution of the instrument to the parents. In this way, the staff was stimulated to understand the importance of FCC, realize the possible need of change in their unit and find out the response rates in order to improve. We believe that staff motivation is a prerequisite to address changes in the organization and cultural background . Furthermore, the present study is the first of a series to investigate FCC in Italian NICUs. One of the next phases will be focused on NICU staff perception of FCC.
A limitation of our study is that it included only participants who could understand the Italian language and they may be not representative of the entire population. The non-Italian speakers could have been found to be less satisfied with medical care [27, 28]. However, we considered that their satisfaction should be explored using instruments culturally developed and translated in their language. Thus, cultural and linguistic aspects strongly influence outcome expectation. A second limitation is that we did not perform a test-retest reliability to not burden parents with two instruments as Latour and colleagues reported . Finally, the study timing was long, but every NICU began in a different time the enrollment of parents and often they needed to be excluded.
In NICU, the positive or negative experiences of parents may influence the lives of the parents and infants over time and healthcare providers might not have sufficient data to increase the awareness of the consequences of a NICU admission. Thus, assessing NICU parent satisfaction is crucial to inform new directions for change. The Dutch EMPATHIC-N is a validated instrument with sufficient psychometric properties designed to assess parent satisfaction with FCC in NICU. Our study translated and validated this instrument into Italian to provide a benchmark outcome measure. Thus, Italian NICUs have now a valid, reliable instrument to measure parent satisfaction regarding FCC. This is fundamental for further research considering that FCC is one the most important issues identified by European researchers .
Confirmatory factor analysis
Comparative Fit Index
EMpowerment of PArents in THe Intensive Care-Neonatology
Neonatal Intensive Care Unit
Pediatric Intensive Care Unit
Root Mean Square Error of Approximation
Standardized Root Mean Square Residual
Grol R. Improving the quality of medical care: building bridges among professional pride, payer profit, and patient satisfaction. JAMA. 2001;286:2578–85.
Ygge BM, Arnetz JE. Quality of paediatric care: application and validation of an instrument for measuring parent satisfaction with hospital care. Int J Qual Health Care. 2001;13:33–43.
Moumtzoglou A, Dafogianni C, Karra V, Michailidou D, Lazarou P, Bartsocas C. Development and application of a questionnaire for assessing parent satisfaction with care. Int J Qual Health Care. 2000;12:331–7.
Latour JM, Haines C. Families in the ICU: do we truly consider their needs, experiences and satisfaction? Nurs Crit Care. 2007;12:173–4.
Latour JM, van Goudoever JB, Hazelzet JA. Parent satisfaction in the pediatric ICU. Pediatr Clin N Am. 2008;55:779–90.
Berwick DM. A User’s manual for the IOM’s “quality chasm” report. Health Aff (Millwood). 2002;21:80–90.
Latour JM, Duivenvoorden HJ, Hazelzet JA, van Goudoever JB. Development and validation of a neonatal intensive care parent satisfaction instrument. Pediatr Crit Care Med. 2012;13:554–9.
Committee on hospital care and institute for patient- and family-centered care. Patient- and family-centered care and the pediatrician’s role. Pediatrics. 2012;129:394–404.
Browne JV, Talmi A. Family-based intervention to enhance infant–parent relationships in the neonatal intensive care unit. J Pediatr Psychol. 2005;30:667–77.
Latour JM, Hazelzet JA, Duivenvoorden HJ, van Goudoever JB. Perceptions of parents, nurses, and physicians on neonatal intensive care practices. J Pediatr. 2010;157:215–220.e3.
Conner JM, Nelson EC. Neonatal intensive care: satisfaction measured from a Parent’s perspective. Pediatrics. 1999;103(Suppl E1):336–49.
Wielenga JM, Tume LN, Latour JM, van den Hoogen A. European neonatal intensive care nursing research priorities: an e-Delphi study. Arch Dis Child - Fetal Neonatal Ed. 2015;100:F66–71.
Tume LN, Coetzee M, Dryden-Palmer K, Hickey PA, Kinney S, Latour JM, et al. Pediatric critical care nursing research priorities-initiating international dialogue. Pediatr Crit Care Med. 2015;16:e174–82.
Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health J. 2005;8:94–104.
Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107:238–46.
Steiger JH. Structural model evaluation and modification: an interval estimation approach. Multivar Behav Res. 1990;25:173–80.
Joreskog KG, Sörbom D. LISREL8 User’s reference guide. Chicago: Scientific Software International; 1993.
Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model Multidiscip J. 1999;6:1–55.
Browne MW, Cudek R. Alternative ways of assessing model fit. In: Testing structural equation models. Newbury Park: Sage. K.A. Bollen & J.S. Long; 1993.
Muthén LK, Muthén BO. Mplus User’s Guide. 6th ed. Los Angeles: Muthén & Muthén; 1998.
Worchel FF, Prevatt BC, Miner J, Allen M, Wagner L, Nation P. Pediatrician’s communication style: relationship to parent’s perceptions and behaviors. J Pediatr Psychol. 1995;20:633–44.
Lynn R. In Italy, north–south differences in IQ predict differences in income, education, infant mortality, stature, and literacy. Intelligence. 2010;38:93–100.
Peracchi F. Height and Economic Development in Italy 1730-1980. https://www.scribd.com/document/194258875/Franco-Peracchi-Height-and-Economic-Development-in-Italy-1730-1980 Accessed 28 Oct 2016.
Gomez DBCA, Vidal SA, Lima LCS. Brazilian adaptation and validation of the Empowerment of parents in the intensive care-neonatology (EMPATHIC-N) questionnaire. J Pediatr. 2017;93:156–64.
Williams DR, Rucker TD. Understanding and addressing racial disparities in health care. Health Care Financ Rev. 2000;21:75–90.
Capell J, Dean E, Veenstra G. The relationship between cultural competence and ethnocentrism of health care professionals. J Transcult Nurs Off. 2008;19:121–5.
Wolfler A, Giannini A, Finistrella MR, Salvo I, Calderini E, Frasson G, et al. EMpowerment of PArents in THe intensive care questionnaire: translation and validation in Italian PICUs. Pediatr Crit Care Med. 2017;18:e77–85.
Sitzia J, Wood N. Response rate in patient satisfaction research: an analysis of 210 published studies. Int J Qual Health Care. 1998;10:311–7.
De Bernardo G, Svelto M, Giordano M, Sordino D, Riccitelli M. Supporting parents in taking care of their infants admitted to a neonatal intensive care unit: a prospective cohort pilot study. Ital J Pediatr. 2017;43:1–11.
We thank the staff of the participating NICUs who embraced the research and motivated parents to respond; Antonella Leto, nurse director of Sant’Eugenio Hospital, and Ercole Vellone, PhD, RN, Assistant Professor in Nursing Science Department of Biomedicine and Prevention, University of Rome “Tor Vergata” for their collaboration in ethical and methodological aspects.
We acknowledge all the parents who agreed to participate in the study.
List of members of the collective name The Italian Empathic-N Study Group: Luca Di Sarra (Neonatal Intensive Care Unit, Catholic University of Rome, Italy), Gina Ancora (Neonatal Intensive Care Unit, “Infermi” Hospital, Rimini, Italy), Sandra Lazzari (Neonatal Intensive Care Unit, “Infermi” Hospital, Rimini, Italy), Marilena Galeazzo (Nursing Management Department, Hospital of Padua, Italy), Elisabetta Lolli (Department of Woman's and Child's Health, Neonatal Intensive Care Unit, Hospital of Padua, Italy), Mariella Frongia (Division of Neonatology and Neonatal Intensive Care Unit, “V. Buzzi” Children’s Hospital ASST-Fatebenefratelli-Sacco, Milan, Italy), Enrica Lupo (Division of Neonatology and Neonatal Intensive Care Unit, “V. Buzzi” Children’s Hospital ASST-Fatebenefratelli-Sacco, Milan, Italy), Daniela Ammazzini (Nursing Management Department “Meyer” Children’s Hospital, Florence, Italy), Silvia Prunecchi (Neonatal Intensive Care Unit “Meyer” Children’s Hospital, Florence, Italy), Rosanna Bruno (Neonatal Intensive Care Unit, San Carlo Hospital, Potenza, Italy), Antonella Raimondi (Neonatal Intensive Care Unit, San Carlo Hospital, Potenza, Italy), Serena Rovei (Neonatal Intensive Care Unit, Department of Public Health and Pediatric, Turin University, Turin, Italy), Liliana Vagliano (Department of Public and Pediatric Health Sciences, University of Turin, Turin, Italy), Daniela Sebastianelli (Neonatal Intensive Care Unit, Sant’Eugenio Hospital, Rome, Italy), Loredana Bonafede (Neonatal Intensive Care Unit, Sant’Eugenio Hospital, Rome, Italy).
This study was funded by the Centre of Excellence for Nursing Scholarship-IPASVI Nursing College of Rome.
Availability of data and materials
The datasets generated and analyzed during the present study are available from the corresponding author.
Ethics approval and consent to participate
The medical ethical review board the Bambino Gesù Children’s Hospital IRCCS approved the study (protocol n. 604.13). Informed consent was taken by each parent.
Consent for publication
The participating parents consented also for data publication.
The authors declare no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Dall’Oglio, I., Fiori, M., Tiozzo, E. et al. Neonatal intensive care parent satisfaction: a multicenter study translating and validating the Italian EMPATHIC-N questionnaire. Ital J Pediatr 44, 5 (2018). https://doi.org/10.1186/s13052-017-0439-8