Abstract
The National Educational Panel Study surveys, among others, cohort samples of Kindergarten children, students in Grade 5 and students in Grade 9. This paper gives details on the applied sampling designs to realize these samples. The implemented designs cover indirect sampling procedures, stratification, and two-stage cluster sampling designs. Details on the derivation of design weights and their successive adjustments yielding nonresponse adjusted weights are presented. The considered adjustments refer to institutional and individual nonresponse and take into account clustering on institutional level by specifying random effects within the corresponding regression models. For Kindergarten children, the empirical results show that a child’s living conditions (with both or only one parent present) influence the participation propensity. For students in secondary schools, a language other than German spoken at home as well as competencies in math and German influence the decision to participate in the panel study. A discussion of strategies to provide cross-sectional and longitudinal weights is provided.
Zusammenfassung
Das Nationale Bildungspanel erhebt unter anderem Kohortenstichproben von Kindergartenkindern, Schülern in der Klasse 5 und Schülern in der Klasse 9. Dieser Beitrag beschreibt detailliert die Stichprobenpläne dieser Kohorten. Die eingesetzten Stichprobenverfahren umfassen dabei indirekte Stichprobenziehung, Schichtung und zweistufige Klumpenstichproben. Im Rahmen der Erstellung der Gewichte werden im ersten Schritt Designgewichte hergeleitet. Im zweiten Schritt werden diese adjustiert, um für Teilnahmeverweigerungen innerhalb der Bruttostichprobe zu kompensieren. Diese Anpassungen kommen sowohl bei der Teilnahmeverweigerung auf institutioneller als auch auf individueller Ebene zur Anwendung. Die Klumpung von Individuen in Institutionen wird durch Zufallseffekte in den Regressionen berücksichtigt. Die empirischen Ergebnisse zeigen, dass die Teilnahme von Kindergartenkindern signifikant dadurch beeinflusst wird, ob das Kind mit beiden Eltern oder nur mit einem Elternteil zusammenlebt. Bei Schülern hingegen wird die Teilnahmebereitschaft durch die zu Hause gesprochene Sprache (Deutsch oder eine andere Sprache), sowie Kompetenzen in Mathematik und Deutsch maßgeblich beeinflusst. Abschließend werden Möglichkeiten zur Bereitstellung von Quer- und Längsschnittgewichten für Folgewellen der Panelerhebungen dargestellt.
Similar content being viewed by others
Notes
First versions of corresponding Scientific Use Files (SUF) for these starting cohorts were released in September and October 2012. See doi:10.5157/NEPS:SC2:1.0.0, doi:10.5157/NEPS:SC3:1.0.0 and doi:10.5157/NEPS:SC4:1.0.0.
The replacement strategy established for Kindergarten institutions was very effective. That is, all Kindergartens refusing to participate could be replaced, see Sect. 2.4 of this paper.
Regular schools are all “allgemeinbildende Schulen”, that is, schools of general education according to the definition of Kultusministerkonferenz (2014). Thus, regular or mainstream schools include all school types listed here with the exception of the school type of Förderschulen (FS).
Note that the scientific use files thus so far are not including a Federal State variable and provide the school type based on reported answers.
In most special schools, students in Grade 7, 8, and 9 are educated together. That is, in the majority of cases the number of Grade 9 students is hard to report, whereas the total number of students in Grades 7 to 9 is mostly available. Therefore, the number of Grade 9 students is approximated by one third of the reported number of students in Grades 7–9.
Here, the design weights for Kindergarten institutions of SC2 are an exception. Due to the indirect sampling approach applied, design weights for Kindergarten institutions cannot directly be derived as the inverse of the inclusion probabilities of sampling units and are computed differently. For details see Sect. 2.2.
Note that some of the schools participating in SC3 also participate in SC4.
In SC3, 5191 students belong to regular schools, 584 students to special schools, and 290 to the migrants supplement. In SC4, 15327 students belong to regular schools and 1286 students to special schools. Differences between these numbers and the case numbers reported in the SUF data are due to the different time points at which panel content was obtained and Wave 1 surveys were conducted. Furthermore, panel consent could be withdrawn during later survey stages so that subsequently complete cases had to be deleted form the SUF data.
The models were also controlled for the following variables: strata (\(h=1,\ldots,7\)), nationality (German, non-German), dyslexia (yes, no), attention deficit hyperactivity disorder (yes, no).
To account for the multilevel structure, differences in probabilities resulting from changing X 1 to X 2 are estimated via simulation (with simulation sample size \(S = 10^6\)) as follows
$$\frac{1}{S}\sum_{s=1}^S \Phi\left(X_1\beta^{(s)}+\alpha^{(s)}\right)-\Phi\left(X_2\beta^{(s)}+\alpha^{(s)}\right)\quad,$$where \(\beta^{(s)}\) and \(\alpha^{(s)}\), \(s=1,\ldots,S\) denote a sample from the estimated asymptotic distribution. The corresponding quantiles of the trajectory \(\{\Phi(X_1\beta^{(s)}+\alpha^{(s)})-\Phi(X_2\beta^{(s)}+\alpha^{(s)})\}_{s=1}^S\) serve as estimates for 95 % confidence intervals as well.
Note that no adjustments are necessary for the group of never-participating individuals.
The NEPS sample SC2 of Kindergarten children is not stratified. Thus, weights and adjustment factors are here independent of the subindex h.
References
Arora A, Foy P, Martin MO, Mullis IVS (eds) (2009) TIMSS Advanced 2008 Technical Report. TIMSS & PIRLS International Study Center Lynch School of Education Boston College, Chestnut Hill
Aßmann C, Steinhauer HW, Kiesl H, Koch S, Schönberger B, Müller-Kuller A, Rohwer G, Rässler S, Blossfeld HP (2011) Sampling designs of the National Educational Panel Study: challenges and solutions. In: Blossfeld HP, Roßbach HG, von Maurice J (eds) Education as a lifelong process: Zeitschrift für Erziehungswissenschaft (Vol. 14). VS Verlag für Sozialwissenschaften, Wiesbaden, pp 51–65
Aßmann C, Steinhauer HW, Rässler S (2012) Aspekte der Stichprobenziehung in der erziehungswissenschaftlichen Forschung. In: Maschke S, Stecher L (eds) Enzyklopädie der Erziehungswissenschaft Online (EEO), Fachgebiet Methoden der empirischen erziehungswissenschaftlichen Forschung, Beltz Juventa, Weinheim und Basel, pp 1–15
Baker R, Brick JM, Bates NA, Battaglia M, Couper MP, Dever JA, Gile KJ, Tourangeau R (2013) Summary Report of the AAPOR task force on non-probability sampling. J Surv Stat Methodol 1(2):90–143
Bates D, Maechler M, Bolker B (2012) lme4: Linear mixed-effectsmodels using S4 classes. http://CRAN.R-project.org/package=lme4. Accessed 3 June 2014
Blossfeld HP, Roßbach HG, von Maurice J (eds) (2011) Education as a lifelong process: The German National Educational Panel Study (NEPS) [Special Issue]: Zeitschrift für Erziehungswissenschaft (Vol 14). VS Verlag für Sozialwissenschaften, Wiesbaden
Brick JM (2013) Unit Nonresponse and weighting adjustments: a critical review. J Off Stat 29(3):329–353
Brick JM, Kalton G (1996) Handling missing data in survey research. Stat Methods Med Res 5(3):215–238
Elff M (2012) memisc: Tools for management of survey data, graphics, programming, statistics, and simulation. http://CRAN.R-project.org/package=memisc. Accessed 3 June 2014
Elliott MR (2009) Combining data from probability and non-probability samples using pseudo-weights. Surv Pract 2(6):1–7
Kalton G (1986) Handling wave nonresponse in panel surveys. J Off Stat 2(3):303–314
Kalton G, Flores-Cervantes I (2003) Weighting methods. J Off Stat 19(2):81–97
Kalton G, Kasprzyk D (1986) The treatment of missing survey data. Surv Methodol 12:1–16
Kiesl H (2010) Selecting kindergarten children by three stage indirect sampling. In: American Statistical Association (ed) Proceedings of the survey research methods section, Alexandria, pp 2730–2738. http://www.amstat.org/sections/srms/proceedings/y2010/Files/307400_63544.pdf. Accessed 9 Jan 2012
Kish L (1990) Weighting: Why, when, and how? In: American Statistical Association (ed) Proceedings of the survey research methods section, pp 121–130. https://www.amstat.org/sections/SRMS/Proceedings/papers/1990_018.pdf. Accessed 3 April 2014
Kish L (1992) Weighting for unequal P i . J Off Stat 8(2):183–200
Kultusministerkonferenz (2014) Definitionenkatalog zur Schulstatistik 2014. http://www.kmk.org/fileadmin/pdf/Statistik/Dokumentationen/Defkat2014.pdf. Accessed 6 May 2014
LaRoche S, Zuehlke O, Joncas M (2009) TIMSS Advanced 2008 Samping. In: Arora A, Foy P, Martin MO, Mullis IVS (eds) TIMSS Advanced 2008 Technical Report, TIMSS & PIRLS International Study Center Lynch School of Education Boston College, Chestnut Hill, MA, pp 51–90
Lavallée P (2007) Indirect sampling. Springer, New York and London
Lepkowski JM (1989) Treatment of Wave Nonresponse in Panel Surveys. In: Kasprzyk D, Duncan GJ, Kalton G, Singh M (eds) Panel surveys, Wiley series in probability and mathematical statistics/Applied probability and statistics, Wiley, New York, pp 348–374
Little R, Vartivarian S (2003) On weighting the rates in non-response weights. Stat Med 22(9):1589–1599
Lynn P, Kaminska O (2010) Weighting strategy for understanding society: understanding society working paper no. 2010–05. http://research.understandingsociety.org.uk/publications/working-paper/2010-05.pdf. Accessed 5 June 2012
Martin MO, Mullis IVS, Kennedy AM (2007) PIRLS 2006 technical report. TIMSS & PIRLS, International Study Center, Lynch School of Education, Boston College, Chestnut Hill
OECD (2012) PISA 2009 Technical Report. OECD Publishing, Paris
Olson JF, Martin MO, Mullis IV (eds) (2008) TIMSS 2007 technical report. TIMSS & PIRLS International Study Center Lynch School of Education, Boston College, Chestnut Hill
Pfeffermann D (1996) The use of sampling weights for survey data analysis. Statistical Methods in Medical Research 5(3):239–261
R Core Team (2014) R: A language and environment for statistical computing. http://www.R-project.org/. Accessed 5 June 2014
Rosenbaum PR, Rubin DB (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70(1):41–55
Schnell R, Gramlich T, Bachteler T, Reiher J, Trappmann M, Smid M, Becher I (2013) Ein neues Verfahren für namensbasierte Zufallsstichproben von Migranten. Methoden-Daten-Analysen 7(1):5–33
Skinner CJ, D’Arrigo J (2011) Inverse probability weighting for clustered nonresponse. Biometrika 98(4):953–966
Statistisches Bundesamt (2010a) Fachserie 1 Reihe 1.3: Bevölkerung und Erwerbstätigkeit: Bevölkerungsfortschreibung. https://www.destatis.de/DE/Publikationen/Thematisch/Bevoelkerung/Bevoelkerungsfortschreibung2010130107004.pdf. Accessed 10 June 2014
Statistisches Bundesamt (2010b) Fachserie 11 Reihe 1: Bildung und Kultur: Allgemeinbildende Schulen: Schuljahr 2008/09. https://www.destatis.de/GPStatistik/servlets/MCRFileNodeServlet/DEHeft_derivate_00006815/2110100097004.pdf. Accessed 10 Feb 2014
Statistisches Bundesamt (2011) Statistiken der Kinder- und Jugendhilfe: Kinder und tätige Personen in Tageseinrichtungen und in öffentlich geförderter Kindertagespflege am 01.03.2011. https://www.destatis.de/DE/Publikationen/Thematisch/Soziales/KinderJugendhilfe/TageseinrichtungenKindertagespflege5225402117004.pdf. Accessed 10 June 2014
Wolter K (2007) Introduction to variance estimation. Springer, New York
Zinn S (2013) Replication Weights for the Cohort Samples of Students in Grade 5 and 9 in the National Educational Panel Study: NEPS Working Paper 27. https://www.neps-data.de/Portals/0/Working. Accessed 13 Aug 2014
Acknowledgement
This paper uses data from the National Educational Panel Study (NEPS). From 2008 to 2013, NEPS data were collected as part of the Framework Programme for the Promotion of Empirical Educational Research funded by the German Federal Ministry of Education and Research (BMBF). As of 2014, the NEPS survey is carried out by the Leibniz Institute for Educational Trajectories (LIfBi) at the University of Bamberg in cooperation with a nationwide network.
The authors would like to thank two anonymous reviewers and the editor for adding valuable comments and suggestions improving the paper considerably.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix A
Appendix B
Rights and permissions
About this article
Cite this article
Steinhauer, H., Aßmann, C., Zinn, S. et al. Sampling and Weighting Cohort Samples in Institutional Contexts. AStA Wirtsch Sozialstat Arch 9, 131–157 (2015). https://doi.org/10.1007/s11943-015-0162-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11943-015-0162-0
Keywords
- Stratified two-stage cluster sampling
- Indirect sampling
- Unit nonresponse
- Weighting adjustments
- National Educational Panel Study