## References on Design and Analysis of Sample Surveys

by Donna Brogan

June, 1999

#### I. BOOKS--RECENT, OLDER AND CLASSIC

Anderson, R., J. Kaspar and M. Frankel. Total Survey Error. 1979, Jossey-Bass, San Francisco. Survey error beyond, and including, sampling error.

Belson, W. A. Validity in Survey Research. 1986, Gower, Aldershot, England.

Biemer, Paul P., Robert M. groves, Lars E. Lyberg, Nancy A. Mathiowetz and Seymour Sudman (editors). Measurement Errors in Surveys. 1991, John Wiley, New York. Thorough discussion of the various errors in surveys, with 32 contributed chapters. Sources of errors include questionnaire design, nonresponse, interviewers and interviewer-respondent interaction. Modeling of measurement errors. Excellent.

Blankenship, A.B. Professional Telephone Surveys. 1977. McGraw Hill.

Briggs, C. Learning How to Ask. 1986, Cambridge University Press, Cambridge.

Bulmer, Martin and Donald Warwick. Social Research in Developing Countries: Surveys and Censuses in the Third World. 1983. John Wiley, New York, NY.

Casley, D. J. and D. A. Lury. Data Collection in Developing Countries, 2nd edn. 1987. Clarendon Press. Oxford. Overview of survey method with particular issues in developing countries.

Cochran, William G. Sampling Techniques, 3rd Edition. 1977, John Wiley, New York. A classic textbook on the theory of sample surveys, i.e. selection methods, sample size estimation and allocation, analysis of data. Broad coverage of selection techniques. Still used today in many graduate courses in sample survey design and analysis. You need calculus background and a first course in mathematical statistics to read this book. Excellent.

Cochran, W. G. Planning and Analysis of Observational Studies. 1983, John Wiley, New York.

Converse, J. M. Survey Research in the United States. 1987. University of California Press, Berkeley.

Cox, Brenda G. and Steve B. Cohen. Methodological Issues for Health Care Surveys. 1985, Marcel Dekker, New York. Issues related to the National Medical Care and Expenditure Survey, including sampling design, analysis, nonresponse and data collection. Excellent.

Deming, William Edwards. Some Theory of Sampling. 1966. Dover Publications, New York. Republication of the work originally published by John Wiley in 1950. A classic.

Deming, W. Edwards. Sample Design in Business Research. 1960, John Wiley, New York. Oriented to business applications.

Dillman, Donald A. Mail and Telephone Surveys: the Total Design Method. 1978, John Wiley, New York. Comprehensive review of mail surveys, in particular, with some good ideas for improving response rates. Excellent. See also--Salant and Dillman.

Fowler, F. J. Survey Research Methods. 1990, Applied Social Research Methods Series, Volume 1, Sage. Very good.

Fowler, F. J. and T. W. Mangione. Standardized Survey Interviewing, 1990, Sage Publications, Newbury Park, CA. Very good.

Fowler, Floyd J. Improving Survey Questions: Design and Evaluation. Applied Social Research Methods Series, volume 38. 1995. Sage Publications. Thousand Oaks, CA. Easy to read, comprehensive guide to writing survey questions, with examples.

Fuller, Wayne A. Measurement Error Models. 1987, John Wiley, New York.

Frey, J. H. Survey Research by Telephone. 1983. Sage Publications, Beverly Hills. Short and easy to read overview. Good.

Goyder, J. The Silent Majority: Non-Respondents on Sample Surveys. 1986, Polity Press, New York.

Groves, Robert M. and Mick P. Couper. Nonresponse in Household Interview Surveys. 1998. John Wiley, New York. Conceptual framework for considering determinants of survey participation. Some practical suggestions. Excellent.

Groves, Robert M. Survey Errors and Survey Costs. 1989, John Wiley, New York. Costs and errors associated with coverage of population, nonresponse, sampling, estimation, measurement error and mode of data collection. Comprehensive discussions. Excellent.

Groves, Robert M. and R. L. Kahn. Surveys by Telephone: A National Comparison with Perosnal Interviews. 1979, Academic Press, New York. The first (I think) comprehensive book on telephone surveys--a classic. Excellent.

Groves, Robert M., Paul P. Biemer, Lars E. Lyberg, James T. Massey, William L. Nicholls II, Joseph Waksberg. Telephone Survey Methodology. 1988. John Wiley, New York. Comprehensive review of state of art of telephone survey techniques by an international group of survey experts. Chapters on telephone coverage, sampling and weighting procedures for RDD surveys, nonresponse, data quality, computer assisted telephone interviewing, administration of telephone surveys. Excellent.

Guenzel, Pamela J., Tracy R. Berckmans, Charles F. Cannell. General Interviewing Techniques. 1983. Survey Research Center, Institute for Social Research, University of Michigan, Ann Arbor, MI. Self-instructional workbook for telephone and personal interviewing training. This is used by ISR for training its interviewers; it is excellent.

Hansen, M.H, W. N. Hurwitz and W. G. Madow. Sample Survey Methods and Theory, Volumes I (Methods and Applications) and II (Theory). 1953, John Wiley, New York. A classic textbook(s). One of the first sample survey theory books to appear, based on a lot of research done at the U.S. Bureau of the Census. Discusses lots of applications in multi-stage sampling. Excellent.

Hedayat, A. S. and Bikas K. Sinha. Design and Inference in Finite Population Sampling. 1991. John Wiley, New York, NY. Introduction to design/analysis of sample surveys from generalized, mathematical viewpoint. Need background in statistical theory to make your way through this book.

Institute for Social Research (Survey Research Center). Interviewer's Manual. 1976. University of Michigan, Ann Arbor, MI. An oldie but goodie. Contains a lot of detailed information about area probability sampling, mapping, listing, etc. This information is hard to find in print.

Johnson, Norman L. and Harry Smith, Jr. (editors). New Developments in Survey Sampling. 1969, Wiley-Interscience, New York. Not so new anymore, but a good collection of articles. Very good.

Kalton, Graham. Compensating for Missing Survey Data. 1983, Institute for Social Research, Univeristy of Michigan, Ann Arbor, MI. Brief overview of techniques for nonresponse adjustment. Excellent.

Kalton, Graham. Introduction to Survey Sampling. 1983. Sage Publications, series 07-035. Beverly Hills, CA. Short paperback introduction to the field written by a sample survey statistician. Very good.

Kasprzyk, D., G. J. Duncan, G. Kalton and M. P. Singh (editors). Panel Surveys. 1989, John Wiley, New York. Thirty-one chapters which discuss all aspects of panel surveys (measuring change over time), including statistical design, collecting data, estimation of cross-sectional and change parameters, data base management, nonsampling error, panel conditioning, nonresponse adjustments and modeling considerations.

Kish, Leslie. Statistical Design for Research. 1987. John Wiley. New York, NY. A general book on research design, but focuses primarily on sample surveys. An update on Kish's ideas since his classic 1965 book on survey sampling.

Kish, Leslie. Survey Sampling. 1965, John Wiley, New York. A classic textbook on the applied aspects of sample surveys. particularly in design and analysis. Lots of discussion of details of multi-stage sampling not discussed in other books; some very practical guidelines. The notation is a little difficult to follow. Very good.

Krewski, D., R. Platek and J. N. K. Rao (editors). Current Topics in Survey Sampling. 1981. Academic Press. New York, NY.

Lavrakas, P. J. Telephone Survey Methods: Sampling, Selection, and Supervision. 1987, Sage Publications, CA. A brief and comprehensive overview of the topic. Good.

Lee, E. S., Ronald N. Forthofer and R. J. Lorimor. Analyzing Complex Survey Data. 1989, Sage, Newbury Park, CA. Brief overview of the issues involved in analyzing data from complex sampale surveys. Excellent. Does not require solid background in statistical theory.

Lehtonen, Risto and Erkki J. Pahkinen. Practical Methods for Design and Analysis of Complex Surveys. 1995. John Wiley, New York. Need sampling theory background and some math-stat.

Lemeshow, Stanley, Davis W. Hosmer, Janelle Klar and Stephen K. Lwanga. Adequacy of Sample Size in Health Studies. 1990. John Wiley, New York. Some formulas and tables to estimate sample size for surveys.

Lessler, Judith T. and William D. Kalsbeek. Nonsampling Error in Surveys. 1992, John Wiley, New York. Sources of nonsampling error, including frame construction, nonresponse, and measurement error. Excellent.

Levy, Paul and Stanley Lemeshow. Sampling of Populations: Methods and Applications, 2nd edition. 1991, John Wiley, New York. Comprehensive introductory text on sample survey design and analysis. Very good. Examples focus on health applications.

Lindstrom, Hakan. Nonresponse Errors in Sample Surveys. 1983. Statistics Sweden, ISBN 91-38-07716-7. Sweden.

Little, Roderick J. A. and Donald B. Rubin. Statistical Analysis with Missing Data. 1987. John Wiley, New York. New statistical techniques for dealing with missing data, whether in surveys or experiments.

Lohr, Sharon. Sampling: Design and Analysis. Duxbury Press, 1999. ISBN 0-534-35361-4. New, haven't seen a copy yet, but recommended to me highly.

Lyberg, Lars, Paul Biemer, Martin Collins, Edit de Leeuw, Cathryn Dippo, Norbert Schwarz, Dennis Trewin. Survey Measurement and Process Quality. 1997. John Wiley, New York. 33 contributed chapters on questionnaire design, data collection, post survey processing and operations, quality assessment and control, and measurement error. Very good.

Madow, W. G., I. Olkin and D. B. Rubin (editors). Incomplete Data in Sample Surveys, Volumes I, II, III. 1983, Academic Press, New York. A very thorough discussion of many techniques for dealing with unit and item nonresponse in sample surveys. Excellent. Volume I is Report and Case Studies. Volume II is Theory and Bibliographies. Volume III is Proceedings of the Symposium.

Mangione, Thomas W. Mail Surveys: Improving the Quality. Applied Social Research Methods Series, Volume 40. 1995. Sage Publications. Thousand Oaks, CA. An easy to read, basic book on the trials and tribulations of mail surveys, including advice and examples.

Namboodiri, N.K. (editor). Survey Sampling and Measurement. 1978. New York, Academic Press.

Rao, P. S. R. S. and Joseph Sedransk (editors). W. G. Cochran's Impact on Statistics. 1984, John Wiley, New York. Most of the contributed chapters are about sample survey issues.

Rossi, P. H., J. D. Wright and A. B. Anderson (editors). Handbook of Survey Research. 1983. Academic Press, Orlando. Comprehensive overview of all aspects of survey research, including sampling strategies and data analysis strategies. Excellent.

Rubin, Donald B. Multiple Imputation for Nonresponse in Surveys. 1987. John Wiley, New York.

Salant, Priscilla and Don Dillman. How to Conduct Your Own Survey. 1994. John Wiley, New York, NY. Have not read, but it should be quite good, based on Dillman's previous book.

Sarndal, Carl-Erik, Bengt Swensson and Jan Wretman. Model Assisted Survey Sampling. 1992. Springer-Verlag. New York, NY. Development of central ideas in survey sampling from unified (general) perspective of unequal probability sampling. Excellent coverage of topics, including nonsampling error, nonresponse, etc. Need good background in statistical theory to make your way through this book.

Scheaffer, Richard L., William Mendenhall and R. Lyman Ott. Elementary Survey Sampling, 5th edition. 1996, Duxbury Press, Wadsworth Publishing, Boston, MA. Low level mathematical presentation of sampling ideas. Many problems. Data diskette. Excellent.

Schuman, Howard and Stanley Presser. Questions and Answers in Attitude Surveys: Experiments on Questions Form, Wording and Context. 1996. Sage Publications, San Diego, CA.

Shah, Babubhai V., Beth G. Barnwell and Gayle S. Bieler. SUDAAN Software for the Statistical Analysis of Correlated Data: Users Manual, Release 7.5. 1997. Research Triangle Institute, NC. One of the major software packages for analysis of sample survey data.

Skinner, C. J., D. Holt and T. M. F. Smith. Analysis of Complex Surveys. 1989. John Wiley, New York, NY. Mathematical treatment of analytical issues for complex sample survey data. Excellent coverage of topics, but challenging to read.

Slonim, Morris James. Sampling in a Nutshell. 1960. Simon and Schuster. New York, NY. Totally nonmathematical overview of sampling. Excellent coverage. A classic oldie, but a goodie.

Stouthamer-Loeber, Maga and Welmoet Bok van Kammen. Data Collection and Management: A Practical Guide. Applied Social Research Methods Series, volume 39. 1995. Sage Publications. Thousand Oaks, CA. Things you need to know in collecting data for a sample survey (or any research project). Data collection & management are not as easy they appear. Easy to read, thorough book.

Sudman, Seymour. Reducing the Cost of Surveys. 1967, Aldine, Chicago. Older, but one of the classic authors on sample surveys.

Sudman, Seymour. Applied Sampling. 1976, Academic Press, New York.

Sudman, Seymour and Norman Bradburn. Response Effects in Surveys, A Review and Synthesis. 1974, Aldine, Chicago. A classic by Sudman and Bradburn.

Sudman, Seymour and Norman Bradburn. Asking Questions. 1982. Jossey-Bass, San Francisco. An update of some of their earlier work on the best way to ask questions in surveys.

Thompson, M.E. Theory of Sample Surveys. 1997. Monographs on Statistics and Applied Probability 74, Chapman & Hall, London and New York.

Thompson, Steven K. Sampling. 1992. John Wiley. New York, NY. First half of book is introduction to standard methods in survey sampling. Second half is discussion of specialized sampling techniques for elusive populations, animal populations, natural resources such as fossil-fuel, rare diseases, etc.

Thompson, Steven K. & George A. F. Seber. Adaptive Sampling. 1996. John Wiley & Sons, New York. Sampling designs where procedure for selecting units to include in sample may depend on values of the variables of interest observed during the survey, e.g. sampling wildlife or rare and clustered populations.

Turner, C. F. and E. A. Martin (editors). Surveying Subjective Phenomena, Volumes I and 2. 1984, Russell Sage Foundation, New York.

Williams, Bill. A Sampler on Sampling. 1978, John Wiley, New York. A very nontechnical and lucid explanation of various types of sampling techniques. Excellent.

Wolter, Kirk W. Introduction to Variance Estimation. 1985, Springer Verlag, New York. A discussion of all the commonly used techniques for variance estimation in complex sample surveys, including random groups, Taylor Series linearization and balanced repeated replications. Excellent.

Yates, F. Sampling Methods for Censuses and Surveys, 3rd Edition. Charles Griffin & Co., London. A theoretical and applied treatment. Lots of examples in agriculture. OK.

### II. JOURNALS THAT PUBLISH ARTICLES ON SAMPLE SURVEY METHODOLOGY

Survey Methodology

Journal of Official Statistics

Public Opinion Quarterly

Survey Research

Journal of the American Statistical Association

Journal of Marketing

Proceedings of the American Statistical Association--Section on Survey Research Methods

Journal of Academy of Marketing Science

Sociological Methods and Research

Journal of the Market Research Society

Journal of Business Research

#### III. SELECTED JOURNAL ARTICLES (OR BOOK CHAPTERS ) ON SAMPLE SURVEY METHODOLOGY

Brock, DB, LA Beckett, JL Bienias (1998). Sample Surveys. In Encyclopedia of Biostatistics, edited by Peter Armitage & Theodore Colton, John Wiley & Sons, New York, volume 5, pgs. 3914-3933. Summary of some of U.S. national health surveys and their history, features of survey design, survey implementation, data analysis, slightly technical.

Brogan, D (1998). Software for Sample Survey Data, Misuse of Standard Packages. In Encyclopedia of Biostatistics, edited by Peter Armitage & Theodore Colton, John Wiley & Sons, New York, volume 5, pgs. 4167-4174. Comparison of SUDAAN to SAS for analysis of complex sample survey data (BRFSS), illustrating how SAS can yield misleading answers. Not technical.

Brogan, Donna, Elaine Flagg, Michael Deming, Ronald Waldman (1994). "Increasing the Accuracy of the Expanded Programme on Immunization's Cluster Survey Design", Annals of Epidemiology, 4(4), 302-311.

Carlson, BL (1998). Software for Sample Survey Data. In Encyclopedia of Biostatistics, edited by Peter Armitage & Theodore Colton, John Wiley & Sons, New York, volume 5, pgs. 4160-4167. Why specialized software is needed, and a review of available software packages. Slightly technical.

Casady RJ & JM Lepkowski. Telephone Sampling. In Encyclopedia of Biostatistics, editors-in-chief Peter Armitage and Theodore Colton, John Wiley, New York, Volume 6, pages 4498-4511. 1998.

Eltinge, JL, VL Parsons, DS Jang (1997). Differences Between Complex-Design-Based and IID-Based Analyses of Survey Data: Examples from Phase I of NHANES III. Stats, 19: 3-12. Similar message to Brogan (1998) reference above, using NHANES-III. Not technical.

Feinleib, M (1998). National Center for Health Statistics (NCHS). In Encyclopedia of Biostatistics, edited by Peter Armitage & Theodore Colton, John Wiley & Sons, New York, volume 4, pgs. 2945-2951. Summary of some of U.S. national surveys. Not technical.

Freeman, Daniel H., Jean L. Freeman, Dwight B. Brock and Gary G. Koch. "Strategies in the Multivariate Analysis of Data from Complex Surveys II: An Application to the United States National Health Interview Survey". International Statistical Review, 44(3), 317-330, 1976.

Freeman, Daniel H. and Dwight B. Brock. "The Role of Covariance Matrix Estimation in the Analysis of Complex Sample Survey Data". In Namboodiri, N. Krishman (editor). Survey Sampling and Measurement. 1978. Academic Press. New York, NY. 121-140.

Freeman, Daniel H. "Modeling Prevalence Rates Based on Complex Sample Survey Data". Journal of Chronic Diseases, 30, 769-779, 1977. Application example in hypertension.

Freeman, Jean L., Daniel H. Freeman and Adrian M. Ostfeld. "Analysis of Data from Successive Complex Sample Surveys, with an Example of Hypertension Prevalence from the United States Health Examination Survey". International Journal of Epidemiology, 12(2), 230-237, 1983.

Graubard, Barry I., Thomas R. Fears and Mitchell H. Gail. "Effects of Cluster Sampling on Epidemiologic Analysis in Population-Based Case-Control Studies", Biometrics, 45, 1989, 1053-1071.

Graubard, Barry I. and Edward L. Korn. "Hypothesis Testing with Complex Survey Data: The Uses of Classical Test Statistics with Particular Reference to Regression Problems", Journal of the American Statistical Association, 88, 1993, 629-641.

Graubard, BI, EL Korn (1996). Survey Inference for Subpopulations. American Journal Epidemiology, 144(1): 102-106. Some warnings and illustrations about analyzing subpopulations of sample survey data, e.g. certain age group, males only, etc. Slightly technical.

Inglis, K. R. M. Groves and S. Heeringa. "Telephone Sample Designs for the U.S. Black Household Population", Survey Methodology. 1987, 13(1), 1-14.

Kalton, Graham and D. W. Anderson. "Sampling Rare Populations", Journal of the Royal Statistical Soceity, Series A. 1986, Vol. 149, Part I, 65-82.

Khurshid A, H Sahai. A Bibliography on Telephone Survey Methodology. J Official Statistics, 1995, 11(3):325-368.

Koch, Gary G., Daniel H. Freeman and Jean L. Freeman. "Strategies in the Multivariage Analysis of Data from Complex Surveys". International Statistical Review, 43(1), 59-78, 1975. This is called the KFF method of data analysis for complex sample surveys.

Korn, EL, BI Graubard (1991). Epidemiologic Studies Utilizing Surveys: Accounting for the Sampling Design. American Journal Public Health, 81(9): 1166-1173. Similar message to reference 5 above, using NHANES-I. Slightly technical.

Korn, EL, BI Graubard (1995). Examples of Differing Weighted and Unweighted Estimates from a Sample Survey. The American Statistician, 49(3): 291-295. Discussion and illustration of the impact upon regression analyses of using or not using the weights when analyzing sample survey data. Slightly technical.

Korn, EL, BI Graubard (1998). Scatterplots with Survey Data. The American Statistician, 52(1): 58-69. Discussion and illustrations about whether to use weights when graphing sample survey data. Slightly technical.

Korn, EL, BI Graubard (1990). Simultaneous Testing of Regression Coefficients with Complex Survey Data: Use of Bonferroni t Statistics. The American Statistician, 44(4): 270-276. Discussion of potential df (degrees of freedom) problems when doing multiple linear regression analyses of sample survey data. Technical.

Korn, EL, BI Graubard (1995). Analysis of Large Health Surveys: Accounting for the Sampling Design. Journal of the Royal Statistical Society, Series A, 158 (Part 2): 263-295. Issues in analyzing sample survey data, written for a statistical audience. Comments on design-based versus model-based philosophies. Technical.

Landis, J. Richard, James Lepkowski, Stephen A. Edlind and Sharon A. Stehouwer. "A Statistical Methodology for Analyzing Data from a Complex Survvey: The First National Health and Nutritional Examination Survey". National Center for Health Statistics Vital and Health Statistics, Series 2, No. 92. 1982. DHEW Publ. No (PHS) 82-1366. Public Health Service, Washington, D.C. U.S. Government Printing Office.

Potthoff, Richard F. "Telephone Sampling in Epidemiologic Research: To Reap the Benefits, Avoid the Pitfalls", American Journal of Epidemiologyl 139 (10), 1994, 967-978.

Potthoff, R.F. "Some Generalizations of the Mitofsky-Waksberg Technique for Random Digit Dialing", Journal of the American Statistical Association, 1987, 82(398), 409-418.

Shah, B.V. "Variance Estimates for Complex Statisics and Multistage Sample Surveys". In Namboodiri, N. Krishnan (editor). Survey Sampling and Measurement. 1978. Academic Press. New York, NY. pgs. 25-34. Problems of variance estimation in complex sample surveys.

Shah, BV (1998). Linearization Methods of Variance Estimation. In Encyclopedia of Biostatistics, edited by Peter Armitage & Theodore Colton, John Wiley & Sons, New York, volume3, pgs. 2276-2279. Explanation of the variance approximation technique commonly used in software packages for complex sample survey data. Technical, requires math-stat background.

Waksberg, Joseph. "Sampling Methods for Random Digit Dialilng", Journal of the American Statistical Association. 1978, 73(361), 40-46. Basis for Mitofsky-Waksberg RDD technique.

Waksberg, Joseph. "A Note on Locating a Special Population Using Random Digit Dialing", Public Opinion Quarterly. 1983, 47, 576-579.

Waksberg, J. Random Digit Dialing Sampling for Case-Control Studies. In Encyclopedia of Biostatistics, volume 5, edited by P Armitage & T Colton, John Wiley, New York, 1998, 3678-3682.