Skip to content

Rating Scale Design and Analysis

Course Title:

Rating Scale Design and Analysis: Theories and Applications


Dr. Ren Liu     (

Course Description:

Rating scales are commonly used in surveys and questionnaires to measure people’s latent traits in social and behavioral sciences. This interactive course provides a detailed introduction, demonstration, and hands-on exercises using the free R software to learn theories and applications in rating scale design and analysis with minimal math burden. Topics covered include: (1) Overview of Rating Scale Design and Analysis Process; (2) Validity; (3) Reliability; (4) Evidence Centered Design; (5) Content Construction; (6) Item Data Exploration w/ Classical Test Theory; (7) Dimensionality Analysis w/ Factor Analysis; and (8) Item Analysis w/ Item Response Theory.

This session is created for researchers and practitioners who are interested in a systematic learning experience in critiquing, designing, and/or developing a rating scale. No previous knowledge of psychometrics is required, although some knowledge/experience of the broad concept of measurement is helpful. For topics 6-8, participants will be provided with datasets and R code to apply what they have learned on their laptops during the training session. After taking this course, participants will be able to critique existing rating scales, and develop and analyze their own rating scales in a scientific manner.

Target Audience:

Graduate students, emerging researchers, continuing researchers

Prerequisite Knowledge:

The audience may find this training session most helpful if they have a rating scale in mind that they want to develop or critique. No previous knowledge of psychometrics is required. The workshop aims to minimize math/statistical burdens.


Only topics 6-8 involve real data analyzing exercises which will be done using the free R and Rstudio software. Participants will be provided all resources to perform these exercises on their own, and we will provide instructions on how to download free versions of the software prior to the course. Participants will also be provided with datasets and well-annotated R code and will be invited to run the code on their laptops.

Dates and Times:

[Times provided in Pacific Standard Time]

October 22, 2021

[breaks will be provided according to the Instructor’s schedule]

Please join before 9am. We will begin at 9am sharp.

9:00 – 11:30: Morning Session (Topics 1-5)

11:30 – 12:20: Lunch Break

12:20 – 12:30: Installation of R and Rstudio software

12:30 – 2:30: Afternoon Session (Topics 6-8)

2:30 – 3:00: Q & A

Online Details:

Participants will receive a code to use on their own computer to access a live-stream of the workshop, which will show the instructor, handouts, and software demonstrations. Electronic workshop materials will be provided ahead of time.

The workshop will NOT be recorded.

How to Register:

Registration is now closed. An email with zoom information and workshop materials has been sent to all registered participants. 

Contact for Questions:

Dr. Ren Liu     (

The Instructor:

Dr. Ren Liu is an Assistant Professor of Quantitative Methods, Measurement, and Statistics at the University of California, Merced. He earned his Ph.D. in Research and Evaluation Methodology with a specialization in Quantitative Methods at the University of Florida in 2018. Dr. Liu has centered his work on developing advanced statistical and measurement methods in behavioral and social sciences and translating innovative methods for applied researchers to promote best practice. These efforts have encompassed methodological research in advancing psychometric theories and their applications in test development and analysis. His work in psychometric methods contemplates ways to measure human complex latent traits more accurately and provide insightful and actionable feedback to stakeholders. All his methodological work has aimed to advance measurement theories and models across different types of data. His pioneering research won him the 2017 Distinguished Paper Award and the 2018 New Investigator Award, which were both conferred by the American Educational Research Association (AERA). In addition to his primary research in methodology, Dr. Liu also collaborates with several colleagues on substantive research projects. This substantive work covers a variety of domains including substance use and self-injury, as well as special populations such as children with altruism. Dr. Liu is a member of the National Council on Measurement in Education (NCME) Mission Fund Committee, and he is the author of the “Anatomy of Measurement”, an NCME Mission Fund YouTube Animation Project. Dr. Liu teaches both undergraduate and graduate-level courses including Analysis of Psychological Data, Item Response Theory, and Measurement Theory and Psychometrics.