Syllabus

What the **** is program evaluation?

That question comes up more than you probably think! In practitioner-oriented fields like (but certainly not limited to) education and counseling, studies are often directed at developing and understanding the impact various programs have on those who receive services, those who provide services, and/or the larger context within which those programs operate. This mode of uncovering if something works and why it works is a basis for evaluating programs and in turn policy decision-making. Program evaluation has philosophical, theoretical, methodological, and practical peculiarities that make it necessary to devote an entire class to understanding why and how it is done. This course is designed to allow you to acquire an understanding of the choices available for and processes involved in conducting program evaluation by giving you hands-on experience conducting a small-scale evaluation study relevant to your interests.

Course objectives

The overall purpose of this course is to empower you to

acquire knowledge of basic models and principles of program evaluation
develop an understanding of program evaluation practice by designing and conducting an evaluation of a relevant program relevant
gain an understanding of ethical, social, political, and cultural issues confronted by program evaluators
write an evaluation report to meet stakeholder information and decision-making needs

Quick Start Guide

Please note that the slideshow goes through highlights of the syllabus, but you are still responsible for reading through the entire site including this page.


Larger version of the Guide	PDF of the Guide

FAQ

Is the course content difficult?

There is really no way to sugar coat it - the first seven to eight weeks prior to your evaluation are fairly intense so please plan accordingly. You are essentially asked to both learn about evaluations and then administer one which for all intents and purposes is not easy. Given that, difficulty is subjective, it varies, and often depends on your academic and practical history. Instead consider focusing on the following question

Can I first search for help independently and then reach out as needed?

If the answer is an overall yes, then you have a good shot at succeeding in this class.

Is there a lot of theory?

Evaluation is foremost a practice based field so much of what you will come across will align with that notion. Given that, there are times where theory will show up but they will be limited.

What if I’m a Visual/Aural/Read-write/Kinesthetic learner?

You’re not one or even any of those. There’s not one accepted study that supports one type of learning style over another. In fact, there aren’t even any studies that support these four classifications either. You learn the best using multiple approaches which is why there are both a Readings and Tasks & Deliverables section. At one time I denoted multiple papers to support this, but many of the sources I listed were also cited in this fantastic video:

What if I find a mistake?

Tell me! I strive to be error free but unfortunately like everyone else, make silly mistakes including grammar and spelling errors as well!

What do the different pages on the site mean?

These are given in the order you see them in top of the site.

Readings	Go	Weekly readings
Tasks & Deliverables	Go	Course materials and assessment items
Due	Go	List of things to do
Resources	Go	Curated directory of resources

Texts

To help minimize costs, you are not required to purchase any physical texts. We’ll rely on the various materials below which, barring the APA 7th edition handbook, are available through WVU Libraries in a digital format.

Required

Davidson, E. J. (2007). Evaluation methodology basics: The nuts and bolts of sound evaluation (1st ed.). SAGE.

Free

Physical

Optional

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.).		Physical
Dillman, D.A., Smyth, J.D., & Christian, L.M. (2014). Internet, phone, mail, and mixed-mode surveys: The Tailored Design Method (4th ed.). Wiley.	Free	Physical
Mertens, D. M. (2018). Mixed methods design in evaluation. SAGE.	Free	Physical

There will occasionally be additional articles and videos to read and watch. When this happens, links to these other resources will be included on the content page for that session.

Software

Quatrics

You will need access to your WVU Qualtrics account. Please go to the site and check if you can login. If you are enrolled as a WVU student, then you should have access. However if you do not, submit a ticket to IT or give them a ring at 304-293-4444.

Assignments and Grades

You can find descriptions for all posted assignments on the Tasks & Deliverables page

Percent	Assignment
15	Weekly Tasks
15	Synchronous Meetings
5	Evaluation Prospectus
20	Needs Assessment
15	Evaluation Proposal
25	Final Report
5	Presentation of Results

Grade	Range
A	90–100%
B	80%-89%
C	70–79%
D	60–69%
F	< 60%

All materials are due on a weekly basis by Sunday at 11:59 PM unless otherwise noted. New material will be posted on the course website by Tuesday afternoon at 12:00 PM.

Grades are determined on an individual basis. Each person will receive the same score for the Tasks & Deliverables unless there is an issue with a team member which has been brought to my attention prior to the end of the term. In that case, the student(s) in question will be assessed through an investigation and arbitration process conducted by me. In cases where the accused are found to be at fault, they will receive an independent separate score which may differ from other teammates. The most common type of incident to fall under this category is a lack of effort with the team environment. In cases where university statues are broken, the student(s) may also be referred to the Office of Academic Integrity.

Meetings

Unless another mode is determined beforehand, we will conduct group meetings via Zoom. This can be scheduled using the class calendar.

Conduct and Expectations

Here are the rules, expectations, and policies that I came up with or stole from other professors:

Late work: Barring the in-class only tasks (if applicable), past due deliverables will only be accepted up to 48 hours after the initial time and due date. For each full day an assignment is late, 10% of the final calculated grade will be deducted. All submissions must be made via eCampus and are not accepted by any other means (e.g. email, paper, singing telegram). There are no exceptions to this overarching policy.
Participation: Please ensure that you are engaged and participate. Engagement is mostly defined by you — if that means commenting and asking/answering questions - great; but if it means being being disrespectful or flaking off - not so great.
Justifications: On any submission or posting, you must justify any assertion. I don’t know what you know so its your job to provide all of the necessary evidence to convince me that you do know what you say you know. While you’ve probably heard this multiple times over your life, think of it this way: if someone tells you that the core of the Earth is made of glitter, the first question you should be asking is what’s your evidence?.
Rubrics: While there are valid reasons for the utilization of a rubric in undergraduate classes, at the graduate level I do not (often) provide nor use a such an item to guide or evaluate your submission due to four primary concerns:
1. When writing anything in academia that is pivotal (a thesis, dissertation, journal article, report, etc.), a rubric isn’t typically provided.
2. If you write within the limitations as defined in a rubric, then creativity may be stifled (i.e. writing to the rubric rather than constructing a product from the ground up).
3. Feedback can only be given along the criteria listed within a rubric which limits your learning as a student and constrains me as the instructor.
4. Unless you are in a very specific area, the real world does not use rubrics!
Technology use: Use phones, computers, etc. responsibly. We’re all adults¹.
Assumptions:
1. Just assume that all submissions are to be written formally and should include with the appropriate use of language, grammar, syntax, etc. and follow standard APA 7th edition formatting guidelines where applicable.
2. People who are easily offended by content, believe their work to be flawless, generally unable to handle criticism, or are looking for rainbows and ponies should consider looking at another course. However if you care about data, real-world practice, and learning a highly marketable skill then you’ve come to the right place.
3. There is a great deal of content in this course and you will likely struggle with some at times. Given that, there is also something to be said about the satisfaction a person gets when figuring something out, but nowhere is it written that has to be on your own. You may find that a nudge here or there elicits the same feeling so please reach out for help.

Communication

Slack

We have a class Slack channel where you can poise a question and post as needed. This is the only mode where I have applicable class notifications turned on my devices so please utilize the platform.

Email

Rather than including a list of things that are appropriate when writing an email, this strip from PhD Comics will do

Zoom

One-on-one video conferences can be created via the class calendar in 15 minute increments. If you require more than 15 minutes, please schedule additional back-to-back sessions.

Course Policies

In a nutshell keep an open line of communication, be honest, nice, respectful to everyone, & follow the Code of Conduct.

Modifications

This syllabus reflects a plan for the term but things change and plans change so deviations may become necessary as we move along during the term. This means that I reserve the right to alter or amend this syllabus.

Health statement

WVU is committed to maintaining a safe learning environment for all students, faculty, and staff. Should campus operations change because of health concerns related to health crisis, it is possible that this course will move to a fully online delivery format. If that occurs, students will be advised of technical and/or equipment requirements, including remote proctoring software.

In a face-to-face environment, our commitment to safety requires students, staff, and instructors to observe guidelines set by the University at all times. Students who fail to comply will be dismissed from the classroom for the class period and may be referred to the Office of Student Conduct for further sanctions.

If a student becomes sick or is required to quarantine during the semester, they should notify the instructor. The student should work with the instructor to develop a plan to receive the necessary course content, activities, and assessments to complete the course learning outcomes.

Psychological and Psychiatric Services

Life at WVU can be complicated and challenging! You might feel overwhelmed, experience anxiety or depression, or struggle with relationships or family responsibilities. Psychological and Psychiatric Services provides free, confidential support for students who are struggling with mental health and emotional challenges. The office is staffed by professional counselors and psychiatrists who are attuned to the needs of all types of college and professional students. Please do not hesitate to contact them for assistance—getting help is a smart and courageous thing to do.

CARE Team

If you or anyone you know may be at-risk such as those listed here, please make a CARE referral. You may do so directly at the main WVU CARE TEAM site.

Lauren’s Promise

Foremost

I will listen and believe you if someone is threatening you!

Lauren McCluskey - a 21-year-old honors student athlete - was murdered on October 22, 2018 by a man she briefly dated on the University of Utah campus. It is imperative that this never happen again!

If you are in immediate danger, please call 911 or the Campus Police at 304-293-3136.
If you are experiencing sexual assault, domestic violence, or stalking, please report it to me and I will connect you to resources or call/text a private Title IX On-Call Line 304-906-9930.

Any form of sexual harassment or violence will not be excused or tolerated at West Virginia University. WVU has instituted procedures to respond to violations of these laws and standards, programs aimed at the prevention of such conduct, and intervention on behalf of the victims.

Academic Integrity

The integrity of the classes offered by any academic institution solidifies the foundation of its mission and cannot be sacrificed to expediency, ignorance, or blatant fraud. Therefore, I will enforce rigorous standards of academic integrity in all aspects and assignments of this course. For the detailed policy of West Virginia University regarding the definitions of acts considered to fall under academic dishonesty and possible ensuing sanctions, please see the West Virginia University Academic Catalog. Should you have any questions about possibly improper research citations or references, or any other activity that may be interpreted as an attempt at academic dishonesty, please see me before the assignment is due to discuss the matter.

Inclusivity Statement

The West Virginia University community is committed to creating and fostering a positive learning and working environment based on open communication, mutual respect, and inclusion.

If you are a person with a disability and anticipate needing any type of accommodation in order to participate in this class, please advise me and make appropriate arrangements with the Office of Accessibility Services ( 304-293-6700).

For more information on West Virginia University’s Diversity, Equity, and Inclusion initiatives, please see the Division of Diversity, Equity, and Inclusion.

Incomplete Grades

Students who want to be considered for an Incomplete must apply to their instructor prior to the end of the term. If the instructor agrees, the instructor and the student must negotiate the conditions under which the grade of I will be changed to a letter grade and sign a contract. The date to submit the incomplete work should not be set beyond the last day of class of the following semester. If the student does not complete the terms of contract, then the instructor should submit a grade of F. All incomplete contracts must be filed with the department and Dean’s Office. Students who want to be considered for an Incomplete must apply to their instructor prior to the end of the term. If the instructor agrees, the instructor and the student must negotiate the conditions under which the grade of I will be changed to a letter grade and sign a contract. The date to submit the incomplete work should not be set beyond the last day of class of the following semester. If the student does not complete the terms of contract, then the instructor should submit a grade of F. All incomplete contracts must be filed with the department and Dean’s Office. See the policy at Advising, Enrollment, & Grades.

Student Evaluation of Instruction (SEI)

Effective teaching is a primary mission of West Virginia University. Student evaluation of instruction provides the university and the instructor with feedback about your experiences in the course for review and course improvement. Your participation in the evaluation of course instruction is both strongly encouraged and highly valued. Results are strictly confidential, anonymous, and not available to the instructor until after final grades are released by Admissions and Records. Information about how you can complete this evaluation will be provided later.

University Attendance Policy

At West Virginia University, class attendance contributes significantly to academic success. Students who attend classes regularly tend to earn higher grades and have higher passing rates in courses. Excessive absences may jeopardize students’ grades or even their ability to continue in their courses. There is a strong correlation between regular class attendance and academic success.

Postings

Following these simple netiquette rules in your online class or education environment will ensure your success:

Never type in ALL CAPS, because it reads as if you ARE SHOUTING AT PEOPLE.
Act as professionally, via your writing, as you would in a face to face classroom.
Refrain from inappropriate language and derogatory or personal attacks.
Do not dominate any discussion. Give other students the opportunity to join in the discussion.
Disagree with ideas but avoid challenges that may be interpreted as a personal attack.
Check that you are replying to the specific person you intend, and not to the entire class.
Never give your password to another person.
Respect the virtual classroom. Never forward in-class communications or posts by others outside of this virtual space.
Never spam your classmates.
If you quote someone’s previous post, only quote enough to make your point.

Be aware of the University’s Academic Integrity and Dishonesty Policy. You can review the rules, regulations, and procedures concerning student conduct and discipline for the main campus of West Virginia University, at Student Conduct.

Response Time

I typically respond to Slack queries on the same day while emails and discussion posts are within 48 hours, except during holidays. Please plan accordingly so that you don’t miss deadlines! I generally return assignments within two weeks after a final submission date.

Technical Requirements

Students need to have access to a computer for word processing, e-mail and access to eCampus. Access to the Internet is necessary for completion of this course. Run the Browser Check. This tool will check that you are using a supported Internet browser and have a valid Java version installed. The required technical skills to participate in this course are:

Navigate the web
Use email with attachments
Create and submit files in commonly used word processing program formats
Copy and paste
Download and install software
Consult software tutorials and other online sources as a method of learning software features
Use syntax when necessary

Technical Support

Technical support regarding your use of eCampus is available by contacting 304-293-4444 (telephone), 1-877-327-9260 (toll free number), itshelp@mail.wvu.edu (email), and/or Information Technology Services (website).

Ownership

Sale of Course Materials

Course materials including but not limited to lectures, class notes, quizzes, exams, handouts, and presentations provided to students for this course are protected under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Please review the sharing and editing restrictions prior to distributing or amending any material on this site. As such, the unauthorized purchase or sale of these materials may result in disciplinary sanctions under the Campus Student Code. Basically you can share what you like under the licence but don’t try to make a buck.

Lisencing Information

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

…most of the time. ↩︎