I Gave ChatGPT a C in my Graduate Applied Research Course

I am currently planning my summer graduate course in Applied Research (Teacher Education). I would like to invite students to use ChatGPT to help them research their classroom problems and plan interventions (and assessments for those interventions). From my own experience, this is a difficult process, and it is probably beyond the skill or interest of those graduate students who have but a mere passing interest in conducting research themselves. In any event, a detailed and original research design is a lot to ask of students during their Summer Break.

And so, I am considering inviting students to use AI to help them with the process. If I move forward with that decision, then I will share my rationale and language for said invitation later. 

For now, however, I am curious what would happen if students simply outsourced all of their work to AI, and then, for example, submitted a completed project to me in about 30 minutes. Would they pass? Could ChatGPT do high enough quality work to get an A all by itself? Am I sabotaging my class by inviting them to use AI to help them with their work?

I tested it to see. Here is the grade it got:

Test: I have just asked ChatGPT to “Synthesize research about improving student motivation in elementary school using APA format.” [Note: This prompt represents ~18% of the work students will be responsible for over the summer. I have invented the problem and the demographic of student; my prospective students will not be limited to examining motivation problems or designing interventions for elementary school children.]

 

Here is what ChatGPT produced:

Improving Student Motivation in Elementary School

Abstract
Motivating elementary school students is a crucial aspect of fostering academic success and personal development. This paper explores research-backed strategies to enhance student motivation, focusing on intrinsic and extrinsic motivational techniques. By incorporating goal-setting, autonomy, engaging instructional strategies, and positive reinforcement, educators can create an environment conducive to learning. The study also examines the role of parental involvement and the impact of technology on motivation.

Introduction
Motivation in elementary school students is a key factor in their academic performance and long-term educational engagement. According to Ryan and Deci (2000), motivation can be categorized into intrinsic and extrinsic types, with intrinsic motivation fostering a deeper and more sustained engagement in learning. This paper examines various approaches to improving student motivation and their effectiveness in elementary education settings.

Theoretical Background
Self-Determination Theory (Deci & Ryan, 1985) suggests that motivation is enhanced when students experience autonomy, competence, and relatedness. Bandura’s (1997) concept of self-efficacy further emphasizes the importance of confidence in students’ abilities. Applying these theories, educators can design interventions to improve student motivation effectively.

Strategies for Enhancing Motivation

1.     Goal-Setting and Autonomy
Providing students with opportunities to set personal learning goals and make choices in their learning activities fosters a sense of ownership and motivation (Schunk & Zimmerman, 2007).

2.     Engaging Instructional Methods
Interactive and hands-on learning experiences, such as project-based learning, have been shown to increase motivation and retention (Thomas, 2000).

3.     Positive Reinforcement
Reinforcing positive behavior and academic effort through praise, rewards, and recognition encourages persistence and effort (Skinner, 1953).

4.     Parental Involvement
Research indicates that parental support and involvement in academic activities enhance student motivation and performance (Fan & Chen, 2001).

5.     Use of Technology
Integrating educational technology, such as gamification and interactive learning platforms, has been linked to increased motivation and engagement in young learners (Gee, 2003).

Conclusion
Improving student motivation in elementary school requires a multi-faceted approach incorporating psychological theories and evidence-based strategies. By fostering autonomy, using engaging teaching methods, and involving parents, educators can create a motivating and effective learning environment. Further research should explore the long-term impacts of these strategies on academic success.

References
Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.
Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. Springer Science & Business Media.
Fan, X., & Chen, M. (2001). Parental involvement and students’ academic achievement: A meta-analysis. Educational Psychology Review, 13(1), 1-22.
Gee, J. P. (2003). What video games have to teach us about learning and literacy. Palgrave Macmillan.
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68-78.
Schunk, D. H., & Zimmerman, B. J. (2007). Motivation and self-regulated learning: Theory, research, and applications. Routledge.
Skinner, B. F. (1953). Science and human behavior. Simon and Schuster.
Thomas, J. W. (2000). A review of research on project-based learning. Autodesk Foundation.

Summary

ChatGPT created a short—400 words or so—essay that introduced self-determination theory, self-efficacy, goal-setting, positive reinforcement, parental involvement, and use of technology. The APA formatting and writing mechanics were fine enough that I wouldn't deduct any points.

Assessment

I assessed the ChatGPT essay using the Inquiry & Analysis VALUE rubric item “Existing Knowledge, Research, and/or Views.” This rubric scores an artifact on a scale between 1 (the artifact barely exhibits the objective) and 4 (the artifact exhibits mastery of the objective). The scoring of the objective looks like this:

 

	4	3	2	1
Existing Knowledge, Research, and/or Views	Synthesizes in-depth information from relevant sources representing various points of view/approaches. Considers generalizability of knowledge to specific demographic.	Presents in-depth information from relevant sources representing various points of view/approaches.	Presents information from relevant sources reprsenting limited points of view/approaches.	Presents information from irrelevant sources representing limited points of view/approaches.

 

Beginning at the basic benchmarks for this objective, I find that ChatGPT has presented information (Level 1), and this information is relevant (Level 2). Check, Check.

For Level 3, ChatGPT seems to present information that represents various points of view. Indeed, it pulls from Behaviorism, family systems theory, and self-determination theory. But the presentation of these points of view is cursory. It is unclear how relevant these points of view are for elementary school children. 

If a score of 3 would be generous, then a score of 4 would be unfair. ChatGPT has not synthesized anything. The different approaches are merely stated without any mention or suggestion of their being combined in any way to solve real problems, and absolutely no mention of their applicability to a specific school district or motivation problem.

Conclusion

In summary, this ChatGPT essay has aptly identified relevant information in the field of motivation psychology as it might be applied in the classroom. This is an accomplishment. But the work requires a good deal of deeper synthesis and greater consideration of the information’s applicability/generalizability to solve a specific problem or address a specific demographic. For graduate students, this represents about the lowest achievement above failure, in my professional opinion--that is, a C.

Based on this one test (I will surely conduct more), I think ChatGPT would help students identify relevant information to explore in order to solve a problem. In this regard, I think ChatGPT would do a better job than I would, even though I have written a book on the very topic. That's because ChatGPT would not be biased in what it shared, whereas I would immediately focus on SDT and humanistic theories of motivation. I would psychoanalyze the problem and why the student was so troubled by it. And so on. ChatGPT just gave the topics and references and said, "Here you go. Explore!"

But ChatGPT cannot go all the way and give the student an excellent essay, at least not how I asked the question. I think I could give a series of prompts to improve the ChatGPT output, but doing so already represents my ability to creatively understand the problem and what a deep synthesis looks like.

End Note

It is important to acknowledge that I asked ChatGPT to do the one research skill that I think is most uniquely suited to it. Therefore I would expect it to do more poorly on the other objectives.