A 59-second TikTok video posted on 28 September 2026 by the account Angel | Tech + AI (@creativelyange) asks whether Harvard has proved that AI can replace colleges. It says Harvard students who used a custom AI tutor in an introductory physics class learned about twice as much as students taught in the regular classroom. The video’s captions do not name the study, but their description matches a randomized trial by Greg Kestin, Kelly Miller and colleagues, published in Scientific Reports on 3 June 2025, whose authors write that an AI tutor “should not replace in-person teaching.” This post sets out what the video says, what the paper reports and where the two differ. Lens did not watch the video and relied on TikTok’s English captions for it. This post reflects sources as of 14 November 2026.

What the video says

  • The question: the video opens with “Did Harvard just prove that AI can replace colleges?” Its post text reads “did harvard prove ai replace colleges?” with the hashtags #techtok, #ai, #researchpapers and #aitutors.
  • The study: the narrator says researchers “got two groups of students at Harvard to either study in the regular classroom or to have an AI tutor,” and that the students who used the AI tutor learned “two times more” than those who went to classes, in an introductory physics class. The narrator says the groups were swapped and “they had the same finding.”
  • The tutor: the narrator says it “was actually custom built, and it’s not just a chatbot,” and that it “walks the students step by step instead of relying on the students to do all the prompting themselves.” The captions say the prompts were refined by the professors. A few seconds of captions are missing at this point, and the next fragment reads “just set up the harness so that it really guides the student around this course.”
  • The narrator’s view: the narrator says the guidance is “what really doubled the results for these students,” “not just the fact that it was an AI tutor rather than a human professor,” and adds “I still do think that this is shocking.” The video ends by saying the result “really shows the power of how AI can be a powerful source for learning and for ed tech.”

What the paper reports

  • Publication: “AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting,” by Greg Kestin, Kelly Miller, Anna Klales, Timothy Milbourne and Gregorio Ponti of Harvard University, Scientific Reports volume 15, article 17458, open access, published 3 June 2025.
  • Setting: Physical Sciences 2, an introductory physics course for the life sciences that the paper calls Harvard’s largest physics class, in the fall 2023 semester. Of 233 enrolled students, 194 were eligible for inclusion, based on consent, taking part in both formats and completing every pre-test and post-test.
  • Design: a randomized crossover. Two groups each took two lessons in consecutive weeks, on surface tension and then fluid flow. In week one group 1 took an AI-supported lesson at home while group 2 took an active learning lesson in class, and the conditions were reversed in week two. Students took a pre-test and a post-test for each lesson. The content and worksheet were identical in both conditions, and the in-class lessons used research-based active learning, not passive lecture.
  • The tutor: a custom tutor named PS2 Pal, built on GPT-4. The paper says a system prompt alone could not reliably structure multi-part problems, so the platform walks students through each part of each problem in sequence, and that the prompts were enriched with step-by-step solutions to limit errors. It describes the prompts as expert-crafted, question-specific and written by instructors experienced with the content. Preparing the prompts for one lesson took a few days, and developing the platform software took several months.
  • Learning gains: the median post-test score was 4.5 for the AI group (N = 142) and 3.5 for the in-class group (N = 174), against a pre-test median of 2.75 (N = 316). The paper says the AI group’s median learning gains were “over double” the in-class group’s, with a Mann-Whitney test giving z = -5.6 and p below 10 to the power of minus 8. Subtracting the pre-test median from each post-test median gives 1.75 points for the AI group and 0.75 for the in-class group (Lens arithmetic from the reported medians). The paper reports an effect size of 0.63 from a regression with controls, which it says understates the effect because of a ceiling effect, and 0.73 to 1.3 standard deviations from a quantile regression.
  • Time: the in-class lesson was assumed to take 60 minutes of a 75-minute period, the remaining 15 minutes going to tests. The median time on task for the AI group was 49 minutes, with 70% of that group under 60 minutes and 30% over.
  • Perceptions: on a five-point scale, the AI group rated engagement 4.1 against 3.6 for the in-class group and motivation 3.4 against 3.1. Enjoyment and growth mindset did not differ significantly. The paper says 83% of students reported that the tutor’s explanations were as good as or better than those from human instructors in the class.

What the paper says about replacing instruction

  • On replacement: “As in a ‘flipped classroom’ approach, an AI tutor should not replace in-person teaching,” the paper says, adding that it should instead bring students to a level where they can get the most from class time. It also states: “We advise against the notion that AI, solely due to its efficacy in enhancing teaching and learning, should entirely supplant in-class instructional methods.”
  • On scope: the paper says the tutor was applied where students were engaging with material for the first time, and that the gains may depend on a mixed-ability class, high-quality instructional videos, a model such as GPT-4, expert-written prompts and a structured framework. It adds: “we do not presume that structured AI tutoring will always outperform in-class active learning in all contexts,” naming those requiring complex synthesis or higher-order critical thinking.
  • On what was not tested: the study had two conditions, the in-class lesson and the tutor lesson. The paper cites earlier studies of unguided chatbot use as a contrast but did not include that condition. Its future-work section lists homework, exam studying, labs, a full course and retention strategies such as spacing.

Background

  • Timeline: the experiment ran in fall 2023. The Harvard Gazette reported preliminary findings on 5 September 2024, saying the final results were pending publication, and the Hechinger Report covered the study on 16 September 2024, before peer-reviewed publication. The Scientific Reports paper appeared on 3 June 2025, and the TikTok video was posted on 28 September 2026.
  • Kestin on replacement: the Hechinger Report quotes Kestin, a Harvard physics lecturer and the tutor’s developer, as saying: “I don’t think that this is an argument for replacing any human interaction.”

Where the video and the paper differ

  • The question asked: the video asks whether AI can replace colleges. The paper compares two ways of teaching one lesson each in one course, and does not test replacing a college or a course.
  • “Two times more”: the paper’s text gives medians and says the median gains were “over double.” The Gazette’s preliminary report said “about double,” and Hechinger said “more than twice as much.” The paper says the AI group’s actual gains are expected to be larger than its metrics show because of a ceiling effect, and that medians are more reliable than means for that reason.
  • What drove the result: the video credits the guided design rather than AI alone. The paper says the interactions “must be carefully designed to follow research-based best practices” and that its authors believe the extra learning is largely due to personalized feedback on demand, with self-pacing also noted. The paper did not test those explanations against each other.
  • Where the tutor sessions took place: the paper says students used the tutor “at home, on their own,” and the Gazette says “at home.” The Hechinger Report, writing about the unpublished version, said students used it in their dorms and that the sessions “were scheduled and proctored over Zoom.” Lens did not read the preprint.
  • Counts: the paper says 194 students took part and that each took each format once, and reports post-test scores for 142 AI-condition and 174 in-class observations. The article text Lens read does not explain the difference between those counts and 194 per condition.
  • Prompts and developer: the video says the professors refined the prompts. The paper says instructors wrote question-specific prompts, and its contribution statement credits the platform’s software design and engineering to its first author, Kestin. The authors declare no competing interests.

Sources and notes

Sources: the TikTok video by Angel | Tech + AI (posted 28 September 2026); the paper, Kestin et al., Scientific Reports, 3 June 2025; the Harvard Gazette, 5 September 2024; and the Hechinger Report, 16 September 2024. The video’s words come from the English caption track in TikTok’s page data, which looks automatically generated; it renders “AI” as “A I.” and has a gap of about three seconds near the 35-second mark, and Lens normalized “A I.” to “AI” and fixed capitalization when quoting it. Lens did not listen to or watch the video, so it could not see whether the on-screen visuals name the paper, and it matched the video to the paper by the captions’ description. Lens read the full text of the paper but did not open its supplementary material (which holds the system prompt), its figures or the GitHub data repository it cites, so the means shown in its figures are not reported here. Lens did not contact the video’s creator, the paper’s authors or Harvard, and did not read commentary on the paper or look for later studies beyond a brief search.

Image: The Lyman Laboratory of Physics, a Harvard University physics building in Cambridge, Massachusetts, photographed on 18 March 2012. Photo: Daderot, released under CC0, via Wikimedia Commons; cropped. Stand-in photo: it does not show the course, its students or the AI tutor described.