Author of the paper here. We found that AI is as good as expert human tutors for immediate GRE learning gains (p=.015, n=2,383 students).
One hour with an expert GRE tutor: $75.
One equivalent hour with an AI tutor: 7 cents.
918x cheaper per point of learning, with equivalent learning (p=.044).
In 5 of 7 GRE topics, the top AI tutor beat the expert human tutors on average.
Motivation is that RSI and most AI research is focused on machines improving machines. We created this suite of benchmarks and evaluations to shift investment to focus on machines improving humans. We were also just genuinely curious how much AI could improve a human's learning gain on real actual standardized exams.
All data from students and teachers is open-sourced to advance research in augmenting human capabilities with AI.
Author of the paper here. We found that AI is as good as expert human tutors for immediate GRE learning gains (p=.015, n=2,383 students).
One hour with an expert GRE tutor: $75. One equivalent hour with an AI tutor: 7 cents. 918x cheaper per point of learning, with equivalent learning (p=.044).
In 5 of 7 GRE topics, the top AI tutor beat the expert human tutors on average.
Motivation is that RSI and most AI research is focused on machines improving machines. We created this suite of benchmarks and evaluations to shift investment to focus on machines improving humans. We were also just genuinely curious how much AI could improve a human's learning gain on real actual standardized exams.
All data from students and teachers is open-sourced to advance research in augmenting human capabilities with AI.
Data: https://huggingface.co/datasets/handshake-ai-research/studen... Code: https://github.com/Handshake-AI-Research/studentbench Try it: https://studentbench.org Video: https://x.com/cgnorthcutt/status/2103171858057277532/video/1
Happy to answer questions.