Reliable AI marking of open-ended answers
Our own language model marks students' open answers. In this internship you find out how to make that marking more reliable, fairer and more useful, with real data on our own hardware.
- Location
- Amsterdam, partly remote
- Language
- Dutch or English
- Hours
- To be agreed
- Period
- To be agreed
- Type
- Bachelor/Master
What it is about
Marking an open question is more than looking for keywords. A student can mean the right thing and write it down slightly differently, or use the right word without understanding it. Our model gets better at this every day, and you will find out where it still judges differently from a teacher, why, and how that can improve. That is a research question with a real bar: a teacher has to be able to trust the judgement.
Why now
The marking is already running, on our own model and our own GPU server. So there is a working system to measure, and more and more teacher corrections are coming in to learn from. That is the moment when thorough research makes the difference: not building a demo, but showing exactly where the model stands and what makes it better.
What you will do
- An error analysis: where does the model judge differently from teachers, and is there a pattern in it
- Investigate whether the marking is fair to weak spellers and students with dyslexia
- Improve the model, for example by learning from teacher corrections, trying a larger model on our own GPUs, or assessing each point separately before arriving at a total score
- Develop a reliable signal for when the model is unsure, so a teacher knows exactly which answers to check themselves
- Work out how feedback to students becomes truly useful: what went well, what is missing, what you can do next time
What you learn, and what you deliver
Learning goals
- Build an evaluation set-up that shows how well a language model marks, and where it does not
- Measure the fairness of an AI system for different groups of students
- Improve a language model with real data, and show that it actually got better
- Translate a model's uncertainty into something a teacher can use
- Work carefully with sensitive education data
End product
A report or thesis, an improved version of our marking model and a benchmark we can use to test every next version in the same way. The data stays on our own hardware in the Netherlands throughout: pseudonymised and under a non-disclosure agreement.
What you work with
The exchange
What we ask of you
- You are in a bachelor's programme (at a university or university of applied sciences) or a master's programme in AI, computer science, data science or something similar
- You have experience with machine learning in Python and can set up an experiment properly
- You are critical of your own results and write down honestly what you find
- You care that a judgement about a student is correct, and you understand that this calls for care
A plus, not a requirement
- Experience fine-tuning or evaluating language models
- An interest in fairness and explainability of AI
- You have taught or tutored at some point yourself
What we offer in return
- A good learning path, with guidance from people who know the craft
- Several hours a week with one of the founders, not a group intake
- Equipment and your own desk if you need them
- Travel expenses covered if you do not have a student public transport pass
- Real barista coffee, made with the best beans in all of Amsterdam
- A reference afterwards, and a chance of a job with us
What happens next
You send your message
Your motivation in your own words plus your CV. No standard cover letter needed; just write why this appeals to you.
A reply within five working days
Even if it is a no. You will always hear from us.
An hour to get to know each other
At our office in Amsterdam or online. We show you the platform and you tell us what you want to get good at.
Join us for half a day
You work on something real and see how we work. After that, we both know.
Making arrangements
An internship agreement with your school or university, a start date, and what you will deliver.
Apply now
You do not need to write a perfect letter. Tell us in a few sentences why this appeals to you and what you want to learn. That tells us more.