2026 Beginner Class - AI Rating & Data Annotation - Complete Roadmap
0:00
Welcome to the AI on Border Masterclass. This isn't a motivational seminar or a get-rich-quick course.
Welcome to the AI on Border Masterclass. This isn't a motivational seminar or a get-rich-quick course.
0:05
It's a complete beginner roadmap based on my experience working in AI rating and data annotation.
It's a complete beginner roadmap based on my experience working in AI rating and data annotation.
0:11
If you watch this from beginning to end, you'll understand how to prepare, where to apply to, and what to expect during assessments, how to work more efficiently, and some of the common pitfalls that catch beginners.
If you watch this from beginning to end, you'll understand how to prepare, where to apply to, and what to expect during assessments, how to work more efficiently, and some of the common pitfalls that catch beginners.
0:22
Your first question may be, is this work for you? So let's look at what this work entails.
Your first question may be, is this work for you? So let's look at what this work entails.
0:27
Okay, you've probably been hearing a lot about AI jobs, AI rater, data annotator, content reviewer, and maybe you're wondering two things.
Okay, you've probably been hearing a lot about AI jobs, AI rater, data annotator, content reviewer, and maybe you're wondering two things.
0:34
Can I do it? And is it worth it? And for me, the answer was yes, yes.
Can I do it? And is it worth it? And for me, the answer was yes, yes.
0:39
And today I want to show you why the answer may be yes, yes for you as well.
And today I want to show you why the answer may be yes, yes for you as well.
0:43
And on the issue of can you, let's start with a question.
And on the issue of can you, let's start with a question.
0:47
A user types in a query, how do I remove wine stain from a shirt? The AI gives three responses.
A user types in a query, how do I remove wine stain from a shirt? The AI gives three responses.
0:54
A, wash the shirt. B. Blot the stain and rinse with cold water. C.
A, wash the shirt. B. Blot the stain and rinse with cold water. C.
1:00
Wine stains can be difficult to remove. Which would you rate as the best response?
Wine stains can be difficult to remove. Which would you rate as the best response?
1:05
Hold your answer and we'll come back to it. In this video, I want to show you how I found out about this work, what it involves, and why your life experience might be more valuable than you think.
Hold your answer and we'll come back to it. In this video, I want to show you how I found out about this work, what it involves, and why your life experience might be more valuable than you think.
1:16
Here's another quick one. A user searches for a German Shepherd puppy. Which image deserves the highest rating?
Here's another quick one. A user searches for a German Shepherd puppy. Which image deserves the highest rating?
1:21
A, B, or C? Pick your answer. We'll reveal it later.
A, B, or C? Pick your answer. We'll reveal it later.
1:27
For most of my life, AI wasn't even on the radar. But my work in accounting and business management had taught me one thing, attention to detail.
For most of my life, AI wasn't even on the radar. But my work in accounting and business management had taught me one thing, attention to detail.
1:37
Learn to spot things that don't add up. And you learn to question things.
Learn to spot things that don't add up. And you learn to question things.
1:41
And I didn't realize that those same skills would eventually matter in a completely different field.
And I didn't realize that those same skills would eventually matter in a completely different field.
1:46
Then someone explained it to me in a way that clicked. Think of AI as a smart student.
Then someone explained it to me in a way that clicked. Think of AI as a smart student.
1:51
The AI gives an answer, a human checks whether the answer is helpful, accurate, and useful.
The AI gives an answer, a human checks whether the answer is helpful, accurate, and useful.
1:57
The AI is a student and you're the teacher, and suddenly it made sense.
The AI is a student and you're the teacher, and suddenly it made sense.
2:02
Now, those two challenges you answered earlier, those are simplified versions of the decisions AI evaluators make every day.
Now, those two challenges you answered earlier, those are simplified versions of the decisions AI evaluators make every day.
2:10
And if you formed an opinion, you're more qualified than you think. Now, here's the basic process.
And if you formed an opinion, you're more qualified than you think. Now, here's the basic process.
2:16
Someone enters a prompt, the AI generates a response. Then a human evaluates whether that response meets the user's needs.
Someone enters a prompt, the AI generates a response. Then a human evaluates whether that response meets the user's needs.
2:25
The more I learn, the more I realize this work isn't really about technology.
The more I learn, the more I realize this work isn't really about technology.
2:30
It's about reasoning, logic, and attention to detail, and a willingness to learn.
It's about reasoning, logic, and attention to detail, and a willingness to learn.
2:35
Companies need people who can tell when AI is being helpful or when it is confidently giving bad information.
Companies need people who can tell when AI is being helpful or when it is confidently giving bad information.
2:43
Now you've probably experienced this in real life. Someone tells you a story and instantly you know something's off.
Now you've probably experienced this in real life. Someone tells you a story and instantly you know something's off.
2:49
The facts don't fit. The timeline makes no sense. The details don't add up.
The facts don't fit. The timeline makes no sense. The details don't add up.
2:54
That same instinct is what AI evaluators use every day. But here's the truth.
That same instinct is what AI evaluators use every day. But here's the truth.
3:00
AI can sound incredibly confident even when it's completely wrong. And here's a humorous example.
AI can sound incredibly confident even when it's completely wrong. And here's a humorous example.
3:06
The query was, is Marlon Brando in heat? Now, we would know in heat is the movie from 1995 about Pacino, Robert De Niro.
The query was, is Marlon Brando in heat? Now, we would know in heat is the movie from 1995 about Pacino, Robert De Niro.
3:13
But unfortunately, the AI did not interpret it that way.
But unfortunately, the AI did not interpret it that way.
3:17
And you can read it from this response that it was a rather hilarious response, yet it thought it was completely right and completely confident in how it responded.
And you can read it from this response that it was a rather hilarious response, yet it thought it was completely right and completely confident in how it responded.
3:27
Now let's look back at that first question that was posed to AI. How do we remove a wine stain from a shirt?
Now let's look back at that first question that was posed to AI. How do we remove a wine stain from a shirt?
3:34
And the AI had responded, A, wash the shirt, B, blot the stain and rinse with cold water,
And the AI had responded, A, wash the shirt, B, blot the stain and rinse with cold water,
3:40
and C, wine stains can be difficult to remove. The best response was B, blot the stain and rinse with cold water.
and C, wine stains can be difficult to remove. The best response was B, blot the stain and rinse with cold water.
3:48
Why? Because it directly helps the user solve the problem. If you pick B, congratulations.
Why? Because it directly helps the user solve the problem. If you pick B, congratulations.
3:53
You just made the same type of judgment AI raters make every day.
You just made the same type of judgment AI raters make every day.
3:58
When I worked as an accountant, I just didn't look at the final number. I look for things that didn't look right.
When I worked as an accountant, I just didn't look at the final number. I look for things that didn't look right.
4:04
And that's exactly how I approach AI responses today.
And that's exactly how I approach AI responses today.
4:08
And over decades of work and life experience, I built those same instincts and you probably have as well.
And over decades of work and life experience, I built those same instincts and you probably have as well.
4:15
Those instincts are valuable, especially when evaluating AI responses.
Those instincts are valuable, especially when evaluating AI responses.
4:20
So if you're over 50, and don't sell yourself short, you've been auditing information your entire life.
So if you're over 50, and don't sell yourself short, you've been auditing information your entire life.
4:27
And now you're just simply going to do it for a machine. And yes, I know what some of you are thinking. I struggle with the TV remote.
And now you're just simply going to do it for a machine. And yes, I know what some of you are thinking. I struggle with the TV remote.
4:34
How am I supposed to work with AI? Fair question. But the good news is, many projects use tools you already know and have.
How am I supposed to work with AI? Fair question. But the good news is, many projects use tools you already know and have.
4:42
For most projects, all you need is a computer, internet access, smartphone, and attention to detail.
For most projects, all you need is a computer, internet access, smartphone, and attention to detail.
4:49
Look, it's not effortless and there is a learning curve and some of the guidelines, they can be more than 50 pages long. And now for that second challenge.
Look, it's not effortless and there is a learning curve and some of the guidelines, they can be more than 50 pages long. And now for that second challenge.
4:57
The query was German Shepherd puppy, and I showed you three images, A, B, and C.
The query was German Shepherd puppy, and I showed you three images, A, B, and C.
Only the first 5 minutes are free. Install the app or extension to keep watching.
DownloadRequest another language





