To make all the modern AI systems work, computers need to process text.

We all communicate with words and language, and to do any kind of intelligent processing of text with a machine, we need to figure out how to represent words. This is not as simple as you might think.

This Chapter introduces how words are represented in computers, and how they can be manipulated in order to achieve different tasks.

It is perhaps one of the more 'technical' chapters, as it introduces a few new concepts, but don't worry, we go through everything step by step. Using both the pdf and the videos will help, as they complement each other, and we all have different ways to learn.

This chapter will also introduce your first machine learning algorithm, so you can see how basic email spam detectors work! We'll also have some fun generating some 'Shakespeare' and Sherlock Holmes stories, so we can already start to see how AI might be used to generate text. We also introduce the idea of hallucination and what it means, which is a big challenge for modern Large Language Model AI systems.