Decision Trees

Hello students, welcome to Kwickprep. Before playing cricket after school, you ask yourself a few questions. Is it raining? Is my homework done? A decision tree lets a computer decide in exactly this way. Today we draw a tree, see how it splits data, read predictions from it, and weigh its strengths and weaknesses.

A decision tree is a model that makes a decision by asking questions, one after another. The root node is the first question, drawn at the top. Branches are the paths that come out of a question, one for each answer. A leaf node is the end of a path, where the final decision is written. It is called a tree because it branches out, though we draw it upside down.

Here is a decision tree for a simple choice, drawn as a flowchart. The root question is, is it raining? If yes, the leaf says stay home. If no, we ask the next question, is the homework done? If yes, we play cricket, and if no, we finish homework first.

Let us name every part of this tree. The root node is, is it raining? The inner node, a question in the middle, is, is the homework done? The leaf nodes are the final answers: stay home, play cricket, and finish homework first. The depth is the longest chain of questions, which here is two.

A computer does not get the tree from us; it learns the tree from data. Here are six past days. On a rainy day with homework done, the child did not play. On a rainy day with homework not done, no play. On a sunny day with homework done, the child played. On a sunny day with homework not done, no play. Cloudy with homework done meant play, and cloudy without homework meant no play.

Now, how does the tree choose its questions? A split means dividing the rows into groups using one question. The computer tries every feature as a possible question. A group is called pure when all its rows have the same answer. It picks the question whose groups are the purest. Then it repeats the same process inside each group, until the groups are pure or small enough.

Let us try splitting on homework first. The not done group has three rows, and all three say no, so it is pure. The done group has three rows, with one no and two yes, so it is mixed.

Now try splitting on weather instead. The rainy group has two rows, both no, so it is pure. The sunny group has one yes and one no, so it is mixed. The cloudy group is also mixed. Pause and predict: which split is better? Homework wins, because its pure group is bigger, holding three rows.

So the tree puts homework at the root. If homework is not done, the leaf predicts no play. The done group was mixed, so the tree splits it again, this time on weather. If it is rainy, the leaf predicts no. Otherwise, the leaf predicts yes, and now every leaf is pure.

To read a prediction, start at the root and follow the answers down to a leaf. A sunny day with homework done passes the homework check, is not rainy, and reaches yes. A rainy day with homework done passes homework but stops at rainy, so no. A cloudy day with homework not done stops at the very first question, so no.

Every model has good and bad sides, so let us compare. A tree is easy to explain, since anyone can follow its questions, but a very big tree becomes confusing. It handles both numbers and words, but it is unstable, since a small change in data can build a very different tree. It predicts fast, but it can overfit.

Overfitting is a word you must know. It means the model memorises the training data instead of learning the general pattern. A very deep tree can create a separate leaf for almost every row. Such a tree scores well on old data but fails on new data, like a student who memorised answers without understanding. We fix this by limiting the depth or cutting off weak branches, which is called pruning.

Decision trees are used where people need to see the reason behind a decision. A bank can check income and existing loans before approving a loan. A hospital can use symptoms to estimate a patient's risk level. A shop can predict whether a customer will buy again. Farmers can get advice on which crop suits their soil and rainfall.

Let us revise what we learned today. A decision tree asks questions, starting at the root and ending at a leaf. The computer learns these questions from past data. At each step, it picks the split that makes the purest groups. To read a prediction, follow the answers down to a leaf. Trees are easy to explain, but deep trees can overfit. Try drawing your own tree for choosing what to eat at the canteen.