Data Science Methodology

Hello students, welcome to Kwickprep. Your school canteen throws away food every day. Can data tell the canteen how much to cook? Yes, if we follow a clear method. Today we will learn what data science is and the ten steps of its methodology. Then we will work an example and link it to the AI Project Cycle.

Let us start with the meaning. Data science is the study of data to answer real questions and solve problems. It mixes mathematics, statistics and computer programming. It also needs domain knowledge, which means understanding the subject area, like health or sport. The result is an insight, a prediction or a decision that helps people.

Now, why do we need a methodology at all? A methodology is a fixed, tested set of steps for doing a job. Without it, people often jump straight to building models and miss the real question. With it, the work can be repeated and checked by others. Also, the steps form a cycle, so we often go back and improve earlier steps.

The methodology most books follow has ten steps, and here are the first five. Step one is business understanding, which asks, what problem are we solving? Step two is the analytic approach, which picks a method, like predicting a number or sorting into groups. Step three is data requirements, which lists the data we need. Step four is data collection, which gathers that data from its sources. Step five is data understanding, where we explore the data and check if it is enough and correct.

Now the remaining five steps. Step six is data preparation, where we clean and shape the data, and it often takes the most time. Step seven is modelling, where we build a model that predicts or classifies. Step eight is evaluation, where we test whether the model is accurate enough. Step nine is deployment, where people start using the model in real life. Step ten is feedback, where users report results so the model keeps improving.

Let us draw the steps as a cycle. First we choose an approach and list the data we need. Then we collect the data, understand it and prepare it. Next we build and evaluate the model, asking, is it good enough? If yes, we deploy it. If no, we go back and improve the data or the model. And feedback from real use starts the cycle again.

Now a full worked example from question to answer. The canteen wastes about forty plates of lunch every day. So our clear question is, how many lunches will be sold tomorrow? The answer is a number, so our approach is predictive, using a method called regression. The data we need is past daily sales, the day of the week, the weather, and school events like exams or sports day.

We collect sixty days of data from the canteen register, and here are four rows. Monday shows one hundred eighty plates, which looks normal. Tuesday is blank, so during preparation we fill it with a typical Tuesday value or remove it. Wednesday was an exam day, so sales were low, and that event must be recorded. Thursday shows one thousand eight hundred, which is surely a typing error for one hundred eighty, so we correct it.

Pause and predict. The model predicts one hundred seventy plates for Friday. But Friday is sports day, and half the students will be away. Should the canteen cook one hundred seventy plates? No, because the event changes the data, so the model must include events, and the staff should check such days.

Now we finish the example. The model learns how the day of the week and events change the plates sold. We evaluate it on two past weeks it has not seen, and it is off by about ten plates on average. We deploy it, so the manager sees tomorrow's forecast every evening. After a month, feedback shows the waste has fallen from about forty plates to about twelve.

You already know the AI Project Cycle, so let us link the two. Problem scoping matches business understanding and the analytic approach. Data acquisition matches data requirements and data collection. Data exploration matches data understanding and data preparation. Modelling matches modelling. Evaluation matches evaluation. Deployment matches deployment, and feedback keeps the deployed model improving.

So the two are the same idea, with a different amount of detail. Both start with a clearly defined problem. Both need good data before any model is built. Data science splits the data work into finer steps, because data work takes most of the time. And both are cycles that loop back to improve.

Let us revise what we learned today. Data science uses data to answer real questions. Its methodology has ten steps, from business understanding to feedback. Data preparation usually takes the most time. Our worked example predicted canteen plates and cut the waste. And the steps map neatly onto the AI Project Cycle.