SDS Data Science Professional

A. Business problem identification and data readiness

A data science project should start with the business problem, not the data or the technique. Define the decision to be supported, the stakeholders, the measure of success and the constraints. A well-scoped problem is specific, measurable and linked to value, for example reducing the share of invoices paid late, rather than a vague aim such as using machine learning in receivables. Engage stakeholders early so that the work is relevant and so that people trust and adopt the result.

Data discovery and quality audit. Establish what data exists, where it is held, who owns it, how it was collected and how reliable it is. Check completeness, accuracy, consistency, timeliness, validity and uniqueness. Common problems are missing values, duplicates, inconsistent formats, outliers, bias in how data was collected and data that is not representative of the future. Poor data gives a poor model whatever the technique.

The CRISP-ML framework (CRISP-ML(Q) adds quality assurance) moves through business and data understanding, data preparation, modelling, evaluation, deployment, and monitoring and maintenance. It is iterative: evaluation may send the team back to data preparation or to a redefined problem, and a deployed model must be monitored because the world changes.

Key terms

• CRISP-ML: A process model for machine learning projects, from business understanding to monitoring.
• Data quality audit: A structured review of the completeness, accuracy and fitness of data.

Study tip: In a scenario, name the decision, the data and its weaknesses, and a stage of CRISP-ML in which the problem will be fixed.

Practice questions for section A

B. Statistics and probability

Descriptive statistics summarise data: the mean, median and mode, the range, variance and standard deviation, percentiles and quartiles, skew and the effect of outliers. The median is more robust to outliers than the mean. Correlation measures the strength and direction of a linear relationship between two variables from -1 to +1, and correlation does not prove causation. Regression estimates a relationship, and R squared gives the share of variation that the model explains.

Inferential statistics uses a sample to draw conclusions about a population. A confidence interval gives a range that is likely to contain the population value. In a hypothesis test, the null hypothesis is tested against an alternative, and the p-value is the probability of seeing data at least as extreme as the sample if the null hypothesis were true. A small p-value, below the chosen significance level such as 5 percent, leads to rejecting the null hypothesis. A Type I error rejects a true null hypothesis and a Type II error fails to reject a false one. Statistical significance is not the same as practical importance.

Probability and decision theory. Probabilities of independent events multiply, and for mutually exclusive events they add. Conditional probability and Bayes' theorem update a probability when new information arrives. Expected value weights outcomes by probability, and decision trees compare actions under uncertainty, with attitudes to risk (risk averse, neutral or seeking) affecting the choice. Business applications include forecasting demand, estimating default rates, sampling in audit and testing whether a change in process made a difference.

Worked figures

Five weekly values are 12, 15, 11, 18 and 14. The mean is 14.0 and the sample standard deviation is 2.74.

Bayes: 2 percent of invoices are fraudulent. A test flags 90 percent of fraudulent invoices and also flags 5 percent of genuine ones. The probability that a flagged invoice is fraudulent is (0.02 x 0.90) / (0.02 x 0.90 + 0.98 x 0.05) = 0.018 / 0.067 = 26.9 percent. Most flags are false alarms because genuine invoices are so common.

Key terms

• Standard deviation: A measure of how far values typically are from the mean.
• p-value: The probability of data at least as extreme as observed if the null hypothesis is true.
• Type I error: Rejecting a null hypothesis that is true.
• Bayes' theorem: A rule for updating a probability when new evidence is observed.

Study tip: Always interpret a result in business terms. A p-value on its own is not an answer.

Practice questions for section B

C. Programming for business: Python and SQL

Python is used for data preparation, analysis and modelling. Key ideas are variables and data types, lists and dictionaries, loops and conditions, functions, and libraries such as pandas for tables (data frames), NumPy for numerical work and scikit-learn for models. Typical tasks are loading a file, handling missing values, filtering rows, creating new columns, grouping and summarising, and plotting. In the exam you may need to read short code and say what it does, find an error or complete a step.

SQL queries relational databases. SELECT chooses columns, FROM names the table, WHERE filters rows before grouping, GROUP BY groups rows for aggregate functions such as SUM, COUNT and AVG, HAVING filters groups after aggregation, ORDER BY sorts, and JOIN combines tables on a key. An INNER JOIN keeps only matching rows, and a LEFT JOIN keeps all rows from the left table. Common mistakes are joining on the wrong key, which duplicates rows, and filtering with WHERE when HAVING is needed.

Good practice for business code: comment and document, keep the raw data unchanged, make steps repeatable, test the results against a control total, and protect personal data.

Key terms

• Data frame: A table of rows and columns used in pandas.
• JOIN: An SQL operation that combines rows from two tables using a matching key.
• Aggregate function: A function that summarises many rows, such as SUM or COUNT.

Study tip: When reading a query, go in the order that the database does: FROM, WHERE, GROUP BY, HAVING, SELECT, ORDER BY.

Practice questions for section C

D. Data science techniques and modelling

Data preparation covers cleaning, handling missing data (delete, impute or flag), treating outliers, encoding categories, scaling numerical features, creating new features and splitting the data into training, validation and test sets. Never let information from the test set leak into training.

Predictive modelling. Regression models predict a number, such as sales, and classification models predict a category, such as default or not. Methods include linear and logistic regression, decision trees, random forests and neural networks, and simple models are easier to explain, which may matter more than a small gain in accuracy. Unsupervised methods find structure without labels: clustering (for example k-means) groups similar customers, and anomaly detection flags unusual transactions.

Validation and interpretation. Overfitting occurs when a model fits noise in the training data and fails on new data, and underfitting when it is too simple. Use a held-out test set or cross-validation. For classification, the confusion matrix gives true and false positives and negatives, precision is the share of flagged cases that are correct, recall is the share of real cases that are found, and accuracy can mislead when one class is rare. Choose the measure that fits the cost of each type of error. Explain results in plain terms using feature importance and examples, and monitor deployed models for drift as data changes.

Worked figures

A fraud model reviews 1,000 transactions. It flags 50. Of those, 30 are real fraud. There are 40 real frauds in total. Precision is 30 / 50 = 60 percent. Recall is 30 / 40 = 75 percent. Accuracy is (30 + 940) / 1,000 = 97 percent, which looks excellent even though 25 percent of frauds were missed, because 960 transactions are genuine.

Key terms

• Overfitting: A model that fits the training data too closely and fails on new data.
• Precision: The share of positive predictions that are correct.
• Recall: The share of actual positives that the model finds.
• Model drift: A fall in model performance because the data or the world has changed.

Study tip: If a rare event is the target, say why accuracy is not enough and what you would use instead.

Practice questions for section D

E. Applications, governance, ethics and storytelling

Applications. In audit, analytics can test whole populations, find anomalies and unusual journals, and support risk assessment. In finance, forecasting, scenario modelling and variance analysis use models. In management accounting, they support costing, pricing and budgeting, and in sustainability reporting they help to collect and check emissions and other data. In each, identify the decision, the data, the method and the control over the output.

Governance, ethics and risk. Principles include fairness and the avoidance of bias, transparency and explainability, accountability for decisions made by a model, privacy and consent, security, and proportionate human oversight. Data protection law restricts how personal data may be used. Governance of models covers ownership, documentation, validation before use, change control, monitoring and an inventory of models. Risks are inaccurate or biased outputs, security breaches, misuse of personal data, over-reliance on a model and reputation damage.

Data storytelling. A message needs a clear point, a suitable chart (a line for trends over time, bars for comparison, a scatter plot for relationships), minimal clutter, honest scales and labelled axes. Explain uncertainty in plain language, for example a range instead of a single number and the chance of an outcome in words. Adapt the message to the audience: the board needs the decision and the risk, and technical staff need the method.

Key terms

• Explainability: The ability to describe in plain terms how a model reaches its result.
• Algorithmic bias: Systematic error that treats groups unfairly, often because of biased data.
• Model inventory: A record of the models in use, their owners, purposes and validation status.

Study tip: For a communication requirement, state the audience, the one message, and the chart you would use.

Practice questions for section E