Businesses now collect data from phones, sensors, online sales, social media and bank systems at a speed and scale that older tools cannot handle. Big data and data analytics are the ideas and tools used to turn that flood into decisions.
This topic explains what big data is, its characteristics, the pyramid that shows how data becomes wisdom, the types and tools of analytics, and the benefits and risks of data mining.
Examiners have asked for the characteristics of big data, the benefits and uses of data mining, and the risks of analytics. Learn each list and be ready to explain every item in one sentence, with a practical example.
In 30 seconds
Big data means huge, complex and fast-growing data sets that traditional methods cannot manage. The syllabus describes it by four Vs: volume, velocity, variety and veracity. The big data pyramid has four layers: data, information, knowledge and wisdom (intelligence). Analytics comes in descriptive, diagnostic, predictive, prescriptive and cognitive forms.
Data mining and analytics bring insight, better decisions, better security and growth, but carry risks of poor data quality, privacy breaches, bias and misuse.
What big data is
Big data refers to massive, complex data sets that are hard to manage and analyse with traditional methods. It covers many data types, including structured and semi-structured data, and it often grows exponentially. More simply, it describes huge and continuously growing volumes of data that arrive in different formats, at different speeds and from different sources, such as photos, music and text.
It also covers the processes, tools and methods used to gain insight from that data. Big data analytics helps companies solve business problems that ordinary data analysis could not.
The four Vs: characteristics of big data
| Characteristic | Meaning | Illustration |
|---|---|---|
| Volume | The amount of data, from terabytes up to petabytes, coming from devices on the internet of things, social media, text files and business transactions. One petabyte equals 1,000,000 gigabytes, and big data involves thousands or millions of such units | Data from sensors, apps and online sales piling up every second |
| Velocity | The speed and frequency at which data is produced and must be handled, from batch reporting to near real-time processing and streaming | A vehicle with a telematics device that sends about 25 gigabytes every hour, most of which must be handled in real time or near real time |
| Variety | The diversity of the data. It is not only structured rows and columns in relational databases. Most big data is unstructured | A WhatsApp message may contain text, video, images and a timestamp |
| Veracity | How far the data can be trusted. Data may be repeated, incomplete, inconsistent or noisy, and its source may be unclear, which lowers the accuracy of analysis. It is often classed as good, bad or unstructured | Medical records, where inconsistency or ambiguity can cause harm |
The syllabus lists the characteristics as volume, velocity, veracity and value. Variety is explained more fully than value. Learn all five names, so that you can answer a question that asks for “five characteristics”. Value means the worth that the organisation can extract from the data, though the term is not developed further. Background: value is named in the syllabus but is not developed there.
Real-life uses and tools
Big data analytics is used in predictive modelling for customer relationship management, market research, product design, sales forecasting, human capital management and audit work. The tools listed for analysing it are Microsoft Excel, structured query language (SQL), QlikView, RapidMiner, IDEA and ACL.
Examples of big data sources
- Social media such as Facebook, Twitter and Instagram.
- Sensor data from internet of things devices and weather sensors.
- E-commerce transactions on sites such as Amazon and eBay.
- Financial data from stock markets and banking.
- Scientific data such as genomics and astrophysics.
- Log files from web servers and applications.
The big data pyramid
The big data pyramid shows how raw data is refined step by step into something a manager can act on. It builds on Jennifer Rowley’s 2007 work on the relationships between data, information, knowledge and wisdom or intelligence. It is a metaphor for how data is structured, analysed and finally used to reach informed outcomes.
| Layer | What it is | Purpose |
|---|---|---|
| Data | Raw, unprocessed facts from social media, sensors, financial systems and so on. Plentiful but noisy, unstructured and not immediately useful | Collection and storage, so data is available for analysis |
| Information | Data that has been organised, categorised and put in context through cleaning, integration and presentation. Typical outputs are structured data sets, charts and simple dashboards | Make data understandable so that people can see patterns, trends and correlations |
| Knowledge | The result of deeper analysis using models, algorithms and subject expertise, for example customer behaviour trends or the root causes of inefficiency | Generate insights that can inform strategy and improve processes |
| Wisdom or intelligence | Knowledge applied with foresight and strategic intent, tied to business goals, enabling predictive modelling, scenario planning and real-time decisions | Support informed decisions, innovation and competitive advantage, bridging data science and strategy |
Purposes of the pyramid
- Guiding data use. It shows how to refine data step by step so it does not sit unused as an overwhelming mass.
- Enhancing decision making. It promotes decisions based on reliable insight rather than intuition or incomplete information.
- Aligning strategy. It links data collection, storage and analysis to organisational goals.
- Supporting scalability. It gives a model that keeps working as data grows in volume and complexity.
- Fostering innovation. Insight from the top layers feeds new opportunities and better supply chains.
Challenges in using the pyramid include poor data quality, a shortage of skilled staff and inadequate technology. A smooth flow between layers also needs strong governance, cooperation and clear objectives.
Data analytics
Data analytics is the process of examining raw data to reach conclusions, using algorithmic or mechanical processes to find insight.
| Type | Question it answers | Example |
|---|---|---|
| Descriptive | What happened? It summarises historical data | Sales reports, website traffic analysis |
| Diagnostic | Why did it happen? It looks for root causes | Finding why sales fell |
| Predictive | What will happen? It uses statistical models and machine learning | Predicting customer churn, forecasting sales |
| Prescriptive | How can we make it happen? It recommends actions | Optimising a marketing campaign, suggesting stock levels |
| Cognitive | It copies human thinking to automate tasks, learn from data and decide | Chatbots, fraud detection systems |
Technologies for storing, processing and analysing
- Hadoop. An open-source framework that stores and processes large data sets across clusters of computers. Its components are HDFS (a distributed file system), MapReduce (a model for parallel processing) and YARN (resource management and job scheduling).
- Spark. A fast in-memory processing engine, often used instead of MapReduce, especially for repeated (iterative) algorithms.
- Cloud storage and computing. Services such as Amazon S3, Azure Blob Storage and Google Cloud Storage give scalable, cost-effective storage and computing power.
- NoSQL databases. Non-relational databases for large volumes of unstructured and semi-structured data, such as MongoDB, Cassandra and Couchbase.
- Analytics tools. Programming languages (Python with Pandas, NumPy and Scikit-learn, and R), visualisation tools (Tableau, Power BI, QlikView), machine learning platforms (TensorFlow, PyTorch, Azure Machine Learning, AWS SageMaker), integration tools (Apache Kafka, Apache NiFi) and data warehouses (Snowflake, Amazon Redshift, Google BigQuery).
Key concepts in data analytics
| Concept | Meaning |
|---|---|
| Data mining | Finding patterns and knowledge in large data sets using statistics, machine learning and database techniques |
| Machine learning | A type of artificial intelligence in which computers learn from data without being explicitly programmed |
| Supervised learning | Training a model on labelled data to predict outcomes |
| Unsupervised learning | Finding patterns and structure in unlabelled data |
| Reinforcement learning | Training an agent to make decisions in an environment so as to maximise a reward |
| Artificial intelligence | The wider idea of machines doing tasks that normally need human intelligence |
| Deep learning | Machine learning with artificial neural networks of many layers |
| Data visualisation | Showing data in graphs and charts to make patterns easier to see |
| Data wrangling or cleaning | Cleaning, transforming and preparing data, including handling missing values, correcting errors and formatting consistently |
| Statistical analysis | Using statistical methods to analyse data and draw conclusions |
| ETL | Extract, transform, load: pulling data from sources, converting it to a consistent format and loading it into a data warehouse or other store |
Applications, benefits and uses
Applications
- Business and health: customer relationship management, marketing automation, supply chain optimisation, fraud detection, risk management, healthcare, personalised medicine, drug discovery, predictive diagnostics and healthcare fraud prevention.
- Government: smart cities, law enforcement, national security and public health.
- Science: climate modelling, genomics research, astronomy and materials science.
Benefits of data mining and analytics
- It uncovers valuable insights.
- It improves decision making.
- It enhances security.
- It drives business growth.
- It spots anomalies in large data sets, so that actions are better informed and the firm gains a competitive advantage.
Uses of data mining and analytics
- Extracting valuable knowledge and insight from large data sets.
- Prediction.
- Fraud detection.
- Customer segmentation and market analysis.
- Analysing customer behaviour, preferences and trends.
- Improving customer experience and developing better products.
Risks and challenges
Data mining and analytics solve problems that traditional analysis cannot, but the risks relate mainly to security, privacy, data quality and misuse of information. They can lead to personal data breaches, inaccurate analysis and illegal or unethical activity. The main challenges are shown below.
| Challenge | Meaning |
|---|---|
| Data quality | Making sure data is accurate and reliable |
| Data security and privacy | Protecting sensitive data from unauthorised access and misuse, and complying with rules such as General Data Protection Regulation (GDPR) and CCPA |
| Data governance | Setting policies and procedures for managing data |
| Scalability | Building systems that cope with growing data |
| Skills gap | Finding and keeping qualified data scientists and analysts |
| Data silos | Data trapped in separate departments or systems, which prevents a complete view |
| Interpretability | Making sense of complex models. “Black box” models are hard to trust |
The syllabus’s sample answer adds further risks an organisation faces when applying data mining: making sure the algorithms are fair and unbiased, validating models for accuracy and reliability, communicating results to stakeholders, complying with relevant laws, facing penalties for breaching data privacy rules, and weighing the costs and benefits of the technology.
Future trends
- Edge computing. Processing data near its source, which reduces delay and bandwidth needs.
- Artificial intelligence (AI)-powered analytics. Using AI to automate analysis and give deeper insight.
- Data fabric. One architecture giving consistent access and management across scattered data environments.
- Cybersecurity analytics. Using big data to detect and prevent cyber threats.
- Quantum computing. Possibly allowing much faster and more complex calculations.
The ideas in more depth
How distributed computing handles big data
One computer cannot store or process a data set of petabytes. Distributed computing splits a heavy task into smaller sub-tasks that run at the same time on many machines. Hadoop’s HDFS divides large files into blocks and spreads them over the cluster, and MapReduce processes those blocks in parallel. The approach is faster, can survive the failure of a machine, and scales by adding more machines.
From data to wisdom in a business example
A supermarket chain records every till transaction (data). A weekly report totals sales by branch and product (information). Analysts discover that sales of bottled water fall when the temperature drops below a certain level and that one branch loses sales when it runs out of stock on Fridays (knowledge).
The chain then builds a model that predicts demand, sets stock levels for each branch and schedules deliveries before shortages occur (wisdom or intelligence). Each layer adds value.
Choosing the type of analytics
Start with descriptive analytics to see what happened, use diagnostic analytics to learn why, predictive analytics to estimate what will happen, and prescriptive analytics to decide what to do. Cognitive analytics automates the whole cycle for tasks such as fraud detection.
Why accountants care about big data
Accountants use analytics for forecasting sales, detecting fraud, analysing costs and supporting audit work. But analysis is only as good as the data. A model built on duplicate, incomplete or biased records can give confident but false answers, which is why veracity and data quality come first.
Worked example: volume and velocity
Scenario. Kaduna Haulage Ltd has 40 trucks, each with a telematics device that sends 25 gigabytes of data every hour. The trucks run 24 hours a day. (a) How much data does the fleet generate in one day, in gigabytes and terabytes? (b) How much in 30 days, in petabytes? (c) Which of the four Vs does this illustrate, and what other challenges apply? Use 1 terabyte = 1,000 gigabytes and 1 petabyte = 1,000,000 gigabytes.
- Data per hour.
40 trucks × 25 GB = 1,000 GB per hour. - Data per day.
1,000 GB × 24 hours = 24,000 GB, which is 24 terabytes. - Data in 30 days.
24,000 GB × 30 = 720,000 GB.
Divide by 1,000,000 to get 0.72 petabytes. - Which Vs. The size is volume. The constant stream, much of which must be processed in real time, is velocity. If the data also includes video from dashboard cameras and text from driver messages, it shows variety. Faulty sensors or duplicate readings affect veracity.
- Other challenges. The firm needs scalable storage, such as cloud storage, secure handling of driver and location data, and skilled analysts. It should also confirm that the benefits, such as predicting breakdowns, outweigh the cost.
Common mistakes in the exam
- Naming the Vs without explaining them.
- Confusing information with knowledge in the pyramid. Information is organised data. Knowledge comes from analysing that information.
- Saying big data is only about size. Speed, variety and trustworthiness count too.
- Listing benefits of data mining and forgetting the risks when both are asked for.
- Confusing predictive analytics (what will happen) with prescriptive analytics (what to do).
- Writing that all big data is structured. Most is unstructured.
Quick revision
- Big data: huge, complex, fast-growing data that traditional methods cannot handle. Four Vs: volume, velocity, variety, veracity (the syllabus also names value).
- Pyramid: data, information, knowledge, wisdom or intelligence.
- Analytics: descriptive, diagnostic, predictive, prescriptive, cognitive.
- Benefits: insight, better decisions, security, growth, anomaly detection.
- Risks: data quality, security and privacy, governance, scalability, skills gap, silos, interpretability.
Practice questions
These are original questions written in the style of the Institute of Chartered Accountants of Nigeria (ICAN) exam. Try each one before you open the answer.
Section A style: multiple choice
Question 1. The speed at which big data is generated and must be processed is its:
- Volume
- Velocity
- Variety
- Veracity
Show the answer
B. Velocity is the speed and frequency of data.
Question 2. Data that is incomplete, inconsistent or noisy affects which characteristic?
- Volume
- Velocity
- Variety
- Veracity
Show the answer
D. Veracity is about how far the data can be trusted.
Question 3. In the big data pyramid, the base layer is:
- Wisdom
- Knowledge
- Information
- Data
Show the answer
D. Raw data forms the base. Wisdom or intelligence is at the top.
Question 4. Organising and contextualising raw data into charts and simple dashboards produces:
- Information
- Wisdom
- Raw data
- Noise
Show the answer
A. That is the information layer.
Question 5. Which type of analytics answers the question “why did it happen?”
- Descriptive
- Diagnostic
- Predictive
- Prescriptive
Show the answer
B. Diagnostic analytics looks for root causes.
Question 6. Forecasting next quarter’s sales with statistical models is:
- Descriptive analytics
- Predictive analytics
- Diagnostic analytics
- Reinforcement learning
Show the answer
B. Predictive analytics estimates future outcomes.
Question 7. ETL stands for:
- Extract, transform, load
- Estimate, test, learn
- Evaluate, transfer, link
- Enter, total, list
Show the answer
A. It moves data from sources into a consistent format in a warehouse.
Question 8. Which of these is a risk of data mining and analytics?
- Improved decision making
- Breaches of personal data privacy
- Better fraud detection
- Customer segmentation
Show the answer
B. The others are benefits or uses.
Question 9. Data trapped in different departments, which stops a complete view of the organisation, is called:
- Data silos
- Data fabric
- Edge computing
- Deep learning
Show the answer
A. This is one of the major challenges.
Question 10. A fleet of 20 vehicles each sends 25 GB an hour. The daily data for 24 hours is:
- 500 GB
- 6,000 GB
- 12,000 GB
- 24,000 GB
Show the answer
C. 20 × 25 = 500 GB per hour, and 500 × 24 = 12,000 GB.
Section B style: written questions
Question 11 (10 marks). State and briefly explain five characteristics of big data.
Question 12 (20 marks). A retail group wants to use big data and analytics.
(a) Explain what is meant by big data. (3 marks)
(b) Explain the four layers of the big data pyramid. (8 marks)
(c) Explain the five types of data analytics, with an example of each. (5 marks)
(d) State four challenges of using big data and analytics. (4 marks)
A further written question
Question 13 (10 marks). (a) State three benefits and three uses of data mining and analytics. (6 marks) (b) A firm has 50 trucks each sending 20 gigabytes of data every hour for 24 hours a day. Calculate the data produced per day in terabytes and the data produced in 10 days in petabytes. (4 marks)
Model answers
Show the model answer to Question 11
1. Volume. The sheer amount of data, from terabytes to petabytes, coming from devices, social media, text files and business transactions.
2. Velocity. The speed and frequency at which data is generated and processed, from batch reports to real-time streaming. For example, a connected vehicle may send about 25 gigabytes an hour.
3. Variety. The diversity of data types. Besides structured data in rows and columns, most big data is unstructured, such as text, images, video and messages.
4. Veracity. How reliable the data is. Data may be repeated, incomplete, inconsistent or noisy, and its origin may be unclear, which reduces the accuracy of analysis.
5. Value. The worth the organisation can get from the data. It is named in the syllabus as a characteristic: data is worth collecting only if analysis turns it into useful decisions.
Show the model answer to Question 12
(a) Big data means massive, complex data sets, in structured, semi-structured and unstructured forms, that are hard to manage and analyse with traditional methods and that often grow exponentially.
(b) The pyramid. Data is the base: raw, unprocessed facts from sources such as social media, sensors and financial systems. The purpose is collection and storage. Information is data that has been organised, cleaned and put in context, shown in data sets, charts and dashboards, so that patterns and trends can be seen.
Knowledge comes from deeper analysis of the information using models, algorithms and expertise, for example customer behaviour trends, and its purpose is actionable insight. Wisdom or intelligence is at the top. It applies knowledge with foresight and links it to business goals, enabling prediction, scenario planning and real-time decisions.
(c) Types of analytics. Descriptive shows what happened, such as a sales report. Diagnostic explains why, such as finding the cause of a sales decline. Predictive forecasts what will happen, such as predicting customer churn. Prescriptive recommends what to do, such as suggesting stock levels. Cognitive copies human thinking to automate tasks, such as chatbots and fraud detection.
(d) Challenges. Any four of: data quality; data security and privacy; data governance; scalability; the skills gap; data silos; and interpretability of complex models.
Show the model answer to Question 13
(a) Benefits: it uncovers valuable insight, improves decision making and enhances security (also drives growth and finds anomalies). Uses: prediction, fraud detection and customer segmentation and market analysis (also analysing customer behaviour and improving products).
(b) Calculation.
- Per hour.
50 × 20 = 1,000 GB. - Per day.
1,000 × 24 = 24,000 GB, which is 24 terabytes. - For 10 days: 24,000 × 10 = 240,000 GB.
- Dividing by 1,000,000 gives 0.24 petabytes.
Pathfinder on Big data and data analytics
These are the ICAN Pathfinder questions that test this topic. ICAN owns the questions and the suggested solutions, so we point you to them and do not copy them. Open your Pathfinder, attempt each question under exam conditions, then check your answer against the notes above.
Part (b), worth 10 marks, asks for five characteristics of big data with a brief explanation of each. Part (a) is a cost coding question that belongs to a later topic.
Revise first: The four Vs: characteristics of big data
Multiple-choice items can test the four Vs, the layers of the big data pyramid, the types of analytics and the risks of data mining. Check Section A of every diet you practise.
Revise first: The big data pyramid
Source: ICAN Pathfinder, Management Accounting. Diets covered: May Diet, 2026. Big data is new to this syllabus, so older diets are not mapped. May Diet, 2025 and November Diet, 2024 are not covered here.
Spotted a mistake? Tell us and we will check it.
