Big Data Principles and the Big Data Pyramid (DIKW) for Management Decisions

A logistics company in Lagos fits a tracking device to every truck. Each device sends position, speed, engine temperature and fuel use every few seconds, all day. A bank records every card swipe, app login and branch visit of every customer. A telecoms firm stores millions of messages, call records and social media posts. Nobody can read this by eye, and an ordinary spreadsheet will not even open it.

Yet buried in it are answers to questions managers have always asked: which routes lose money, which customers are about to leave, which machines are about to fail.

This topic explains what big data is, the five Vs that define it, how it differs from the traditional data that accountants have always used, the principles for managing it well, and the data, information, knowledge and wisdom pyramid that shows how raw data becomes a decision. It builds on the social and ethical issues topic.

The next topic covers the uses, benefits, challenges and risks of big data, data mining and data analytics.

What big data is, and the five Vs

Big data refers to massive, complex data sets that are difficult to manage and analyse with traditional methods. They include structured, semi-structured and unstructured data, and they often grow very fast. The term also covers the processes, tools and approaches used to gain insight from the data. Big data analytics helps a business deal with problems that traditional analysis could not solve.

• Volume. The sheer quantity of data, from terabytes to petabytes, coming from sensors, social media, text files and business transactions. One petabyte is 1,000,000 gigabytes, which is more than 300,000 high-definition films, and big data means thousands or millions of such units.
• Velocity. The speed and frequency at which data is produced, and so the speed at which it must be processed: batch reports, near real-time or real-time processing, and data streaming. A truck with a tracking device may send about 25 gigabytes an hour, and most of it must be handled at once.
• Variety. The range of forms. Structured data sits in rows and columns in relational databases. Semi-structured data, such as files in standard exchange formats, has some organisation. Unstructured data, such as text, images, video and messaging posts, has none, and most big data is unstructured.
• Veracity. The quality, accuracy and trustworthiness of the data. Data may be repeated, incomplete, inconsistent or noisy, and its origin may be unclear. This is the most contentious V, because poor data weakens the analysis, and in areas such as medical records an inconsistency can do harm.
• Value. The insight and business outcome that analysis delivers. Data that cannot be turned into a better decision has no value, however big it is.

Early descriptions used four Vs (volume, velocity, variety and veracity). Value is the fifth. In an exam, give the five and say which version you are using.

Big data compared with traditional data

Traditional data is structured data that is easy to organise, store and maintain: accounting records, sales information and production results, held in ordinary databases and spreadsheets. Its advantages are easy storage and management, efficient access because of its structure, and low cost for small data sets. Its disadvantages are that it exists only in structured form, handles text, images and video badly, and is hard to scale when inputs grow.

• Volume. Traditional data is small to medium and can be processed by traditional technology. Big data is too large for it.
• Variety. Traditional data is structured in tables. Big data can be structured, semi-structured or unstructured.
• Velocity. Traditional data is relatively static and updated periodically. Big data changes constantly, in real time or near real time.
• Complexity. Traditional data is simple to manage and analyse. Big data needs specialised tools and techniques.
• Value. Big data has greater potential value, because it can show customer behaviour, market trends and other insights that small data sets cannot.

They also have things in common: data quality matters in both, both need analysis (statistics and charts for traditional data, machine learning and other advanced methods for big data), both must be stored, secured and managed, and both can give business value, one from historical patterns and the other from new opportunities.

The data pyramid (DIKW)

The pyramid shows the relationship between data, information, knowledge and wisdom. Each step answers a different question about the original data and adds value.

• Data: raw, unprocessed facts and figures without context. It is the foundation.
• Information: data that is organised, structured and given context, answering who, what, where and when.
• Knowledge: the result of analysing and interpreting information to find patterns, trends and relationships, answering how and why.
• Wisdom: the ability to make well-informed decisions and take effective action based on the knowledge.

Example. A supermarket chain records every till transaction (data). Daily sales by branch and product are totalled (information). Analysis shows that sales of cooking oil in one city rise sharply in the last week of each month, when salaries are paid (knowledge). Management decides to stock more and schedule extra staff for that week, and to agree supplier deliveries in advance (wisdom).

The purposes of the pyramid are to give a framework for the levels of meaning in data; to support data-driven decisions; to give people a common language for communication and collaboration; to be a foundation for analytics, business intelligence and knowledge management; and to help people see how their skills fit into a data-driven organisation.

Principles for managing big data

• Data quality. Accuracy (no errors or inconsistencies), completeness (all necessary data present), consistency (uniform across sources and formats) and timeliness (up to date). Data profiling continuously analyses data for patterns and errors, and data cleansing corrects problems. Poor data gives poor results: garbage in, garbage out.
• Accessibility. Availability to authorised users when needed, ways to find the right data, integration of data from different sources into one view, and controlled sharing for collaboration.
• Security. Protection from unauthorised access, use, disclosure, disruption, modification and destruction; access control by role; encryption of sensitive data; back-up and recovery; compliance with privacy law and industry standards.
• Ethics. Privacy of the people whose data is used; transparency about collection, use and sharing; fairness, so that data is not used to discriminate; accountability, with clear responsibility for data practices; and informed consent.

Other key considerations are data governance (policies and procedures for managing data as an asset), scalability and performance (systems that handle large volumes and complex analysis), and skills and expertise (training and hiring people who can manage and analyse big data).

Where big data is applied

• Business: understanding customer behaviour, optimising supply chains, detecting fraud, managing risk and forecasting.
• Finance: credit risk assessment, fraud detection, algorithmic trading.
• Marketing: targeted advertising, product recommendations, customer segmentation.
• Healthcare: predicting disease, personalised treatment, drug discovery, medical image analysis.
• Transport and public administration: traffic management, public health monitoring, crime prevention, resource management.

Data protection rules for big data in Nigeria

Big data projects often use personal data, so the Nigeria Data Protection Act 2023 applies, with the General Application and Implementation Directive 2025 (in force from 19 September 2025). The main points are these.

• Principles (section 24). Personal data must be processed lawfully, fairly and transparently; collected for specified purposes; limited to what is necessary (data minimisation); accurate; kept no longer than needed; and processed securely with accountability. Collecting everything because storage is cheap breaches data minimisation.
• Lawful basis and consent (sections 25 and 26). Processing needs a lawful basis. Where consent is relied on, the controller must be able to prove it, and silence or inactivity is not consent.
• Impact assessment (section 28). A data protection impact assessment is needed before processing that is likely to be high risk, which includes large-scale processing, new technologies and profiling.
• Automated decisions (section 37). A person has the right not to be subject to a decision based solely on automated processing, including profiling, that has legal or similarly significant effects, subject to stated exceptions.
• Security and breach (sections 39 and 40). Appropriate security measures are required, and a breach likely to put people at risk must be reported to the Commission within 72 hours.
• Cross-border transfers. Sending personal data abroad, for example to a foreign cloud, is allowed only where the receiving country provides adequate protection or safeguards are in place.

These duties are costs and constraints on any big data project and belong in a cost-benefit appraisal. Confirm current Commission notices before advising a real client.

Sources and references

• Nigeria Data Protection Act 2023, sections 24 to 26, 28, 37, 39 and 40, and Part VIII (cross-border transfers).
• Nigeria Data Protection Commission, General Application and Implementation Directive 2025.

These points are stated as at 6 November 2026.

Measuring volume, velocity and data quality

Questions ask you to size the data a business will generate, cost its storage, and measure data quality. The layouts are:

1. Daily data. Number of devices × data per device per hour × hours per day.
   Use 1 terabyte (TB) = 1,000 gigabytes (GB) and 1 petabyte (PB) = 1,000 TB.
2. Period volume. Daily data × days in the period.
3. Storage cost. Terabytes held × cost per terabyte per month.
4. Completeness %. Complete records ÷ total records × 100.
5. Duplicate %, invalid %. Duplicate (or invalid) records ÷ total records × 100.
6. Timeliness %. Records updated within the required time ÷ total records × 100.
7. Cost of poor data. Number of bad records × cost per record per use × number of uses.

Money is in ₦ million to two decimal places unless stated, percentages to one decimal place, and each answer is rounded once, at the end.

Worked examples

1. Daily data.
   120 × 25 × 10 = 30,000 GB = 30 TB.
2. 30-day volume.
   30 TB × 30 = 900 TB.
3. Monthly storage cost.
   900 × ₦38,000 = ₦34,200,000 = ₦34.2m.
4. Annual data.
   30 TB × 365 = 10,950 TB = 10.95 PB.
5. Comment. The volume and velocity make traditional tools useless. The firm should decide which data has value (engine faults, route times) and discard or summarise the rest, because keeping everything costs money and, where it includes personal data about drivers, breaches the minimisation principle.

1. Completeness.
   7,432 ÷ 8,000 = 92.9%.
2. Duplicates.
   264 ÷ 8,000 = 3.3%.
   Invalid. 160 ÷ 8,000 = 2.0%.
3. Timeliness.
   7,120 ÷ 8,000 = 89.0%.
4. Waste on duplicates.
   264 × ₦450 × 12 = ₦1,425,600 = ₦1.43m.
5. Waste on invalid records.
   160 × ₦450 × 12 = ₦864,000 = ₦0.86m.
6. Total. About ₦2.29m a year, before counting the harm to customer relations. Data profiling and cleansing cost money, but if they cost less than ₦2.29m they pay for themselves.

Expect variants: a different unit (megabytes, petabytes), several device types, storage priced per gigabyte, or quality measures tested against thresholds.

Common mistakes

• Listing the Vs without examples. Give a business example for each.
• Saying big data is just a lot of data. It is also fast, varied and uncertain, and the value comes from analysis.
• Confusing data, information and knowledge. Data is raw; information has context; knowledge explains patterns; wisdom is the decision.
• Forgetting veracity. More data does not mean better data.
• Mixing up units. One terabyte is 1,000 gigabytes and one petabyte is 1,000 terabytes in this working.
• Ignoring privacy law. Personal data in a big data project brings the Nigeria Data Protection Act 2023 into the answer.

Quick revision

• Big data: massive, complex data sets and the tools to gain insight. Five Vs: volume, velocity, variety, veracity, value.
• Traditional data: structured, smaller, relatively static, simple, cheap to hold.
• Principles: quality (accuracy, completeness, consistency, timeliness, profiling, cleansing), accessibility, security, ethics (privacy, transparency, fairness, accountability, consent).
• DIKW: data (raw), information (context), knowledge (patterns), wisdom (judgement and action).
• Nigeria Data Protection Act 2023: section 24 principles, section 28 impact assessment, section 37 automated decisions, section 40 breach notice within 72 hours.

How the marks are earned

• State: one mark per item, with an example.
• Distinguish: a point for each side of each difference.
• Explain the pyramid: each level defined, with the question it answers and a business example.
• Calculate: show units and workings; give the conclusion.

Practice questions

These are original questions, written in the style of the Skills examination. Work each one before you open the answer.

Recall check: multiple choice

Section B style: written questions

Model answers

Spotted a mistake? Tell us and we will check it.