Chat with us, powered by LiveChat

Big Data Systems

This subject is available under ICMS undergraduate degrees, please click the button below to find an undergraduate course for you.

Subject Code:

DAT203A

Subject Type:

Specialisation 

Credit Points:

3 credit points

Pre-requisite/Co-requisite: 

 DAT201A Applied Data Mining 

Course level of study pre-requisite: a total of 12 credit points including ICT101A, ICT102A, ICT103A and DAT101A from level 100 core subjects prior to enrolling into level 200 core and specialisation subjects. 

Subject Level:

200

Subject Rationale:

Due to the increasing importance of large and diverse volumes of data in today’s business world, organisations need systems and technologies to extract value from big data to enhance business decision making and remain competitive. Therefore, there is an increasing demand for data analytics graduates who can conceptualise, design, and implement big data systems. 

 The subject introduces big data and big data systems, their requirements and functionalities. It covers key areas such as databases and memory systems for big data, data privacy and security issues. As an essential enabler in managing big data systems, cloud computing will also be discussed along with the exploration of emerging trends and technologies in the big data systems domain. 

 The subject equips students with an understanding of how, when, and why these systems can be implemented and best used to solve specific business problems and create value within organisations. 

Learning Outcomes:

a) Identify the unique properties and attributes of big data sets and the need for adopting big data systems within organisations.

b) Evaluate various big data database systems and their appropriateness in specific business scenarios.

c) Plan the application of various big data systems to manage big data sets.

d) Analyse key issues involved in the security and privacy of big data systems.

e) Examine emerging trends and technologies in the domain of big data systems.

Student Assessment:

Broad Topics to be Covered:

Topic: 
Week 1: Introduction to Big Data 

  • Characteristics of big data  
  • Types of big data 
  • Overview of big data lifecycle 
  • Overview of big data architectures 
  • Overview of big data practice cases 
Week 2: Review of Big Data Systems 

  • Architectures of big data systems  
  • Lambda architecture 
  • Kappa architecture  
  • Parallelism and cluster architecture  
  • Memory systems  
  • Structure of traditional shared memory systems  
  • Introduction to distributed memory systems, edge, and fog computing  
Week 3: Big Data Storage Systems 

  • Database systems 
  • CAP Theorem 
  • ACID and BASE 
  • Shared vs. distributed memory  
  • Pre-processing and processing of big data 
  • Storage systems and methods  
Week 4: NoSQL Systems 

  • Relevance of NoSQL Systems 
  • Recap BASE  
  • Parallel RDBMS 
  • Types of NoSQL systems 
  • KD, DD, CD, and G databases 
Week 5: Big Data Frameworks 

  • Requirements  
  • HDFS 
  • Recovery functions 
  • Spark operation 
  • Spark SQL 
  • Spark Streaming 
Week 6: Cloud Computing 

  • Cloud Deployment models and workload management  
  • Layered architecture of cloud  
  • Platform as Service (P-as-S) 
  • Function as Service (F-as-S 
  • Software as Service (S-as-S 
  • Virtualisation 
Week 7: New SQL Systems 

  • From NoSQL to NewSQL 
  • Types of NewSQL systems 
  • Functions of NewSQL systems  
Week 8: Security for Big Data 

  • Attack types and mechanisms 
  • Attack detections and preventions  
  • ZTS model 
Week 9: Privacy for Big Data 

  • Direct privacy violation 
  • Indirect privacy violation  
  • Privacy protection solutions 
Week 10: Introduction to Common Big Data Platforms 

  • Microsoft Azure 
  • Amazon Web Service  
  • Oracle 
  • SAP HANA 
  • Cloudera 
Week 11: Future Trends in Big Data 

  • Availability 
  • Power efficiency 
  • Efficient network processing  
  • Heterogeneity and heterogenous environments  

Please note that these topics are often refined and subject to change so for up to date weekly topics and suggested reading resources, please refer to the Moodle subject page.