Machine Learning Driven Web Log Tracing system
Project Description

Use behavior analysis is nowadays the challenge for IT management on how to stop network security incident. The key is to determine if the user account is legitimate or credential is already been hacked.

Many of the corporations considered that implementing of log management system is equivalent to the use of big-data platform for log management and analysis. However, that is just a mis-understanding of true big-data log management platform.

Besides, collection of web logs cannot immediately generate alerts and are rather difficult to identify whether logs are generate from human, bots or attacker.

Machine learning is specialised to be used for determining patterns of user behaviour and web logs can be used for tracing & tracking user activities determine the baseline and profiling of user patterns.

IEONG Sze Chung Ricci
IEONG Sze Chung Ricci
Course type
Applicant's Roles

In this research, applicants will have to handle the following:
- Setup and implement the log processing in ElasticSearch, Logstash and Kibana
- Analyse the web logs and define the data model for web log analysis
- Define mechanism for determining whether logs are generated by bots
- Define the web log distribution pattern and detect abnormality based on pattern as well as risk score evaluation

Applicant's Learning Objectives

- To use current open source big-data log analysis platform
- To perform security log data analytics
- To identify web log distribution pattern
- To understand mechanism of abnormal attack patterns

Complexity of the project