• Data Lake Project Lead: leading and contributing to the development, maintenance, and enhancing data quality of our current Data Lake and designing and planning our new version of Data Lake through close communication with Data Science use cases.
• CDC migration supervisor: guiding the migration from a Scala-based CDC pipeline to Qlik Replicate/Compose.
• Supervising and contributing to enhancing documentation and standardizing library and toolchains throughout the department.
• Contributed to enhancing team's procedures and communication within the department by preparing and reviewing RFCs.

• Contributed to the maintenance and development of Data Lake for the Data Science department
• Enhanced Data Lake's data quality by detecting and addressing tens of important data quality issues.
• Developed and extended Data Lake to adapt to the department's analytics needs.
• Contributed to enhancing team's procedures and communication within the department by preparing and reviewing RFCs.
• Mentored junior members of the team.

• Designed, developed and employed a large-scale data-intensive network provisioning system for ISPs called FlowDNS.
• Contributed to the design and development of a large-scale network monitoring system for ISPs and IXPs called Flowyager.
• Enhanced VPN traffic detection rate by 300% using active measurements of the whole IPv4 address space and IPv6 hitlists.
• Advanced the understanding of internet traffic using port number 0 by performing the first active measurement on this port, combined with large-scale flow-level analytics.

• Deployed and customized my self-designed FlowDNS system to align with customer requirements.
• Enabled real-time management of data using FlowDNS which was not possible using Kafka and ClickHouse without huge pre-assumptions.

Prototyping a system to correlate live streams of DNS and Netflow data in real-time written in Golang. Concurrenct goroutines needed to access on in-memory shared database. Choice of data structures and cleaning mechansism were important.

• Learned the complexities of data management including the large scale of data, data dependencies, and ethics in large ISPs.
• Designed and prototyped a system to manage and correlate two important sources of data in ISPs including Netflow and DNS live streams, trying multiple options, called FlowDNS.