Adaptive real-time machine learning requires efficient reinforcement learning (how an algorithm should continuously interact with its environment to maximise its reward), online learning (dealing with continuous sequences of real-time data), and adaptive learning from a small sample size. This project aims at developing a series of real-time meta-learning algorithms that can be utilised to achieve continuous learning, predicting and controlling when functioning in a changing environment. The algorithm should function in a wide range of real-world applications, such as urban sensing, industry, and precision agriculture.
Explaining the science
Efficient reinforcement learning
Unlike supervised or unsupervised learning based on large amounts of unified and stationary datasets, reinforcement learning focuses on how an agent should continuously interact with its environment to maximise its reward. Under such a setting, massive trials are generally necessary before or during the learning process, and success still highly depends on manually crafted learning architectures and targets. This project aims at solving the sample inefficiency problem in existing approaches.
'Off-line' machine learning approaches typically learn from well-organised and stationary large-scale datasets through batch processing. In stream-oriented, sensor-based systems that are continuously delivering data, this would be unrealistic and inappropriate for many real-world infrastructure applications. Ideally, the learning of targeted hidden knowledge should be incremental, leveraging multi-source data streams. This project aims at developing online algorithms for continuous learning with guarantees of success.
Small-sample-size adaptive learning
Existing machine learning approaches generally train their models with massive amounts of data collected in stationary environments. However, such approaches are not applicable to the ever-changing environments typically found when sensor/actuator systems are deployed to make an infrastructure smart (e.g. smart cities, autonomous vehicles etc.). It's unfeasible to re-collect and re-organise the data and re-train the previously learnt models whenever the environment changes. This project aims at developing high level 'meta-learning' algorithms that can rapidly notice environmental changes based on a limited amount of sensing data samples, and continuously adjust the rest of the learning model accordingly.
The overarching aim is to deliver real-world application-driven research into real-time adaptive machine learning techniques, by developing the applicable theory, algorithms, architectures and applications. Theoretical foundations will drive the distributed online meta-learning algorithms that can run at the network edge. Potential solutions to specific real-world problems in fields like urban sensing, cyber-physical system (CPS) controlling and precision agriculture will also be explored.
The expected results in terms of effective reinforcement learning, online learning and small-sample-size adaptive learning will bridge the gap between the theoretical study of static batch machine learning and the wide range of real-world applications within changing environments.
This project is part of the Data-centric engineering programme's Grand Challenge of 'Resilient and robust infrastructure'.
The expected results can be applied to complicated systems which are difficult to explicitly describe, and where hidden knowledge about the system can be learnt continuously and adaptively. For example, promising applications include (but are not limited to) short-term rainfall prediction for urban infrastructure scheduling in smart cities, continuous control of CPS assets under changing environmental influences, and adaptive yield prediction based on real-time crop vigour analysis in precision farming.
Working with National Institute of Agricultural Botany's East Malling Research (NIAB EMR) station to deploy precision farming solution test.
Working with Housing and Development Board (HDB) Singapore to develop precise short-term rainfall prediction solutions for the city's water buffer scheduling and adaptive controlling.
Working with Ridgeview Winery to deploy precision farming solution test.
Researchers and collaborators
Dr. Cong Zhao - [email protected]