Data Efficient Sequential Decision Making in High Dimensions

As machine learning (ML) systems rapidly advance, their scale and data requirements have surged, increasing the need for efficient data use while maintaining high performance and accuracy. This dissertation addresses the challenge of data efficiency in large-scale machine learning, particularly in sequential decision-making (SDM) problems. Numerous modern applications, from drug discovery to robotics to online recommendation systems, can be framed as SDM problems. While many frameworks exist for addressing SDM, this work focuses on two key paradigms: Federated Learning (FL) and Stochastic Bandits (SB). In both FL and SB, an agent learns iteratively by observing data, making decisions, and refining those decisions to increase cumulative reward. This dissertation aims to reduce the data exchanged per interaction round between the agent and the environment and to minimize the total data required to achieve optimal model performance. The main contributions include: first, a communication-efficient FL methodology to address bandwidth limitations, noisy communication, and client data heterogeneity, ensuring robust performance; and second, a partial client participation strategy that enhances data efficiency in large, distributed user settings. For SB, a method is introduced to leverage low-dimensional structures in high-dimensional, partially observed data, enabling effective learning despite incomplete information. Additionally, SB is extended to accommodate periodically changing reward patterns, adapting the model to non-stationary environments. These contributions advance data-efficient SDM strategies for complex, distributed environments, enabling scalable systems that adapt to dynamic, real-world contexts.