Abstract
In today’s era, mobile and embedded devices at the network’s edge generate unprecedented amounts of data. To efficiently utilize the vast amount of data from decentralized devices, our focus shifts to Decentralised Machine Learning (DML), a framework where participants or staff members are trained to collaborate on models in a peer-to-peer (P2P) architecture. This approach bypasses centralized parameter servers (PSs) by facilitating direct model exchange between staff members, bypassing the traditional bottlenecks associated with centralized PSs. Despite its advantages, DML encounters several significant obstacles, including constraints on resources, diversity of system functionality, fluctuating network conditions, and the challenges of non-iid data distribution. To address these issues, this study proposes a novel DML mechanism to improve efficiency. We introduce a system where heterogeneous workers are assigned different pruning ratios, allowing the training and sharing of sub-models tailored to the specific resource capabilities of each worker, thus minimizing computational and communication burdens. Through rigorous theoretical testing, we formulate an optimization algorithm that coordinates the determination of pruning ratios with the structure of the network topology, delicately balancing the need for resource expenditure with the requirement for training efficiency.