Correlated electronic structure calculations enable accurate determination of the physicochemical properties of complex molecular systems. Nevertheless, the computational cost of these calculations sets constraints on their ability to be scaled up. The Fragment Molecular Orbital (FMO) method is widely recognised for its effectiveness in minimising computational costs while still achieving a high level of predicted accuracy. We introduce a novel distributed methodology and implementation of the modified FMO method called Coulomb-Perturbed Fragmentation (CPF) approach, which makes use of many GPUs. The objective is to enhance computational efficiency and accuracy. The study primarily conducted performance analysis on the Setonix system at the Pawsey Centre. The X23 datasets are examined utilising the FMO and CPF methods, which provide an extensive and varied standard for assessing and enhancing computational techniques. The approach demonstrates significant enhancements in velocity when compared to alternative GPU and CPU algorithms. Additionally, it demonstrates robust scalability on Setonix, attaining parallel efficiency rates of $98\%$ and $86\%$ on 8 and 64 nodes, respectively.