The inversion of sparse linear systems gives rise to dense matrices. Their computation poses not only a computational but also a memory bottleneck. Numerous applications from various fields require, however, only particular, i.e. selected entries of the complete inverse. Applications range from areas like statistical learning where the computation of marginal variances requires the selected inversion of the associated sparse precision matrices, to nano-electronics in device physics, where quantum transport simulations necessitate selected matrix inversions to model electron flows. The most common elements of interest in the inverse are the (block) diagonal elements or entries that correspond to non-zero elements in the original sparse matrix. We present a selected inversion algorithm for matrices with block tridiagonal arrowhead sparsity patterns which recovers all block diagonal entries of the full inverse. Our implementation relies on block-wise dense GPU computations and scales efficiently across multiple GPUs.