The Hitchhiker's Guide to Training on Compute Islands – Core Automation

This post argues that the real bottleneck for frontier AI training is not raw GPU supply but power, grid equipment and the software needed to combine scattered compute "islands." The authors report training a sparse mixture-of-experts model across two clusters roughly 9,000 miles apart, on different GPU types over a…