Vision Transformers for Geospatial Perception
Transformer architectures for satellite and aerial imagery.
A study of transformer variants for geospatial perception, benchmarked across segmentation, detection, and change-detection tasks on planetary-scale datasets. Results inform Vision Systems technology and continental inference serving strategies.
Architectures
Hierarchical and windowed attention for large images.
Benchmarks
Segmentation, detection, and temporal change.
Scaling
Compute-optimal recipes for geospatial pre-training.
Deployment
Tile-serving strategies for continental inference.
Motivation
Geospatial images dwarf typical vision inputs. Naive global attention is intractable; aggressive downsampling destroys thin infrastructure features. We compare hierarchical, windowed, and hybrid convolutional-transformer backbones under fixed compute budgets.
Benchmarks and findings
Across segmentation, object detection, and bi-temporal change detection, hierarchical attention consistently improves thin-object recall. We publish scaling curves for pre-training on multi-sensor corpora and show where additional parameters stop helping.
Serving implications
Continental inference requires tiling, overlap handling, and cache-aware schedulers. We describe a serving recipe later productized in Vision AI and Distributed Cloud for large-area jobs.
Discuss this work with our labs.