← Blog
#AWS#Kubernetes#Docker#DevOps#Cloud Infrastructure

AWS SOCI Parallel Pull: Cutting Container Startup Time by ~60% on EKS

webhani·

The bottleneck nobody budgets for

When people estimate autoscaling latency, they think about scheduler decisions, readiness probes, or JIT warm-up. They rarely account for the image pull. But for many workloads — especially anything bundling model weights or large ML dependencies — pulling and unpacking the container image accounts for more than 75% of total startup time. A 5-10 GB inference image on a cold node can take minutes before the container process even starts.

AWS's Seekable OCI (SOCI) snapshotter has addressed part of this for a while through lazy loading: a container can start before the full image is downloaded, pulling only the layers it actually touches first. The newer piece — Parallel Pull mode — attacks the rest of the problem, and it's worth understanding because AWS also upstreamed the underlying unpacking work into containerd itself, which means the benefit isn't locked to EKS.

What Parallel Pull mode actually changes

SOCI works by building an index over an image's layer contents, so the snapshotter knows which files live in which chunk without downloading the whole layer first. Parallel Pull mode builds on that index in two ways:

  1. Parallel chunk downloads — instead of pulling layers serially over a single connection, multiple chunks download concurrently over separate connections.
  2. Parallel unpacking — decompression and filesystem writes, previously a serial step after download, now happen concurrently with remaining downloads.

The unpacking piece was contributed upstream into containerd's core, so it's not an AWS-only patch — any containerd-based runtime benefits once the feature lands in a released version, not just SOCI users on EKS.

Combined, AWS reports close to a 60% reduction in startup time for containers with large model files — the category that benefits most, since the win scales with image size.

Turning it on for EKS

SOCI is enabled per node group, not cluster-wide, which is useful — you can pilot it on the node pools that actually run large images (inference, batch ML) without touching everything else.

# Example: enabling the SOCI snapshotter on an EKS managed node group (Terraform)
resource "aws_eks_node_group" "inference" {
  cluster_name    = aws_eks_cluster.this.name
  node_group_name = "inference-gpu"
  node_role_arn   = aws_iam_role.node.arn
  subnet_ids      = var.subnet_ids
 
  launch_template {
    id      = aws_launch_template.soci_enabled.id
    version = "$Latest"
  }
}
 
# In the launch template's bootstrap user-data, enable the snapshotter:
# --local-disk-max-images 50 --soci-snapshotter-enabled true

The exact bootstrap flag naming can shift between EKS AMI releases, so check the current EKS optimized AMI documentation before copying this into production — treat the snippet above as the shape of the change, not a literal drop-in.

Where this matters, and where it doesn't

This is a clear win for:

  • Spiky inference workloads where you scale pods up from zero or near-zero and can't afford to keep everything warm just to avoid pull latency.
  • CI/CD environments that spin up short-lived pods per job — faster pulls directly shorten pipeline time.
  • Large, infrequently-changing base images (CUDA, PyTorch, bundled model weights) where the layer index amortizes across many pulls from the same registry.

It matters less for small, frequently-changing application images — a 100 MB Node.js service image is already fast enough that pull time isn't your bottleneck; the added CPU and network overhead of chunked parallel downloads isn't worth it there.

What to actually check before rolling it out

  • Registry compatibility — you need to build or convert images with a SOCI index (via soci CLI or your build pipeline) for the lazy-loading and parallel benefits to apply; a plain OCI image without an index just falls back to a normal pull.
  • Network and CPU headroom during pull — parallel chunk downloads and parallel unpacking both consume more concurrent bandwidth and CPU on the node during the pull window. On network-constrained node types this can offset some of the gain.
  • Benchmark on your own images — the ~60% figure is AWS's number for large AI/ML images; a general-purpose web app image won't see the same ratio. Measure before and after on your actual workload rather than assuming the headline number transfers.

webhani's take

For clients running GPU or inference workloads on EKS, this is a low-effort change with a real payoff — it directly reduces the cost of autoscaling headroom, because you no longer need to keep as many pods artificially warm just to hide pull latency. We'd recommend piloting it on one node group, measuring pod-ready time before and after with your actual images, and rolling it out to the rest of the inference fleet once the numbers hold up.

The fact that the unpacking work is now part of containerd upstream is the more interesting long-term signal — expect parallel pull behavior to become a default expectation across container platforms, not an AWS-specific feature, over the next couple of release cycles. Worth tracking even if you're not on EKS today.

References: Introducing Seekable OCI Parallel Pull mode for Amazon EKS (AWS), Accelerating Container Startup with SOCI Snapshotter on Amazon EKS (AWS re:Post)

← Back to Blog