Featured in the Engineering section of eBay's Innovation Stories. Also posted as an article on LinkedIn.

Key Takeaways

• Platform services supporting diverse use cases from a multitude of business functions catering to members of a large e-commerce member base cannot use static warmup lists because their hot code paths are temporal and unpredictable, shifting based on which business functions are actively sending traffic at any given moment.
• Maintaining an LRU cache of recent production requests in each running instance creates a self-updating warmup dataset that automatically adapts to traffic patterns without manual curation.
• New instances can query service discovery to fetch cached payloads from healthy in-traffic instances before registering with the load balancer, ensuring they warm using production-realistic workloads.
• This approach requires services to be idempotent, as warmup payloads are replayed during instance startup - non-idempotent operations (payments, writes) must use synthetic payloads or dry-run modes instead.
• Temporal warmup eliminates the deployment reliability tax where new instances perform worse than warm instances, making rollouts invisible to downstream callers and preventing false-positive alerts from deployment monitoring systems.

Introduction

Deployment reliability is often treated as a solved problem - blue/green deployments, canary releases, and feature flags have become standard practice. But for platform services operating at scale, a subtler problem persists: even with sophisticated rollout strategies, each new instance goes through a cold-start period where it performs significantly worse than an already-warmed instance. Connection pools are empty, internal caches are cold, JIT compilation hasn't optimized hot paths, and Spring bean initialization has only just completed.

For domain services with focused, predictable workloads, the fix is straightforward: define a static warmup sequence, run it before registering with the load balancer, done. Platform services don't have this luxury. A platform serving dozens of teams processes workloads that vary dramatically by team, time of day, and business cycle. No static list captures that diversity.

The consequence was visible on every deployment: a sustained window of elevated tail latency and error rate increases that downstream teams noticed and escalated. The goal was simple - make rollouts invisible to callers.

Background: The Platform Service Challenge

The service in question is a critical component of a large-scale e-commerce platform's content delivery infrastructure, used by multiple internal teams to send communications to members. Unlike a focused domain service with a single code path, this platform service integrates with several backend services and databases to compile fresh, relevant content for end users. This architecture - serving multiple teams with different data sources and content generation logic - creates dozens of distinct code paths through the service. What gets "warm" depends entirely on which teams are actively sending traffic and which backend integrations they're invoking at that moment. This makes the warmup problem fundamentally different from domain services with predictable workloads, requiring a specialized approach.

The service itself is a template-based content generation engine built on Spring Boot. It accepts content generation requests, orchestrates data fetching from multiple internal services, executes Apache FreeMarker (FTL) scripts against that data, and returns generated HTML or JSON. Think of it as a mail merge engine at platform scale: content teams express generation logic in FTL templates, and the service executes those templates with real data to produce final content for web and mobile delivery.

Figure 1: End-to-end content generation pipeline. Content team authors an FTL template. The content generation service fetches data from internal services, executes the script, and produces generated output for delivery.

Because this service orchestrates calls to many internal systems and runs a scripting engine, its initialization cost is high and its warm-path behavior depends heavily on which templates and which downstream integrations are actively in use. This is precisely what makes static warmup insufficient - the warm paths are a function of current traffic, not a property of the service itself.

The Cost of Cold Starts

Before implementing temporal warmup, deployments exhibited predictable degradation patterns:

• P95 latency increase: 285ms baseline → 6s during rollout
• P99 latency increase: 350ms baseline → 9s during rollout
• Error rate spike: 0.4% baseline → 5% during cold-start window
• Duration of degradation: 3-4 minutes per instance until organic traffic warmed all major code paths, with symptoms persisting throughout multi-hour rollout windows

The root causes were well understood:

• Connection pool initialization: First requests to each downstream service paid TCP handshake + TLS negotiation costs
• Cache population: Internal caches for frequently accessed data started empty
• JIT compilation: HotSpot hadn't optimized hot paths yet
• Spring lazy initialization: Beans configured for lazy loading only initialized on first use

Approach: Temporal Warmup via Production Traffic Sampling

Our solution has three components that work together to ensure new instances are warmed with production-realistic payloads before accepting traffic.

1) Continuous Payload Caching

Every running instance maintains a bounded LRU cache of recent generate requests. Each cache entry stores both the template identifier and the complete request payload - enough information to reproduce the generate operation locally. The cache size is tuned to balance memory footprint against coverage breadth, and LRU eviction ensures the cache always reflects recent traffic rather than historical patterns.

This is the key insight: rather than maintaining a manually curated warmup list, each instance continuously self-updates its warmup dataset from actual traffic. The cache is always current, always representative, and requires no maintenance.

Figure 2: The LRU cache lives inside each running instance - every handled request is captured in-process, keeping the cache current with no external dependency.

Here's the core cache implementation:

@Component
public class WarmupPayloadCache {
    private static final int MAX_CACHE_SIZE = 100;

    private final Map<String, ContentRequest> cache =
        Collections.synchronizedMap(new LinkedHashMap<>(MAX_CACHE_SIZE, 0.75f, true) {
            @Override
            protected boolean removeEldestEntry(Map.Entry eldest) {
                return size() > MAX_CACHE_SIZE;
            }
        });

    public void captureRequest(String templateId, ContentRequest request) {
        cache.put(templateId + ":" + UUID.randomUUID(), request.deepCopy());
    }

    public List<ContentRequest> getCachedPayloads() {
        synchronized (cache) {
            return new ArrayList<>(cache.values());
        }
    }
}

The cache is populated inline with request processing:

@PostMapping("/generate")
public ContentResponse generate(@RequestBody ContentRequest request) {
    ContentResponse response = contentService.generate(request);
    if (response.isSuccess()) {
        warmupCache.captureRequest(request.getTemplateId(), request);
    }
    return response;
}

Cache sizing considerations: The cache is bounded at 100 entries based on empirical observation that 95% of production traffic concentrated across 60-80 distinct template/payload combinations during any 10-minute window. The LRU eviction policy ensures the cache reflects current patterns rather than historical data.

2) Service-Discovery-Based Warmup Seeding

When a new instance starts, before registering with the load balancer, it queries the service's CNAME to fetch warmup payloads. The CNAME resolves to the service's load balancer, which routes the request to any healthy in-traffic instance. Because only instances that have completed warmup register with the load balancer, the response is guaranteed to come from a warmed instance with a populated cache. The new instance executes those cached payloads locally, driving initialization of Spring beans, connection pool establishment, JIT compilation of hot paths, and population of internal caches - using the same workload that production is handling at that moment.

Figure 3: Warmup sequence - new instance queries service discovery, fetches payloads from in-traffic instance, executes locally, then registers with load balancer.

The warmup endpoint exposes cached payloads to new instances:

@GetMapping("/internal/warmup/payloads")
public WarmupPayloadResponse getWarmupPayloads() {
    return WarmupPayloadResponse.builder()
        .payloads(warmupCache.getCachedPayloads())
        .capturedAt(Instant.now())
        .build();
}

The startup warmup orchestrator runs before the instance registers with the load balancer:

@Component
public class StartupWarmupOrchestrator implements ApplicationListener<ContextRefreshedEvent> {

    @Override
    public void onApplicationEvent(ContextRefreshedEvent event) {
        try {
            WarmupPayloadResponse payloads = restTemplate.getForObject(
                "http://" + serviceCname + "/internal/warmup/payloads",
                WarmupPayloadResponse.class);

            payloads.forEach(request -> contentService.generate(request));
            logger.info("Warmup complete: {} payloads executed", payloads.size());
        } catch (Exception e) {
            logger.error("Warmup failed - proceeding with cold start", e);
        } finally {
            loadBalancerRegistration.register();
        }
    }
}

The critical design choice is ordering: instances register with the load balancer only after warmup completes. This guarantees that CNAME queries land on warmed instances (which either fetched payloads during their own startup or accumulated fresh ones from production traffic). Executing 100 cached payloads takes 45-60 seconds, compared to the previous 3-4 minute organic warmup period.

3) Self-Sustaining Bootstrap

A natural question: what happens during a large-scale rollout where many instances start simultaneously? If every new instance queries service discovery and lands on another new instance, the warmup payloads come from a cold cache.

The design handles this gracefully. When a new instance fetches payloads from a live node and executes them locally, those payloads are immediately written into its own cache. So even if another brand-new instance queries this instance for warmup, it can already serve a meaningful seed - the same production-sourced payloads it just warmed from. The warmup data propagates through the cluster from the moment the first instance fetches it, with no dependency on accumulating traffic over time. There is no single point of failure and no scenario where the system deadlocks waiting for a warm instance to exist.

Figure 4: Self-sustaining bootstrap - even in full cluster restart scenarios, instances accumulate cache entries from warmup traffic and converge to a fully warm state rapidly.

Implementation Considerations

Idempotency Requirement

This approach requires that the service being warmed is idempotent - executing the same request multiple times must produce no harmful side effects. For a content generation service that reads data and produces output, this holds trivially. For services that write state, process payments, or trigger external actions, replaying production payloads during warmup would cause those operations to execute twice, which is unacceptable.

Teams considering this pattern should verify idempotency explicitly before adopting it. If full idempotency cannot be guaranteed, alternatives include:

• Synthetic warmup payloads: Craft representative requests that don't trigger side effects
• Dry-run execution modes: Add a flag that bypasses writes while preserving read-path warming
• Selective warmup: Warm only the read-path components (cache population, connection pools) using a subset of operations

Cache Size Tuning

The cache size directly impacts memory footprint and warmup coverage. Too small, and you miss important traffic patterns. Too large, and you consume excessive memory or warm with stale patterns.

We analyzed 7 days of production traffic and found that 80% of requests concentrated across 45-60 distinct template/payload combinations during any 10-minute window. Setting cache size to 100 provided 95%+ coverage while consuming only ~15MB per instance. The bounded LRU approach ensures predictable memory usage on each instance, which is critical for capacity planning in large-scale deployments.

Failure Modes and Resilience

The warmup system must fail gracefully:

No healthy instances available: If the CNAME query fails (first deployment to new environment, full cluster outage), skip warmup and register with load balancer immediately. A cold instance is better than no instance.

Warmup endpoint failure: If fetching payloads fails (network partition, unhealthy source instance), log the error and proceed to registration. Don't block startup indefinitely.

Warmup execution failure: If individual payloads fail during warmup (data changed, downstream service unavailable), continue warming with remaining payloads. Partial warmup is better than none.

The implementation above demonstrates this defensive approach - warmup failures are logged but don't prevent the instance from entering service.

Security Considerations

The warmup endpoint (/internal/warmup/payloads) exposes production request payloads, which may contain sensitive data including PII. This requires strict security controls:

• Internal-only access: The endpoint must not be exposed to external networks. Use internal service mesh routing, firewall rules, or VPC-level isolation to ensure only instances within the same application pool can access it.
• Authentication and authorization: Implement token-based authentication that restricts access to instances of the same service. In our implementation, we use application-level tokens that verify the caller is part of the same service pool.
• Network-level isolation: Rely on network segmentation to prevent cross-service access to warmup endpoints.

For services handling highly sensitive data, consider additional measures such as payload scrubbing (removing PII while preserving the structure needed for warmup) or using synthetic payloads that approximate production patterns without containing actual user data.

Evaluation

The results were immediate and clear. Following deployment of the temporal warmup system:

Latency Impact

Before temporal warmup (cold start during rollout):

• P95 latency: 6s (vs 285ms baseline)
• P99 latency: 9s (vs 350ms baseline)

After temporal warmup:

• P95 latency: Remained at 285ms baseline throughout deployment window
• P99 latency: Remained at 350ms baseline throughout deployment window

Improvement: Eliminated 21x P95 latency spike and 26x P99 latency spike during rollouts.

Error Rate Impact

Before: Error rate spiked from 0.4% baseline to 5% during cold-start window. After: Error rate remained at 0.4% baseline throughout deployment window. Improvement: Eliminated 4.6 percentage point error rate spike, representing a 12.5x reduction in deployment-induced errors.

Deployment Velocity Impact

Automated rollback systems stopped triggering false positives. Deployment monitoring that previously flagged every rollout as anomalous now correctly identifies only genuine regressions. Prior to temporal warmup, 25% of automated rollouts were getting rolled back by automation due to false errors from cold-start degradation. This dropped to zero after implementation.

Deployment scheduling became flexible. Engineers stopped scheduling deployments around low-traffic windows to minimize blast radius. Prior to this implementation, deployments had to be scheduled during "quiet time" to avoid false-positive induced rollbacks. After temporal warmup, deployments occur throughout the day without coordination.

Operational Stability

The solution also proved robust to scenarios beyond planned rollouts:

• Infrastructure-triggered restarts: Automated patching and instance replacements benefited from the same warmup mechanism
• Auto-scaling events: New instances added during traffic spikes immediately warmed to handle load
• Full cluster restarts: The self-sustaining bootstrap ensured rapid convergence to fully warmed state even when all instances started simultaneously

Broader Applicability

While this service was unique in its specific architecture and deployment patterns, the temporal warmup pattern extends to any read-heavy, idempotent platform service - API gateways, GraphQL query layers, search and analytics services, data transformation pipelines. If your service's hot paths are temporal and unpredictable, production traffic sampling is almost certainly more effective than any static warmup list.

The technique is particularly valuable for services that:

• Serve multiple tenants or business functions with varying workload patterns
• Have unpredictable hot paths that shift based on time of day or business cycles
• Experience cold-start degradation during deployments
• Can guarantee idempotent request handling (or support dry-run modes)

Conclusion

The temporal warmup approach reframes a classic problem. Instead of trying to predict what a platform service will need to be warm for, we observe what it's currently doing and use that as the warmup dataset. The result is a self-maintaining warmup system that requires no manual curation and automatically adapts to changes in traffic patterns.

For teams experiencing deployment-induced latency spikes or error rate increases, the first question to ask is: does your warmup strategy reflect what your service is doing right now, or what someone predicted it would do when they wrote the configuration?

The code patterns shown here are straightforward to implement in any Spring Boot service, and the principles apply to other frameworks and languages. The key architectural requirements are:

• Idempotent request handling (or ability to execute dry-run mode)
• Service discovery (to locate healthy instances)
• Internal endpoint (to expose cached payloads)
• Startup hook (to execute warmup before load balancer registration)

If your platform service exhibits cold-start degradation during deployments, this pattern offers a practical, zero-maintenance solution that eliminates the deployment reliability tax.

About the Author

Abhijeet Apsunde is a Senior Software Engineer with expertise in distributed systems and platform reliability. He has worked on large-scale e-commerce infrastructure, focusing on deployment reliability, performance optimization, and multi-tenant platform services. Connect on LinkedIn.

Technologies and Tools Discussed

• Spring Boot: Application framework and dependency injection container
• Apache FreeMarker (FTL): Template scripting engine
• Java: Implementation language
• Service Discovery: DNS/CNAME-based instance lookup (adaptable to Consul, Eureka, etc.)
• Load Balancer Registration: Health check and traffic routing integration

Listed under Engineering eBay on innovation.ebayinc.com/stories, September 2026.

This article is based on real-world production experience building and operating platform services at a large-scale e-commerce company.