E-commerce platforms fail in predictable places. Catalogue pages that worked fine at ten thousand products become unusable at a million. Checkout flows that handled fifty concurrent orders start dropping transactions during flash sales. Search that returned acceptable results with keyword matching becomes a conversion problem as the catalogue grows. The architectural decisions that prevent these failure modes are well-understood — but they are rarely made at the beginning, when the system is small and everything works.
This article covers the architectural patterns for e-commerce platforms that need to operate at real scale: not necessarily Amazon-scale, but the scale of a growing brand with hundreds of thousands of products, tens of thousands of daily active users, and occasional traffic spikes from campaigns and promotions. The decisions here are based on platform work across multiple e-commerce clients.
Catalogue and Search Architecture
Product catalogue storage and product search should be separate systems with different optimisation targets. The authoritative product record lives in a relational database — PostgreSQL is the correct choice for most platforms: ACID transactions, mature tooling, strong support for JSON fields for variable product attributes. But relational databases are poor search engines: LIKE queries on a million-product catalogue with complex attribute filtering are slow and do not return relevance-ranked results.
Elasticsearch or OpenSearch should serve search queries, with the product record from the relational database treated as the source of truth and the search index treated as a read model. Synchronise the index using a background worker that consumes product update events. This decoupling means catalogue writes never block search queries, the search index can be rebuilt without downtime, and search performance scales independently from database write throughput.
Product variants (size, colour, material) are a canonical source of schema complexity. Storing variants as a separate table with a foreign key to the parent product is correct, but querying across parent-and-variant data for search and filtering requires careful query design. The search index should index both the parent attributes and all variant combinations, with each variant becoming a searchable document that links back to the parent product. This enables filtering by available sizes without expensive application-layer logic.
Catalogue Architecture Split
- PostgreSQL: authoritative product records, inventory, pricing
- Elasticsearch/OpenSearch: search index, faceted filtering, full-text
- CDN-cached HTML: generated product page content with long TTL
- Redis: real-time inventory availability, session cart data
Cart and Checkout
Shopping cart state is session-scoped and high-write — thousands of cart additions and modifications per minute during peak traffic. Storing cart state in the relational database with a row per cart item adds write load that competes with order inserts, the most critical write path in the system. Redis is the correct store for active cart data: fast, in-memory, with TTL-based expiry for abandoned carts. Persist cart contents to the relational database only when the user checks out or when they log in and merge an anonymous cart with their account.
Checkout is the highest-stakes path in the entire system. It involves price calculation (which must be authoritative, not cached), inventory deduction (which must be atomic), payment authorisation, and order creation — all of which must succeed together or none of which should succeed. This is a textbook case for a database transaction, and it is the one place in an e-commerce system where you should accept the write latency overhead of a serialisable transaction.
Race conditions during checkout cause inventory overselling — the scenario where two users both successfully purchase the last item. The correct prevention is a database-level locking strategy on inventory deduction, not application-level checks. Check inventory at the start of the checkout transaction, decrement it within the transaction, and handle the deduction failure case explicitly rather than assuming the check and the decrement will observe consistent state.
Order Management
Order records are immutable append-only facts — an order should never be modified after creation, only supplemented with status events. Model order state as a sequence of events: order_placed, payment_captured, fulfillment_started, shipped, delivered, return_requested, refunded. This event log is both the audit trail and the source of truth for order status. Application code computes current state by reading the event log, not by reading a status field on the order row.
Order notifications — confirmation emails, shipping alerts, return processing confirmation — should be triggered by order events rather than written inline in the checkout or status update code. A background job processes the event stream and triggers the appropriate notification. This decouples notification logic from order logic and allows retries without affecting order state. It also makes it easy to add new notification types (SMS, in-app) without touching the order processing code.
Fulfillment integration typically requires connecting to a warehouse management system or 3PL provider via their API or EDI. Design this integration as a separate service that consumes order events and manages its own state for fulfillment status. The order service should not contain fulfillment-provider-specific logic. When you change 3PL providers — which happens more often than you expect — the order service is unchanged.
Performance at Scale
Page caching for product listings is the single highest-leverage performance optimisation for content-heavy e-commerce. Generate HTML at page request time and cache it at the CDN with a TTL matched to how frequently the content changes — product listing pages can typically tolerate 5-minute cache TTLs, which eliminates database load for the most frequent reads. Dynamic elements (cart count, personalised recommendations) load asynchronously after the cached page is served.
Database connection pooling becomes critical at scale. Django's default database connection management creates a new connection per request, which works fine at low concurrency but collapses under thousands of concurrent requests. PgBouncer in transaction pooling mode maintains a pool of database connections and multiplexes them across application threads, allowing hundreds of concurrent application workers to share tens of database connections. Add this before you need it; adding it during a traffic incident is a bad time to discover the configuration nuances.
Denormalise read-heavy data deliberately. Product listing pages that repeatedly join across products, pricing, and availability tables in production workloads are better served by a materialised view or a scheduled denormalisation job that builds a flat representation for listing APIs. The normalised structure is correct for writes; the denormalised representation is correct for reads. Embrace the duplication when it is the right tool.
Operational Patterns
Deployment strategy for e-commerce platforms must account for database migrations. Django migrations that add new columns or indices can lock tables for seconds or minutes on large tables, causing checkout failures. Adopt a migration pattern that decouples schema changes from application changes: add the column as nullable in one deployment, backfill it in a background job, make it non-nullable in a subsequent deployment after the backfill is complete. This pattern eliminates migration-related downtime.
Price and inventory data are the two things e-commerce operators panic about when they are wrong. Add monitoring that alerts on inventory levels approaching zero for high-demand products, on pricing anomalies (a product whose price changes by more than a threshold in a short window), and on checkout error rates exceeding baseline. These alerts should go to operations staff who can investigate, not just to engineers.
Retain order data indefinitely. Financial regulations in most jurisdictions require multi-year order record retention. Archive old orders to cold storage after a defined retention window in the primary database, but ensure they remain queryable for customer service and compliance purposes. Deleting old orders to save database space is a compliance risk that is not worth the storage cost.
