A/B testing is an engineering system
An experiment is more than two page designs and a conversion chart. The website must assign a visitor to a variant, deliver it consistently, record exposure, track the outcome and avoid harming performance or search visibility.
React and headless websites give teams flexibility, but client-side rendering, cached pages and distributed services make casual implementations unreliable.
Start with the decision, not the tool
Write down the decision the experiment will inform. A useful plan includes:
The user problem and evidence behind it
The exact change between control and variant
The primary success metric
Guardrail metrics such as errors, performance or revenue quality
The audience and exclusions
The minimum runtime or sample plan agreed with the analyst
What the team will do for each possible result
Avoid running several unrelated changes in one variant unless the question is whether the complete experience performs better. Otherwise, a result will not tell you which change mattered.
Choose where variant assignment happens
Client-side assignment
A browser script selects and renders the variant. It is easy to start with, but it can cause flicker, delay interaction and add JavaScript. It also needs careful handling during React hydration and route changes.
Server-side or edge assignment
The server or edge layer selects a variant before HTML is returned. This can provide a consistent first render and reduce flicker. The assignment must still persist across navigation and reach analytics.
Application-level assignment
The product decides the variant using its feature-flag or experimentation system. This is often appropriate for logged-in products and deeper workflow changes because the experiment can follow the user across devices and services.
Choose based on the experiment, architecture and data requirements. Do not force every test through the same delivery mechanism.
Prevent flicker and layout instability
If the control renders first and is replaced later, visitors may see both experiences. That can distort results and increase Cumulative Layout Shift.
Assign the variant before rendering where practical
Keep control and variant containers dimensionally stable
Avoid hiding the entire page while an experiment script loads
Load experiment code only on eligible routes
Set a timeout and fail safely to the control
Measure performance separately for each variant
Record exposure separately from assignment
A visitor can be assigned to a test without seeing the changed component. Record an exposure event when the relevant experience is actually rendered. Include experiment ID, variant ID and a stable anonymous or account identifier allowed by the measurement design.
This prevents unrelated visitors from diluting the analysis and makes debugging possible.
Track outcomes from application events
Conversions should come from the same dependable events used by the rest of the analytics setup. Do not create a second conversion definition inside the testing tool.
Track successful outcomes, not attempted clicks
Use stable transaction or lead identifiers
Keep revenue and currency formats consistent
Preserve experiment context through multi-step journeys
Handle cross-domain checkout and authentication
Test consent behavior for exposure and outcome events
Handle React navigation and hydration
Do not reassign a visitor on every route change
Make assignment available during the first render when possible
Ensure the server and browser render the same variant
Record one exposure per intended experience
Clean up subscriptions and observers when components unmount
Test browser back, forward and cached navigation
Check whether streaming boundaries reveal control content prematurely
A hydration warning is not just a developer-console problem. It can mean the server delivered one experience and the browser replaced it with another.
Use the CMS without making content editors own experiment logic
A headless CMS can store approved variant content, but assignment, exposure and analysis should remain in the application or experimentation layer. Editors should see clear fields for control and variant, scheduling and status without handling IDs or scripts.
Preview both variants and verify that unpublished CMS changes cannot leak into the live experiment.
Protect website performance
Measure added JavaScript and network requests
Load testing vendors only where an experiment is active
Keep third-party scripts out of the critical rendering path
Track LCP, INP and CLS by variant
Remove completed experiment code
Do not leave several paused vendors installed
A conversion uplift that comes with a slower experience may not survive when rolled out broadly. Performance should be a guardrail, not a cleanup task after the test.
QA the complete experiment
Control and variant render correctly on supported devices
Assignment persists across refreshes and routes
No flicker or unexpected layout shift appears
Exposure fires once at the correct moment
Conversions include the right experiment context
Consent choices are respected
Analytics and platform counts are explainable
Errors and performance are segmented by variant
Search crawlers and canonical pages receive the intended experience
Avoid common interpretation mistakes
Do not stop an experiment because an early chart looks promising. Agree on the analysis plan before launch, account for repeated checking and include complete business cycles. Novelty, campaign changes, outages and audience shifts can all affect the result.
A neutral result is useful when it rules out a change or reveals that the expected effect was too small. Record the result and remove the experiment cleanly.
A maintainable experimentation workflow
Define the customer problem and decision.
Review the hypothesis with product, design, engineering and analytics.
Choose assignment and delivery architecture.
Implement exposure, outcomes and guardrails.
QA every journey and consent state.
Run according to the agreed analysis plan.
Decide, document and remove temporary code.
Turn a winning variant into normal product code.
When to bring in growth engineers
Growth engineers are useful when the experiment touches routing, rendering, analytics, consent, CMS content, checkout or website performance. They connect the marketing question to an implementation that can be trusted.
Makers' Den helps teams build this foundation through our Web Performance & Growth Engineering service . Start with our tracking audit checklist if the current conversion data is not yet dependable.