A/B Testing & Experiments

A/B Testing & Experiments The two-day seminar for everyone who no longer wants to decide by gut feeling in the shop: you learn to turn an observation into a testable hypothesis, to set up a test properly and to read a result without being fooled by a nice percentage. By the end you know which change really brings more revenue.

A/B Testing & Experiments at a glance

• Format: 2 days on site + self-learning track
• Level: basic to advanced
• Participants: max. 16 people
• Language: German (English on request)
• Certificate: AEI certificate of attendance
• Investment: €290.00 plus VAT

Most shops either do not test at all or test too early. Whoever does not test decides on the basis of the loudest opinion in the room. Whoever tests too early wins tests that mean nothing and then changes things that nobody can justify afterwards.

The difference is rarely in the tool. It is in the question: what exactly should change, for whom, and how would you notice that it is not working? A test without those three answers is a poll among visitors, not an insight.

This seminar works exactly at that point. You formulate hypotheses that can be refuted, you fix sample size and duration before the start and you learn what a result means and what it does not. The test itself is the easy part afterwards.

What you take away

• A procedure for turning an observation into a testable hypothesis.
• A rule of thumb for sample size and duration that you fix before the start.
• A list of the mistakes that make a test worthless before it has begun.
• A routine for reading results without glossing over them.
• A format that makes testing a habit instead of a project.

Your learning objectives

• You tell apart situations where a test makes sense from those where it would waste time.
• You formulate a hypothesis with a clear, measurable expectation.
• You fix sample size and duration so that the result is reliable.
• You recognise significance and understand why it is not the same as relevance.
• You draw an insight from a lost test as well.

The levers at a glance

A test answers one question, not all of them. Whoever changes headline, price and button at the same time only learns afterwards that the package worked — or did not. One variable per test is not a convenience but the condition for learning anything.

The sample size is decided before the start. Stopping after two days because it looks good is the most common way of building yourself a result. Whoever fixes sample size and duration in advance protects themselves from themselves.

Significance is a probability, not a guarantee. A significant test says that a difference is probably not chance. It does not say that the difference is large enough to roll out.

Losers are the cheaper teachers. A test that changes nothing costs time; a test that prevents something saves money. Whoever documents only winners learns nothing about their audience.

Who is this seminar for?

For people who have to justify changes in the shop instead of asserting them.

• Conversion and e-commerce managers who want to run optimisation systematically.
• Marketing leads who have to defend budgets with results.
• UX and content teams who want to measure their work by impact.
• Shop owners who already run tests and cannot interpret the results.

Basic knowledge of web analytics helps but is not a requirement. Anyone who has heard the terms session and conversion will be faster; anyone who has not will keep up during the seminar.

The seminar is not suitable for teams looking for an introduction to a specific testing tool. We work on the question and the procedure — the tool changes, the way of thinking stays.

Investment and dates

Participation costs €290.00 plus VAT, including the self-learning track, materials and certificate of attendance. There are six dates per year in Berlin, Hamburg, Cologne, Munich and live online, plus an in-house variant for teams. You will find all dates in the list on this page.

For the exercises you work with test data and a sample shop, so you can take part without access to a live system. If you have running tests of your own, bring them along.

After the seminar

You keep the materials, a hypothesis form and a template for a test record containing question, expectation, duration and result. Those three sheets later replace any discussion about gut feeling.

Two weeks after the seminar you receive a short follow-up: write down a test you were planning anyway as a hypothesis and send us the wording. We give feedback before you spend traffic on it.

For teams the seminar is available as an in-house event with your own questions from the shop. The procedure stays the same, the examples come from you.

Programme: 2 days

Day 1 – A/B testing fundamentals

• What an A/B test can do
• When testing makes sense
• From observation to hypothesis
• One variable per test
• Sample size and duration
• Avoiding sources of error

Day 2 – Execution

• Understanding significance
• Reading results correctly
• Testing as a habit
• Learning from losers

Both days run from 09:00 to 16:15 with breaks in the morning, at midday and in the afternoon. Short self-checks sit between the blocks so you can test where you stand before moving on.

Each day has three parts: an introduction to the pattern, an exercise on your own case and a review in the group. At the end of day one you formulate one hypothesis that you test before day two.

The seminar includes a self-learning track. It contains the patterns from both days as short units with examples and one self-check each. You can work through it before the date to clarify terms, or afterwards to repeat what stuck and what did not.

Our approach

We work on your questions, not on a collection of best practices. At the start of each exercise you decide which observation bothers you and what you want to derive from it; at the end stands a test you can stand behind.

The exercises are short and repeat in variations. That is deliberate: a hypothesis is not formulated by listening but by sharpening it repeatedly on real cases.

We work in pairs, in small groups and in plenary, with changing partners. Every participant gets feedback on their question, not just on their test setup. Group level and participant count are deliberately limited: with 16 people no anonymous lecture is possible.