Bias

What bias means

Bias is a systematic distortion in data or models. The decisive part is the word systematic: an error is random, a distortion occurs consistently in the same direction. A recommendation engine that systematically shows women more expensive products is not making a one-off mistake, it is reproducing a pattern that came from the data.

Importantly, bias does not arise from intent. Usually nobody in the business believes one group should be treated worse. The pattern comes from data reflecting an earlier world and is then projected into the future.

How distortion arises

• Data selection: Training data comes from the past. If a shop sold mainly to one group for years, examples for other groups are simply missing.
• Proxy features: A feature stands in for something not measured. The postcode stands for purchasing power, the device for income. The distortion travels into the model without anyone intending it.
• Objective: A model optimises what is measured. If it optimises clicks rather than satisfaction, it produces attention rather than value.
• Feedback: What the model shows gets bought, and what gets bought confirms the model. Popular items become more popular, the rest disappears.

Where it shows up in a shop

Three places are typical. First, search ranking: a model learning from clicks favours items that were clicked often, so new items barely get a chance. Second, recommendations: the feedback loop visibly narrows the assortment until the shop feels interchangeable. Third, pricing: when dynamic prices react to features such as device or region, one group can permanently pay more without any objective justification.

A fourth area is text generation. A language model writing product copy carries over phrasing and stereotypes from its training data. That is especially delicate because it happens at volume and anew in every text.

What can be done

Complete neutrality is not achievable, because every model learns from an incomplete past. Four measures are realistic. First, measure: break results down by group and check whether the distribution differs. Without that measurement the problem stays invisible. Second, explain: for a recommendation, state why an item appears, because an explanation makes errors visible.

Third, mix: add a share of randomness or diversity to the recommendation so the feedback loop does not take over. Fourth, limit: drop features that serve as proxies for personal traits, even if the model becomes slightly less accurate in the short term. Giving up a feature usually costs less than the distortion costs later, once customers notice it and make it public.