Angles and split tests
Each step of a sequence has a few versions, called angles. Kata tests them on real sends and moves the traffic to the best one.
What counts as a success
A sent email is a success when the lead clicks the tracked link, replies with interest, or converts. The rate of an angle is successes divided by emails sent.
How each send picks an angle
Kata uses Thompson sampling. For each email, it draws a plausible rate for every live angle of that step, from its results so far, and uses the highest. New angles get a fair chance; clear winners get most sends.
campaign_report and list_variants show p_best for each live angle: its chance of being the best of its step.
The evolve job
Once an hour, for each active campaign with its autopilot on, and each step:
- 01
Wait for enough data.
An angle is judged only after a minimum number of sends (40 in the current setup).
- 02
Retire the losers.
A judged angle that is not the best, with less than a 5% chance of being the best, is retired. The last live angle is never retired.
- 03
Write a new one.
When there is room under the maximum of live angles (3 in the current setup), the AI writes a new angle from the best one and the results of the others.
Run it now with evolve. Every decision is logged as an event.
This is the campaign's autopilot, on by default. Turn it off with the switch on the Autopilot block of the Canvas, or update_campaign with autopilot: false: the angles in test then stay as they are and share the sends equally.
Writing and editing angles
generate_angles: the AI writes 1 to 6 angles for one step.add_variant: add one you wrote.set_variant: retire one, put it back, or edit it. Editing an angle that already sent mixes its stats: add a new one and retire the old one instead.
Testing whole campaigns
Connect one lead list to two campaigns and Kata splits its people between them, each person to one campaign. The planner gives each campaign its turn, so both arms get the same share of the day's capacity. See Lead lists.