
Designing a LiveOps alerting system to reduce production incidents
- Product Designer
- 3 months
- Product Manager, CTO, 2 BE Engineers, 1 FE Engineer
- Product Design, Product Strategy, Prototyping
22%fewer incidents
- Things broke in production, and the people who shipped them were the last to know.
- I put failures where creators already work, and gave every one of them an owner.
- 22% fewer production incidents. Failures now surface in minutes, not hours.
Live mobile games rely on continuous updates after release.
At Tactile, updates are managed through an internal LiveOps Dashboard, where teams configure events, experiments, player targeting, and feature rollouts without shipping new app versions.
Creator schedules a rewardHoliday bonus500 coins to all playersScheduleStatus: live
!Critical: reward not deliveredRewards service timed out · View issue →
Backend delivers it
Delivered
Player gets 500 coins
The problem
Failures were detected by backend systems, but not surfaced in the LiveOps Dashboard.
Goal
Surface backend failures to LiveOps creators early so they can act before players are affected.
Discovery
Critical signals lived outside the LiveOps Dashboard
Engineering tools
Monitoring!
Logs!
Slack!
Support tickets!
Player metrics!
LiveOps Dashboard
Hello, DianaNo alerts
Solution
Add password to keep reading.