Meera runs bookings for a mid-size Mumbai tour firm, and last month her inbox started closing suspicious emails on its own. Nobody had promised her anything grand — the IT vendor had simply switched on AIOps for Security tooling one Friday and gone quiet. So she did what a good operations lead does when a machine starts making calls: she gave it a week of her mornings and audited the thing herself. What follows is her week, reconstructed from her notes, because it doubles as a usable audit plan for any small team facing the same quiet switch-on. The technology she was probing — machine-learning scoring and LLM-assisted investigation lumped under one editorial label — arrived, as these things do, without a standard, a certification, or anyone obliged to prove it safe first.
Monday: What Decisions Is the Machine Actually Making?
The first morning produced the most important finding, and it cost nothing but questions. Meera asked the vendor one thing: which actions does the system take by itself, and which does it only suggest? The honest answer separated cleanly into rungs. Observe-only sits at the bottom — the system watches and scores, touches nothing. Next comes recommendation, which vendors may offer once a system has studied a meaningful pile of labelled alerts — roughly two hundred, of which thirty or more proved genuine. Above that sits automatic closing of cases, then reversible actions such as disabling an account, and finally irreversible ones — a rung nobody should ever grant to software, on any evidence. Her vendor had started at rung three, skipping the bottom two entirely, and had no observe-only mode to fall back on. In a bookings office the difference is not abstract: a recommend-only assistant flags a weird refund request for a human; an auto-closing one may bin it before anyone sees it. That single fact reframed her week. She was not evaluating a feature; she was deciding whether to keep a level of trust she had never consciously given.
Tuesday: Pull the Dispositions Before the Vendor Does
Tuesday was about evidence. She asked for an export of every disposition the assistant had recorded since switch-on — closed, escalated, or marked benign — before anyone could tidy it. The request felt bureaucratic until her consultant explained the maths: a claim as modest as “we miss at most two in a hundred” cannot be supported by a handful of cases. One observed error demands a sample north of two hundred clean verdicts; a second error pushes the required count past three hundred. Absent such volumes, absence of complaints proves nothing. Meera’s export contained a few hundred closures, which meant the vendor’s reassurance — “no false closures reported so far” — was statistically weightless. She also noticed the export took eleven days to arrive and needed two follow-up calls, which told her something about how ready the vendor was for scrutiny. She wrote in her notes: silence is not safety. The line went into the audit file next to a printout of the vendor’s own confidence statement, and the two documents told very different stories.
Wednesday: Splitting AIOps for Security Numbers by Category
Wednesday’s session covered the oldest trick in reporting: the blended average. A score pooled across all case types can look healthy while one category quietly fails — the documented example is an overall miss figure of 9.1% concealing a 42.3% miss rate on cloud-identity cases buried in the same denominator. So Meera split her export: phishing in one column, account anomalies in another, password-reset oddities in a third. Two of three columns looked fine. The third was thin and unconvincing, mostly because nobody had labelled enough of those cases to say anything — which itself became a finding. Labelling takes an hour a week, she estimated, and it is the only way the third column will ever mean anything. An audit that cannot split results by category is not finished. She booked the gap as an action item, roughly the way she would flag an unvisited neighbourhood when preparing a complete Mumbai sightseeing itinerary: the map is only done when every district has been walked.
Thursday: Canaries for the AIOps for Security Rollout
The vendor’s monthly quality report claimed a random spot-check of closed cases had found no errors. Meera’s Thursday research dismantled that reassurance. Sampling a twentieth of about 1,845 quarterly closures and finding zero mistakes still leaves the true miss rate bounded near 38% — a number indistinguishable from total ignorance. AIOps for Security programs that want real assurance plant the evidence instead: fifty seeded canary cases per quarter, each a known genuine threat, plus a rule that pulls the plug once three of them get closed. The contrast between sampled history and planted truth became the centrepiece of her audit summary. The instinct matches a hotel inspector who plants a booking under a false name instead of trusting the guest reviews: test the system with something you control. Her note read, in full: planted cases or no case.
Friday: Boundaries, Permissions, and the Honest Exit
The final morning covered what surrounds the model. The assistant read untrusted text — forwarded mail, attacker-crafted content — which makes it prompt-injection territory; the UK’s NCSC has been blunt that such systems are an inherently confusable deputy and the problem may never be fully mitigated, so deterministic guards and least privilege must carry the load. Meera confirmed the assistant held no outbound channel, no power to move beyond the triage queue, and a working one-click reopen for every closed case. She asked where the product stood on data residency and got a partial answer, which she recorded as a partial answer. Then she drafted the arrangement she would accept: observe-only for a quarter, seeded canaries, a monthly per-category review, and a signed register of every granted permission. The vendor pushed back, calling the conditions unusual. She kept the draft anyway and pinned it above her desk, next to the monsoon ferry timetable — both documents being, in her words, lists of things that look fine until they are not.
What the Week Left Behind
Meera’s audit ended without a dramatic confrontation, which is how most good audits end. The assistant stayed, demoted two rungs, and the firm’s security posture improved — not because the software changed, but because the terms did. Total spend for the week: five breakfasts, two phone calls, and one uncomfortable meeting. Her closing checklist, the part worth stealing, fits on an index card:
- Name every autonomous action the system takes today, and demote any that nobody consciously approved.
- Export dispositions first, label a blind sample second, and only then listen to accuracy claims.
- Split every metric by category; retire any blended number that cannot be split.
- Replace random spot-checks with planted canaries and a written withdrawal trigger.
Travellers do this instinctively before committing to an itinerary — verify the operator, read what other guests actually experienced, and keep a fallback, the way seasoned visitors do when consulting a Mumbai city guide covering heritage and street food before booking anything. Security deserves the same patient scepticism. The full picture of what this technology can and cannot promise is documented in the full AIOps for Security entry at the Ops Atlas — better Monday-morning reading than any vendor deck.
