Dashboard
About ATLAS
ATLAS runs MTN's Zigi test cases for you. It takes a test case out of Azure DevOps, drives the real app — Android or web — the way a tester would, then judges what came back against the acceptance criteria written on that case and files the evidence. You read the result; you do not run the steps.
How ATLAS works
Six steps from a test case to a filed result. Click any step to see what happens, what it needs from you, and where to watch it in ATLAS.
Mission
Do the repetitive half of manual QA end to end: read the case, work the journey on the real app, capture what happened, decide pass or fail against the case's own acceptance criteria, and hand the result back to Azure DevOps with the evidence attached. A human still approves it.
Signing in
Each tester has their own account, so runs and evidence are theirs. Two roles: a Tester runs cases and sees their own runs; a Super Admin also invites people, sees every run and reads the audit trail on Activity. Every action is checked on the server, not just hidden in the page.
Azure DevOps integration
Connect once on ADO Test Cases, then browse plans → suites → cases, open a case to read its steps (shared steps included, with the expected-result screenshots), start it, and post the result back — pass/fail, the reason, a link to the run and the scorecard row. Only ATLAS's own clone cases can be written to; real MTN cases are read-only.
Channels
- Web — the SIT web app in a real browser. No phone, no Appium.
- Mobile — the MyMTN Android app on a real handset or an emulator, driven over Appium/ADB.
- Voice — speaking a turn in and transcribing the spoken reply. Not implemented yet — picking it stops the run rather than quietly typing the words and calling it a voice test.
How it drives the app
ATLAS is self-driving: it reads the case goal and the acceptance criteria, looks at the live screen, decides the next action, does it and looks again — so it copes when a layout moves or a card is slow, instead of failing the case over a changed button. It is not a choice you make per run; it is simply how a run works.
How a run is decided
The verdict is pass or fail. Captured facts settle it first — an error tile, a missing component, a stall — and only then the AI judge, which compares the actual screens and reply against the acceptance criteria on the ADO case (and against MTN's expected-result screenshot when the case carries one). Anything it cannot confirm is held for review; nothing passes by default. Because the same evidence can be judged slightly differently twice, a run asks the judge three times by default and keeps only what the passes agreed on — a criterion they split on is recorded as a fail, never a pass.
How scoring works
Alongside the verdict, each run scores the KPIs that this case actually asks for, drawn from the team's 39-column scorecard. Those numbers come from ATLAS's own built-in deterministic scoring engine — fixed formulas over response times, on-screen text and error states. They measure how the machinery behaved, not whether the answer was right: a failed verdict forces the overall score to zero. A run also lists the defects a tester would have written down, and finishes with a self-check that reconciles the verdict, the criteria, the KPIs and the defect list against each other — and says so on the page when they disagree.
Evidence you can check
Every run keeps screenshots, a before/after frame for each action the agent took, a video of the whole session, the conversation and the audit log. You can watch the phone or browser live while it runs, re-run a case from its own page, see the history of every previous run of that case, and copy a link that opens straight back to this run.
What ATLAS will never do
ATLAS never completes a payment. A deposit, adding money, or entering a PIN or card is refused outright — there is no setting that turns it on. Pressing Pay, Checkout, Confirm or Subscribe is never done by the self-driving agent. Entering a purchase flow (Top-up, Recharge, Buy) is refused by default and only opens if a person switches on the audited purchase override, which is how MTN's own purchase case can be run as far as the Confirm-purchase screen and no further. Every refusal is written to the run's audit log.
How much of the pipeline is AI
The agent understands, drives and judges the test. Evidence capture is automated, and the safety boundary stays deterministic on purpose, so money actions can never be automated away.
How a test runs
Set the three things below once. After that, start a case from here or from ADO Test Cases — ATLAS does the rest.
Choose the test case
Pick one below, or browse ADO Test Cases and open a case from Azure DevOps. The case decides what is typed and what "correct" means.
Give it a test number
The MTN line the run signs in with. ATLAS fetches the one-time code itself, so nothing is needed from you while it runs.
Connect a phone
Only for mobile app cases. Plug in over USB or pair over WiFi; the checklist finds it.
Press Start Test
Watch it live on the Test Runs page: every message, screenshots, a video, and a pass or fail with the reason.
The checklist on the right turns green when ATLAS has what it needs. Anything still red tells you exactly what to do.
Your settings
The four things ATLAS needs from you. Everything else is set for the whole installation and looked after for you.
Advanced — rarely needed
Connect a phone
More help
- The pop-up must stay open the whole time. If it closes, the code stops working.
- The 6 numbers and the address change every time you open the pop-up. Always copy them fresh.
- Wireless debugging needs Android 11 or newer. On an older phone, use the USB cable instead.
Readiness — auto-checked
Results — every test case this framework has recorded
Each finished run appends one row to the scorecard, in the same 39-column format the team already uses. Pick a date range to filter what you see below — the download gives you exactly those rows, plus a Summary sheet and an Evidence sheet pointing at the screenshots, video and transcript behind every one of them.
Quality trends — is Zigi getting better?
Direction of travel across the runs in the date range above. A KPI needs at least two runs of the same case before it can show a direction — one result is never called a trend. Judged KPIs are labelled, because a small move can be scoring variation rather than a real change.
Test case reference — what each case actually sends
The exact messages sent to Zigi, and what each one is checking. Read this before running a case, or when justifying a score.
Runnable cases
Every case ATLAS can run right now, with what it will judge against. Re-fetch pulls the case again from Azure DevOps, shared steps and all.Azure DevOps — connection status
Where the test plans below are being read from. Nothing here is writable except Sync all, which re-imports every case already imported so its steps, criteria and screenshots match ADO today, and imports every new eligible case across all suites in the plan (surface inferred — confirm it before the first run).
Test plans → suites → cases
Loading…
Test cases
Loading…
Run history for this case
Every ATLAS run of this case, newest first — when it ran and what it scored, so a result can be read as stable or a one-off.
Add a test case — describe it, AI drafts the steps, you save & run
Pick whichever form you already have the case in. All five end the same way: the messages land in the one editor below, you review and edit them, then Save. The new case is runnable immediately — no restart, no code.
Review & save — this is what will run
Messages sent to Zigi, in order
Test cases you have added
Authored cases are stored as data and read fresh on every run, so they work straight away. Built-in cases are not listed here and cannot be deleted.
Your team
Invite testers, see who is on the team, and manage access. Only Super Admins can open this — testers never see it, and every action here is enforced on the server, not just hidden.
Invite someone
There is no mail server, so the activation link appears here after you invite — copy it and send it to the person yourself.
Pending invitations
Team members
Recent activity
The audit trail — every sign-in, invitation and run start, newest first. This is the record of who on the team is doing what.
What the AI costs us
Every model call ATLAS makes, counted per execution. The gateway reports the tokens it billed; where it does not, the figure is our estimate and is labelled as one. Money is an estimate either way — the rate below is a list price until MTN confirms what the approved gateway charges.
Server doctor — internal · admin only
End-to-end readiness of THIS server — system tools, the Web (Chromium) and Android tiers, network egress and run config. Every failing check carries the exact fix. It only reports; it never changes the box.