TestinGil - Agile SW Testing Consultancy

17/06/2026

"Here's an API, test it".

Ok, it works. And let's say I really put in the work to see that it does.

What does it say about the system? The feature? The workflow?
All the things that are a lot more than one API call?

Most of the time - not a lot. Yes we have a passing test or even a few. But do I know how the workflow containing that API call works?

No. We need a lot more than singular API tests.
Here's the blog:
https://testingil.com/2026/06/test-the-feature-not-the-endpoint.html

16/06/2026

What's the difference between a unit test and an API test?
That's an easy one, right?

Here's a harder one. What's similar between a unit test and an API test?

No, it's not the test framework. And if you get this, you get a more confident test result.

Read the new blog:
https://testingil.com/wp-content/uploads/2026/06/au2c751.jpg

15/06/2026

What is it that makes AI features, or AI agents, testing different?
Even if you haven't gone in there yet, you have a feeling. Because we've all used chatbots.

And we know, we need to process answers in a way that make sense to the situation. Our context, our problems, what we know. What they don't. Our mind made the changes.

And in testing AI features we need to do the same.

So if you're planning to go into the murky waters of AI feature testing, or even if you're already knee deep in them, why not get cheat-sheet that can show you the steps?

I've built one. It's going through the new questions you want to ask, and what to change your approach.

Download:
https://testingil.com/ai-quality-testing-cheatsheet

14/06/2026

We all love tools. AI has granted the power to create some, so I went and create a mind reading tool.

Ok, I'm not taking over the world. Yet. But I've got an assumption identifier, and it works on tests. Yes, we put assumptions in them. Even if AI generates them.

So why not know in advance?

In my recent webinar I showed the suspects of flakiness - assumptions, usually implicit - that we make and cause the Monday morning blues. And I show my tool working on them.

Watch it.
https://youtu.be/RXvrOVv0kXo

Flaky tests. We've all got them. We rerun, they pass. We shrug and ...

11/06/2026

Do you know when you're making an assumption?
Because if you do, thinking through it can bring up interesting ideas.

Let's say we write a test that has State Pollution assumption in it - we're using a unique ID, that we expect not to be there, when we create an item.

If we're not saying to ourselves: "There is not going an item with that ID when the test runs" - Nothing will happen, until that ID shows up out of nowhere.

But if we're having the conversation and say "Well, we assume the ID won't be there. But what if does?", we can also ask:
- Will it fail our test?
- Do we need to guard against it?
- How can it show up uninvited?
- Can we make sure it doesn't show up uninvited?

Just by thinking about it, making the assumption explicit, we can start handling problem that haven't happened yet.

The "Fighting Flakiness" webinar recording is up. I walk through seven suspects (including State Pollution) — each one is an assumption we don't want to leave implicit.
https://youtu.be/RXvrOVv0kXo

Flaky tests. We've all got them. We rerun, they pass. We shrug and ...

10/06/2026

You just finished writing your 8535th test. Congrats!

When we write tests, we rely on our experience, other people's experience, and use the past as a guide to the future.

We know, from experience, this works most of the time. What experience will teaches us, is that we don't get the full picture every time. What we don't, we fill with assumptions. (there's a webinar for that, coming soon!)

And now, we don't need to write anymore. AI will do this for us!
More tests, more coverage, more... assumptions?

Yes, dude. The assumptions are not limited to your experience. No, you just inherited assumptions of so many people in the world. Whatever model you use, it was trained on a whole lot of code, and newsflash: people who had assumptions wrote it. And I'm not going into the inception angle now - generated code from generated code.

That means, that tests - our source of truth - need to be reviewed a lot more surgically. Not just if the tests do what they need to check, and check it correctly - what assumptions have stallaway-ed inside? What does it mean for future failures and log reading sessions?

AI amplifies everything - from productivity to complexity and flakiness?

Have you noticed that in your tests?

09/06/2026

We call an API. It returns 200.
Did it work?

I mean… it returned a success code. That's something. But I don't trust it yet.

So we check the response body. There's an ID. Ok, better. But did it actually save anything?

There more stages that can prove it works.
The thing is - we're not just calling an API. We're testing a feature.

And it takes more than a 200.

https://testingil.com/2024/07/api-testing-does-my-api-work-work.html

Where do you stop and say it's working?

08/06/2026

For years and years I've been telling people (especially my students, they paid for it) that tests are the real documentation of the system.

You can have all kinds of stale documents, collecting digital dusts. Sometimes real dust.

But if you have tests - running and passing tests, I need to be more specific here - that's current documentation.

That means they better be readable and covering a lot of cases. Because up until now, that was already a lot to ask for.

But with AI? Tests - as documentation - become even more important. AI generates more code than can be read, and fewer people over time know what the system actually does (The knowledge void).

So, in a couple of years, when someone will need to maintain the code, how well can they do it? With tests as good documentation, they have a better chance.

I know, AI pundits will tell you AI will have all the answers. But even if it did (I don't see that happening), it still needs good documentation. And I'd still want answers from a human that worked on the system for a couple of years.

So, are you treating your tests as documentation? Do you (a human) review all the tests? Makes sure names are good, and workflows explained? What else do you do to future proof your app?

07/06/2026

Your test passes. Every single time.
Then CI runs it alongside everything else, and it fails.

What happened to your mighty test? You see, when you ran it on your machine, you ran it alone. Nothing in parallel. Not other nosy tests poking data in the database, while your test does what it needs to.

We don't write tests to run in parallel. But the CI may run them this way. And then surprises arrive.

I made a video showing exactly this. How a test gets interrupted in just the right moment, and flakiness ensues.
https://youtu.be/esTjLsxDLXE

Your test checks if the app counts correctly. It adds an item andd ...

04/06/2026

We write tests one at a time. We focus on each one. We're proud of each one, like art we made ourselves.

But then, something weird happens. They run. And in the CI some run in parallel. Not exactly the way we envisioned them running.

Our brains don't do parallel. We tell ourselves we can multi task, but you know how that goes. Imagining how things run in CI - it's the same. We're just not good at imagining a race condition.

Until it happens.

I just published a video about this. I'm showing an example of a counting test that passes alone and fails in CI. Perfect on its own, but when it has company, it loses its shine.

https://youtu.be/esTjLsxDLXE
The Fighting Flakiness webinar was yesterday - but if you missed it, the recording is coming soon. Stay tuned.

Your test checks if the app counts correctly. It adds an item andd ...