TestinGil - Agile SW Testing Consultancy

08/09/2026

We still don't trust code agents completely (good!), but we know that every diff gets bigger. And we can't review everything. So what do we do?

Scan.

Yes, sometimes with models. That doesn't help with trust.

But, let's say we do trust them. the question is - do they find the problems.

According to Veracode's research, the answer is yes. But for syntactical stuff. 95% syntax correctness. Which is not really surprising, because most of the code out there runs. And that's what models have been trained on.

But get this: just 55% of the code passes security. 45% doesn't. Almost half.

And not a single run or model. On 150+ models and over two years. The security grading stayed flat.

You say "models will get better"? Well, they did over two years. The results didn't change.

So, maybe models will get better in five years. By then, how much AI generated code will be already in place with so many holes in it?

03/09/2026

Developers don't like writing tests. They see the value, but they don't like writing them.

So, when AI said: "I've got this", they said "you're my bro, bro!".

But bros got a problem. Do you think that devs who didn't like writing tests before, say "Bro gave me tests! I need to review them!"?

No. Bros trust bros.

We're trusting AI to write code, which we know isn't perfect, and to write tests we know aren't perfect, and we trust bro.

See the problem? Not the bro problem, the trust problem.
What to do about it?

I got you, bro!

My next webinar, 16-Sep, is about all kinds of risks that come with AI generated code. And what to do about them if you don't trust your bros.

Free. 3PM CEST / 9AM EDT.

Register here:
https://us02web.zoom.us/webinar/register/8517677800361/WN_2G3Dv53sSeaDaPjtwOXeRQ

02/09/2026

I'm going to repeat myself, because this is a repeating experience. Development is fast with AI. But then there's the backtracking.

It doesn't work exactly like I wanted. It understood differently. It implemented things I didn't need. And fixes break things that worked.

It's like working with a human, but without the intention. Humans make mistakes, but you know they tried. LLMs compare vectors. Hard to develop empathy for that.

And there's the big diffs to review - which I try to control, but that's extra micro-management. And those big blocks of text win. I can't review all of them, not the way I want anyway.

I've read that QA workload since teams started using AI has gone up roughly 58%. Who's picking this up? No new guys. Headcount stays the same. Oh, and 43% of AI changes still need debugging in production. After QA passed them. Which if you think about, makes sense. AI generates so much code, the weeding through it requires a machete.

Seems like management is starting to understand the cost of using AI, but the penny (or token) hasn't dropped about the risks.

We're playing with quality, as if there's no legal or economic price to pay. I think both are going to hit us from everywhere pretty soon.

Where's your organization on that scale of risk obliviousness to "what have we done"?

01/09/2026

Everyone tells you that if you don't use AI, AI will replace you. But what they don't tell you, is that if you're using it incorrectly, you're making it easier for AI to get your job.

We don't want that, do we? Good. That's what my keynote is about, at BrowserStack's AI x Testing Bootcamp & Hackathon on Thursday.

It looks like using AI in different aspects of our job speeds us up. Making plans. Generating automated tests.

But if we're not careful, we're delegating a lot of decisions, and that may not be a good idea. I'll show what to look out for.

Apart from my keynote, there's a whole hackathon you'll be able to join.

Free. 3-Sep, 10am CET.
https://www.browserstack.com/webinars/aixtesting-bootcamp-hackathon-eu-sep26?utm_source=&utm_medium=Influencer&utm_platform=&utm_content=digitalevent&utm_campaign=DevRel%20Event-03-Sep-2026-AI-X-Testing-boot-camp-EU&utm_campaigncode=701OW00000ySV6MYAW&utm_term=GilZilberfeld_Post

AI in Testing : Workshop + Hackathon Live 2-Hour AI Lab + Hackathon Become the Tester AI Can't Replace Unlock 4X productivity with AI across your test cycle, through a live workshop, followed by a hackathon. 3rd September 2026 10:00 AM CEST Event Starts in 00 Days : 00 Hours : 00 Mins : 00 Secs Limi...

27/08/2026

Last week I presented at TestMu on testing chatbots. One of the questions was about how to handle prompt injections.

I said that this requires a whole session. But there are key points we need to remember.

- Learn about prompt injections. Testing security is now a full part of our job.
- Define the never events - what should never happen. From unsolicited advice to accessing the internal network.
- Understand the surfaces where injections can come from - The UI, the APIs, files, databases.

When you got all three, that's a start of a plan.

And if you're thinking, thank god, I'm not testing AI features - well, you are testing code generated by AI. Maybe prompt injections are not a worry, but another security issue?

You bet.

Join me on Sep-16 for a free webinar, "Testing Code Nobody Reads: Handling the Risks of AI-Generated Code", to learn how to handle those guys.

Register ->
https://us02web.zoom.us/webinar/register/8517677800361/WN_2G3Dv53sSeaDaPjtwOXeRQ

26/08/2026

Why do APIs exist?

You can go all philosophical about it, but in the end, there was a request to access a resource. And the API was the solution to that request.

Testability is a feature too. Customers may not ask for it, but if testers ask for it (and they do), you should listen.

And it's more important than ever, since, you know, if AI generates the code - it doesn't listen to testers.

But we can make it listen. We got good reasons to.

Read the full blog:
https://testingil.com/2026/08/testability-of-ai-generated-code.html

25/08/2026

We're now 4 years into the AI revolution.

There are two sides here. In the left corner, we have automation. All the things we couldn't have done before. Or didn't have time for.

And you know what - we can do this for a lot less! Let's fire half our workforce!

And in the other corner - verification. We need to make sure that everything produced is working as expected.

And you know what - we need more people for a lot more verification for all that slop.

Ha. Not happening.

4 years, and we still don't understand the Trojan horse we got here. Sure looks nice, but it's not free, and if we don't prepare for those kind of gifts, well, you know how that ends.

Who picked up the verification work on your team? And what did they drop to do it?

20/08/2026

We all use AI now, and the productivity boost is nowhere more apparent than when you give your favorite code agent a spec, go out to lunch, and get back to a fully working app.

Fully working? Well, it needs to be tested. And the code reviewed. And may be refactored for reusing the rest of your codebase.

Whatever happened over lunch got you a full list of things to do that will make you sure that the app is indeed fully working.

Oh, what about the code you didn't ask for? What hides in it? A fully working app is not enough, it needs to do what you asked, not what you didn't.

AI generated code changes a lot of how we perceive quality and testing. It affects everyone who has a quality responsibility. Which is everyone.

In the next webinar, I'm going to talk about the risks involved in using code agents, and how to handle them, so nobody loses sleep over that fully functional app.

Testing Code Nobody Reads: Handling the Risks of AI-Generated Code.
September 16, 15:00 CEST. An hour, free.
Register →
https://us02web.zoom.us/webinar/register/8517677800361/WN_2G3Dv53sSeaDaPjtwOXeRQ

19/08/2026

You know what's cool about CI?

Lots of things. But think about this - you're in control. When you push a single file in, the whole automation pipeline is triggered.

A complex, multi-operational automation engine is at your disposal. It will build, deploy, run and test your app, and give you the feedback you crave.

All starting by pushing a file. Because all the changes you wanted, were in that file.

Unless of course, the changes are not there. If your AI agent changed behavior, for the same prompts - you wouldn't know until it's too late.

AI vs CI. New blog:
https://testingil.com/2026/08/ai-vs-ci-repeatability.html

18/08/2026

What makes prioritization hard?

We've always prioritized things - tasks, testing features, testing methods. The reason is that we don't have all the time in the world, so we need to drop things.

It's painful, I know, especially when they drop on your foot.

But it's still a decision. All you need is good information. In testing, we'd like to know what changed in the app since last time - new features, new bug fixes. New bugs.

The picture is never complete, but we have methods of filling it in - especially by the people who designed and wrote it.

So what happens, when we get more and more code written by agents? How much of the code is actually reviewed? Do we know if other features have been affected? What other side effects are waiting to jump us?

Prioritization is about making good decisions based on good data. Unfortunately, our picture is becoming murkier by the minute.

Because it's not a one time hit. Code changes compound. The changes are there, but since nobody reads all of them, we very quickly lose sight of the big picture.

And our decisions become bets. Not the informed kind.

What's the last call you made that was really a bet - and what would you have needed to know to make it a decision?