TestinGil - Agile SW Testing Consultancy

27/10/2026

I've been getting into software factories and how they automate the SDLC. We're not at full automation yet, don't worry.

We used to talk about maintenance as the big cost of change. But now that building (and rebuilding) is cheap, the cost is moving. To debugging, and to product knowledge.

As factories go, they are about production. But as we know, however mighty we build them, things still break down.

What do you do then?

First you need to notice things went wrong. Tests, reviews, process checks. Whatever the genie can't catch, a human should.

Then comes the fix. Actually, there's debugging first. Fixing is easy once you know what to fix.

But debugging is becoming more complex. I mentioned before how we're losing skills, and debugging is one of them. On one hand, we got better probes. AI can send its tentacles to different logs, machines and memories and try to identify what happened.

Wait. No, it finds patterns. Maybe those apply to that weird timeout. Maybe not.

The wrong conclusion leads to a perfectly bad rebuild. Not to mention the token cost.

And it's only getting harder. Debugging complex systems is hard enough. Now we're tasked with ever growing complexity (thanks to our AI generators) and less product knowledge (thanks to us, delegating to AI).

Building is cheap. Knowing why it broke isn't.

When your factory breaks, who on your team still knows why it was built that way?

20/10/2026

Before Zoom was a video conference software, it was an actual word.

We'd use it in sentences, like zoom in and zoom out. Or get the zoom out of my office.

Zooming in is great. It allows us to focus, and leave all the noise outside. Zooming out allows us to see the whole picture, without the details.

We need both. I'll show you why.

Your developers are using AI, and development speeds up. That's what you see when you zoom in.

The testers are fighting a lot more bugs than before. One vendor report saw more than 50% more bugs per developer. That's what you see when you zoom in.

Zoom out and you'll see - they are CONNECTED.

A zoomed-in solution could be throwing more testers in. A zoomed-out one is lowering the bug count.

Bug count is easy. Everyone can count bugs. And the numbers just wait for us to connect the dots.

When did you last put your velocity and your bug count on the same slide?

15/10/2026

How is a code review like candy?

As you can see in the image, it has to do with Lucille Ball.

In one of the funniest scenes in I Love Lucy, she and Ethel are instructed to wrap the candy as it comes out of the machine on a conveyor belt. (The whole bit is in the first comment).

Once the candy is wrapped, it has to go back on the conveyor belt for testing. I mean, packing.

It starts off slow, and sustainable. But then the conveyor speeds up. They didn't have AI in Lucy's day, but we have it today.

And the machine keeps producing more chocolate, and Lucy tries to keep up with the reviews, I mean wrapping, but it's just no use.

As Lucy says to Ethel: "We're fighting a losing game." So they hide the chocolates. In their mouths, their hats, their blouses.

I mean, they skip reviews.

It looks like that with code reviews too. And they are not as tasty as the candy.

Then the supervisor sees an empty belt and yells: "Speed it up!"

Don't be the supervisor. Fix the line.

How? That's my next webinar, for engineering managers, directors and VPs: AI Code Review Bottleneck: Your Team's Opportunity.

Wednesday, November 11. 3PM CET / 9AM EST. Free, and everyone who registers gets the recording. Link in the comments.

Not a manager? Send it to yours.

13/10/2026

It's not easy being green.

"Green" in the build dashboard means "everything is ok". But what is everything, really?

Ask the developers, and they will say "everything is ok". But that means that all tests are passing. Which is not a bad start, if you trust your tests.

But you know the truth - everything is NOT ok. Your other dashboard says so. The one with the customer complaints.

And it's not just you. One vendor tracked 22,000 developers over two years. Where AI use was high, incidents per pull request more than tripled.

The build dashboard is not really lying. We just need to use another interpretation. The developer green is "what we built works", according to some definition of "works". Not the customers' one.

With code agents, it's even easier to fall into this trap. The agent writes the code, then writes the tests from the code it just wrote. The tests check what it built, not what you needed.

Of course they pass.

The misalignment between "what we think works" and "what actually works" gets bigger in giant AI steps.

Kermit had it easy. At least in the old days everyone agreed what green was.

So, here's one for you - does your dashboard tell you what you need to know? Or just what the developers see?

08/10/2026

In one of the assignments in "Hell's Kitchen", Gordon Ramsay explains exactly how to make a dish. He shows the contestants everything he does.

Then they go and do it by themselves.
And of course, they get it wrong every time.

How? Turns out just knowing the words is not enough to make a perfect dish.

And it's the same with using coding agents. You can explain. But they make the dish.

New blog - "Knowing the Words Isn't Knowing the Language":
https://testingil.com/2026/10/is-ai-making-code-worse.html

07/10/2026

Imagine your company just got everyone a free gym membership.

Then a year later, management asks: Who's fitter?

Oh, and nothing in between. No surveys, or tracking of who's going. And what they do there.

Doesn't make sense, right?

Ok, let's replace the gym membership with a Cursor license. Everyone gets one.

Then, a year later, management asks - was it worth it?

I talked to someone last week whose company is investing heavily in AI. I asked how they measure it. They don't, and they are not sure what to measure either.

The company is sure just having the tool will guarantee success, even if they can't even define success. Or what failure looks like.

Then token maxing happens - throw more money, expecting more miracles.

Tool implementation, and yes any AI agent or subscription is a tool, needs some kind of ROI vision. Some metrics and measurement tracking. A method that will say - we're on the right track, or we've taken a wrong turn and need to correct.

In a gym, you get a plan. A trainer. You can weigh and see progress.

Instead, people stumble around, each finding their own way and use, which may or may not help the org.

Software engineering is still engineering. It's a process we track and improve. And we know what success looks like.

I thought we'd outgrown this by now.

How does your company know if its AI licenses are paying off?

06/10/2026

One of the cool things about the AI, is it reverses the polarity.

I always wanted to use that phrase.

What I mean, is that instead of the user needing to adapt their ways to use tools, we can easily build tools that help our way of working.

Take task management for example. We can pick millions of tools for task and project management. But then we need to adapt our way of working to them.

For a while, my go-to was a simple notepad list (because I'm a Luddite. And I could move lines up and down to change priority).

But with every other tool, I needed to fill forms, or tag stuff that wasn't important.

So I built a tool to help me. One that fits the way I like to work. In fact, I've built a whole system. Customized for me. And the basis for all my business information.

And without AI it wouldn't have happened.

The first thing it had to do? Let me move lines up and down.

On Thursday, Oct-8, 11am (Israel time) I'm doing a talk (in Hebrew) about how I engineered my way of working (not just tasks), gradually over time, and created a system that doesn't forget, nags at times, and helps me run my business.

Link below. And if you want to see it in English, let me know.
https://luma.com/ai-alter-ego-gil-zilberfeld-2026

01/10/2026

"You are responsible for our 2.5 hour shutdown in production".

They removed the blindfold.

I was in a dark room. Couldn't see a lot. Then again, I left my glasses on my desk.

"It wasn't me", I said.

"You can't run away from this one. Your name is all over this."

Couldn't run anyway, not in a good shape. And the last coffee was five hours ago.

"Can't be, I've got the tests to prove it".

"You mean those tests?" they asked, pointing at a block of text (with some nice colors, I like dark mode).

"Yup. They pass, Claude wrote the code, the tests, ran them, and promised everything will be ok".

The room went silent. I guess they went for coffee.

"Do you know what the code really does?" They asked.

"Hmm. Sort of". I had to admit, not all the generated code fits inside my head.

"We'll need to take your Claude privileges, until you start reading all this code again."

"Well, I guess we'll meet again here sooner than you think".

They let me go. But I know Claude will sabotage me again. And I'm willing to live with it.

Are you?
https://youtu.be/IAZj1CwKBI0

Ever been handed a flaky test to fix, and the flakiness turned out ...

30/09/2026

Have you ever looked at a legal form, 28 pages long, awaiting your signature in the end? I have.

All you have to do is read everything, find every minute detail that needs fixing, and then you can sign it.

Legal stuff, I'm sure you'll do your best to identify all the small bugs before you sign.

Now imagine having to read things like that ten times a day, but in code font, with as much risk.

And in it there may be some "oopsies", "I apologize" and other confessions. Because our code agents love to tell a story.

New blog - "The Test Was Wrong. Rewriting."
https://testingil.com/2026/09/ai-coding-agent-says-the-test-was-wrong.html

29/09/2026

I love coffee.

I'm one of those weirdos who follow a ritual for making it. (Moka and cold brew, these days).

I have a scale, and I'm using it, and I look at the coffee coming out of the pot, then turn the heat down to control the flow.

Weird, but works. And it works repeatedly, even if there are all kinds of uncontrolled variables. Temperature, beans, grind size, time of day.

And it works because the process has a controller (me). I set the grind size (as much as my grinder lets me), I control the gas flame (as much as my hob lets me) and other stuff. And as long as I pay attention - I get great coffee.

The attention part is crucial here. A controller without attention is like a broken pencil - pointless. (If you know the reference, put a meme in the comments.)

With code agents we abdicate our roles as controllers. We tell ourselves that it's ok, we'll see the end result and tweak then. But then the tests pass, and it looks mostly ok. And we move on to the next task.

I do it too. I gloss over the tests and code, sometimes not even that. We hope it's going to be ok.

With the moka pot, I turn the flame down the moment it spurts. With an agent, I wait until the pot is empty.

In fact we're losing two things - control over the result, and our ability to shorten the feedback loop. If the masses of code are in control, we have no chance of stopping in the middle.

And it does seem like - so what? These are not days or weeks lost. It's hours. Worst case - we'll regenerate.

That's how slop comes out. Like coffee out of an unwatched moka pot.

When did you last stop an agent in the middle, instead of waiting for the end result?