Anyone using tools like ndepend or others to help guide agents in refactors?
Personally I have a some tools that build dependency graphs (C# and Python) and store the results in a local database. Agents seem quite good at poking at this and coming up with refactor ideas. Graph analysis tools are useful here, simple application will detect cyclical dependencies, but I encourage the agents to use more complex tools like clustering to poke at the data.
I've been feeding agents dependency graphs plus CC and coverage data from a local store, and it works well for spotting cyclical deps and high-CC hotspots
From a security perspective cc is highly relevant. I use it to get a solid rating of the security aspects of Python code. I use [1] which is solid and proven.
Worst things happen always when 2 or more systems are combined because each system might be simple on its own, yet a combination is always much more complex.
It’s not obvious to me because cyclomatic complexity is not a straightforward proxy for the number of systems that are being combined.
It’s also the case that some of the most common sources of vulnerabilities, such as SQL injection, introduce no additional cyclomatic complexity. Heck, buffer overflows are good for your cyclomatic complexity - those array bounds checks are all extra branches.
Buffer overflow checks are really only going to be a linear growth in CC. It's when things move towards exponential growth or higher that it gets really easy to introduce flaws of many kinds.
Now, it's probably not a direct correlation. I'd think security bugs are more likely from programmers that unintentionally raise CC without really realizing it. Aka, overreaching their own knowledge when simpler structures are avaliable.
Sure. It’s just that there’s also so much research that has found that cyclomatic complexity is theoretically ill-founded, and that it tends to underperform other ways of measuring complexity. Most notably, just counting lines of code. (Not per function, in total.)
For something the the prior statement it is never a weird question to ask of there actually evidence of this or just it seems like it should be true so we believe it.
There are tons of things that seem like they would obviously be true, but it turns out they aren't.
I'd argue that assuming something is obvious without any empirical validation is the root of a huge number of misconceptions that humanity has historically had. There's a reason science suddenly started moving a lot faster after we moved past Aristotle and started measuring things in experiments.
Overall cyclomatic complexity is a useful metric, but it does have one shortcoming when used with modern languages: it was invented before polymorphism really became a thing.
That means that it really only counts explicit branching. So, for example, in an OO language like C#, calling a virtual method doesn’t increment cyclomatic complexity even though the method invocation could go down many code paths. Potentially thousands if you’re dealing with a common interface like IEnumerable. If you’re working on a library then the number of potential code paths in this kind of situation is unbounded.
As an aside, it’s interesting to think how it might apply to a language like Smalltalk that doesn’t even have if or switch statements.
OO isn’t the only monkey wrench, either. Higher-order functions also introduce forms of branching that cyclomatic complexity doesn’t measure.
Again that doesn’t make it a useless metric. Just don’t think that a cyclomatic complexity limit in your codebase is some sort of maintainability panacea. Some of the least comprehensible functions I’ve deciphered had quite low cyclomatic complexities.
In reading the article, it did feel somewhat wrong in a way that I couldn't quite describe. But after reading this comment, maybe it just seems old and maybe obsolete.
Taking the example provided in the article, I don't feel like the new code is meaningfully less complex. In fact, since it added some additional indirection, I could argue it's slightly more complex.
The core code with the nested if statements is something that I would probably refactor in some other way entirely. Maybe by taking advantage of other language features. It is a toy example so it's hard to say but that's part that feels like it needs simplification and untouched in this example.
Agreed. I also don’t really agree that the refactored code is any easier to test. ProcessOrder’s behavior hasn’t changed, only its implementation details, so the minimal test surface is the same for both: just test ProcessOrder.
You could additionally test the three helper functions. But the original tests against ProcessOrder would still be needed for completeness, so they wouldn’t necessarily add much except in an Uncle Bob style, “He who dies with the largest burden of gratuitous micro-tests wins,” sort of way.
Now if I really wanted to make that code easier to test, I’d instead be looking into ways to make the whole thing less stateful. Temporal coupling is much more confusing than if statements.
Some of the least comprehensible functions I’ve deciphered had quite low cyclomatic complexities.
In agreement with your post, this research that measures cognitive load via EEG and time spent shows that the metrics we use for complexity and readability are only partial matches to what is going on:
https://pmc.ncbi.nlm.nih.gov/articles/PMC9942489/
Fun story: at my previous aaaawful company CC was discovered as a thing to care about at about the same time as PMs and managers were encouraged to land code changes using the _then_ quite terrible AI tooling (this was a year or two ago). Cue an avalanche of completely unreviewable diffs.
Anyone using tools like ndepend or others to help guide agents in refactors?
Personally I have a some tools that build dependency graphs (C# and Python) and store the results in a local database. Agents seem quite good at poking at this and coming up with refactor ideas. Graph analysis tools are useful here, simple application will detect cyclical dependencies, but I encourage the agents to use more complex tools like clustering to poke at the data.
I've been feeding agents dependency graphs plus CC and coverage data from a local store, and it works well for spotting cyclical deps and high-CC hotspots
From a security perspective cc is highly relevant. I use it to get a solid rating of the security aspects of Python code. I use [1] which is solid and proven.
[1] https://nocomplexity.com/documents/codeaudit/complexitycheck...
Is there research that show if and how much a low complexity improves security?
Weird question to ask, that is pretty obvious.
Worst things happen always when 2 or more systems are combined because each system might be simple on its own, yet a combination is always much more complex.
It’s not obvious to me because cyclomatic complexity is not a straightforward proxy for the number of systems that are being combined.
It’s also the case that some of the most common sources of vulnerabilities, such as SQL injection, introduce no additional cyclomatic complexity. Heck, buffer overflows are good for your cyclomatic complexity - those array bounds checks are all extra branches.
Buffer overflow checks are really only going to be a linear growth in CC. It's when things move towards exponential growth or higher that it gets really easy to introduce flaws of many kinds.
Now, it's probably not a direct correlation. I'd think security bugs are more likely from programmers that unintentionally raise CC without really realizing it. Aka, overreaching their own knowledge when simpler structures are avaliable.
Sure. It’s just that there’s also so much research that has found that cyclomatic complexity is theoretically ill-founded, and that it tends to underperform other ways of measuring complexity. Most notably, just counting lines of code. (Not per function, in total.)
Here’s an oldie but goodie: https://cs.du.edu/~snarayan/sada/teaching/COMP3705/lecture/p...
I’ve personally had better success thinking of it as more of a measure of readability than of quality.
For something the the prior statement it is never a weird question to ask of there actually evidence of this or just it seems like it should be true so we believe it.
There are tons of things that seem like they would obviously be true, but it turns out they aren't.
I'd argue that assuming something is obvious without any empirical validation is the root of a huge number of misconceptions that humanity has historically had. There's a reason science suddenly started moving a lot faster after we moved past Aristotle and started measuring things in experiments.
Overall cyclomatic complexity is a useful metric, but it does have one shortcoming when used with modern languages: it was invented before polymorphism really became a thing.
That means that it really only counts explicit branching. So, for example, in an OO language like C#, calling a virtual method doesn’t increment cyclomatic complexity even though the method invocation could go down many code paths. Potentially thousands if you’re dealing with a common interface like IEnumerable. If you’re working on a library then the number of potential code paths in this kind of situation is unbounded.
As an aside, it’s interesting to think how it might apply to a language like Smalltalk that doesn’t even have if or switch statements.
OO isn’t the only monkey wrench, either. Higher-order functions also introduce forms of branching that cyclomatic complexity doesn’t measure.
Again that doesn’t make it a useless metric. Just don’t think that a cyclomatic complexity limit in your codebase is some sort of maintainability panacea. Some of the least comprehensible functions I’ve deciphered had quite low cyclomatic complexities.
In reading the article, it did feel somewhat wrong in a way that I couldn't quite describe. But after reading this comment, maybe it just seems old and maybe obsolete.
Taking the example provided in the article, I don't feel like the new code is meaningfully less complex. In fact, since it added some additional indirection, I could argue it's slightly more complex.
The core code with the nested if statements is something that I would probably refactor in some other way entirely. Maybe by taking advantage of other language features. It is a toy example so it's hard to say but that's part that feels like it needs simplification and untouched in this example.
Agreed. I also don’t really agree that the refactored code is any easier to test. ProcessOrder’s behavior hasn’t changed, only its implementation details, so the minimal test surface is the same for both: just test ProcessOrder.
You could additionally test the three helper functions. But the original tests against ProcessOrder would still be needed for completeness, so they wouldn’t necessarily add much except in an Uncle Bob style, “He who dies with the largest burden of gratuitous micro-tests wins,” sort of way.
Now if I really wanted to make that code easier to test, I’d instead be looking into ways to make the whole thing less stateful. Temporal coupling is much more confusing than if statements.
In agreement with your post, this research that measures cognitive load via EEG and time spent shows that the metrics we use for complexity and readability are only partial matches to what is going on: https://pmc.ncbi.nlm.nih.gov/articles/PMC9942489/
Fun story: at my previous aaaawful company CC was discovered as a thing to care about at about the same time as PMs and managers were encouraged to land code changes using the _then_ quite terrible AI tooling (this was a year or two ago). Cue an avalanche of completely unreviewable diffs.