Understand It Before You Swat It
My son found a display bug in Versine. Behind it was a worse one: a skill map rendering in an order its own dependencies didn't agree with.
My son found a bug in Versine this morning, and when I watched him demonstrate it over his shoulder, I saw another bug to boot. His first bug was easy enough to fix: in one Versine level, the student compares fractions to see which is bigger. The display showed them in one order, and the answers sometimes showed them in a different order. Worse than that though was a different issue: he had not completed factoring and had not learned how to find the lowest common multiple (LCM), which is required for numerical comparison of unlike fractions.
Digging a little deeper on that second issue revealed that the order in which I’m rendering skills in Versine sometimes places foundational skills later than the skills that depend on them, simply because they come from another category. This means my son was working on a skill without having completed the foundational material for it yet, and to me, that’s the bigger bug.
The first bug (and also the not-exactly-a-bug of the display basically giving the answer away graphically by showing the fractions next to each other as partially filled-in progress bars) is easy to fix, one simple AI prompt and it’s gone. The second bug is a lot more complicated, is related to dependency trees and dependency management, and fixing it in the right way will absolutely improve Versine.
How do you even begin to fix a problem like this?
First, Understand
You can’t fix a bug you don’t understand, and surprisingly, it can be dangerous to just let AI fix it for you as a cop-out. The harder bug about rendering order for skills is complicated enough that telling AI to hit it with a hammer can make things much worse by potentially deciding a direction for your project that you don’t want it to go in. So, where do we begin on this one?
First, you have to know that the way skills render in the skills map is that they render with their category. The problem is, some skills depend on other skills from other categories that have not rendered yet. Fractions is the second category in the list. Factors & Multiples is the fifth. And Add & Subtract Fractions, sitting right there in category two, depends on Factors & Multiples, sitting way down in category five!
We need to get deeper in order to fully understand what’s going on here.
When you look at how skills are defined, they depend on other skills. Currently there are no circular dependencies anywhere in there, which is kind of an accident of design. Because circular dependencies (two skills depending on each other) would be a real problem, I have informed the AI that this should be a rule in this project: no skills should ever have circular dependencies. If they end up having circular dependencies, then the levels in those skills should be broken up into separate skills that are more granular and incremental so that the parts that don’t have the circular dependency can be handled separately.
That’s worth locking down, but I should be honest: it isn’t what broke here. The category list is just a list, ordered by hand, and the dependency graph is built from whatever each skill declares it needs. Nothing was checking the two against each other. So the moment I added a skill in the wrong spot, they quietly disagreed, and my son walked straight through the gap.
That’s one part of the problem understood, but how then do we render skills in the skills map?
What we know:
- Rendering by category alone doesn’t work; some skills might depend on something from a later category.
- Versine has a dependency tree, where skills can depend on other skills, to help students pick up skills in the right order.
- The category list is ordered by hand, and nothing checks that order against the dependency graph.
- Circular dependencies are bad in this case, so we need to make sure they don’t bite us later.
With that, we’re ready to start attacking this bug!
Swat That Thing!
The circular dependency rule is the easy half: just don’t let there be any! That’s a validation check, not a complete fix. We still need to give some love to the skills rendering order logic. That can be complicated, and if ever you get stuck on a thing like this, you can always ask a sufficiently advanced AI model to explore the idea, come up with solutions, and tutor you through the options.
My thinking is already in the trees, so I came up with this idea: while rendering, keep track of which skills have already rendered. Render as before, category by category, but when we encounter a skill that depends on something from another category, render what is depended on first, all the way up that tree until all of the dependencies have been rendered exactly once so far. THEN render the current skill that had the dependency. Once you get to the later categories that dependencies might have been loaded from, continue rendering whatever skills remain in that category that have not yet rendered.
This does mean that categories will be kind of mixed into each other in the skills map, but I’m totally okay with that; I would much rather have my kids understand what they are working on than have them confused about how to do something and not realize that’s not their fault.
So, here’s what I did: I opened up my AI assistant and explained the problem to it, in language something like the following:
Versine currently renders Skills in category order.
This causes a bug where some skills render earlier than other skills they depend on.
To fix, let’s update the code so that for each skill, we check the “depends on” list, and render all unrendered skills from that list, recursively.
Then, proceed rendering the dependent skill and all remaining skills from the current category.
We’re aiming for category-first, with dependencies injected when they are needed.
AI got to work, fixed the bug, committed (to source control, in this case git, which you’ll need to know all about if you want to write software with AI), and pushed the changes to the remote repo. It was able to do all of that because I provided it with enough detail and direction in that prompt to get it to make the right decisions. Your results will vary, especially if you use different models from what I used.
In all, it took me maybe 5 minutes to fix all of these bugs. It took me longer to write this blog post!
Go Deep
Fixing bugs like this requires DEEP understanding, and non-technical folk might not be used to going that deep. That’s okay! AI is reasonably good at it, and so long as you pay attention to it and ask all of your questions, even when you think your questions are stupid, you should be good to go.
That does mean though that if you try to build something, you’ll be learning things about it that are detailed in the extreme. This can be intimidating, but again, lean on AI, it can tutor you through the worst of it. In the end, you’re re-structuring your own way of thinking so you can logic through the un-logic-able. That skill translates into everything else in your life and can help you realize where you’re right and where you’re wrong in all other areas.
If you ask me, I think that’s pretty cool!