You Know Me, AI

A new tool for reviewing new technologies, the Seinfeld Test:

There has been some discussion in our office about the use of AI. There has not, to date, been any use outside of a couple of experiments. Before I describe the way we see the issues, I should discuss my bias: I am pretty much against the use of generative AI in design, although some lesser forms of machine learning are fine. (I’m not giving up predictive spellcheck any time soon.) I’m not particularly against new technology, but I look for technologies that reduce work rather than create it, which brings us to accuracy.

There are multiple criticisms of AI, but in an engineering design, the top of the list is accuracy. When I see AI output in engineering discussions, it seems to always have errors. Sometimes I see it in terms of “we used AI for the first pass and now we’re cleaning it up,” which is fine. Sometimes – particularly on LinkedIn – I see people using AI to make illustrations of engineering principles and those pictures are nearly always full of mistakes. I’ve seen examples with types of bridges, types of building frames, and types of analysis, and they all contained wildly-wrong information. This is not fine. Even if it’s just for show on LinkedIn, having wrong information distributed in public poisons the well.

So any AI engineering work product needs to be reviewed by a human capable of performing the work that the AI is doing. If it’s frame analysis, than the reviewer has to be able to perform that analysis. If it’s drawing types of bridges, then the reviewer has to know (or be able to research) different types of bridges. To get past needing that kind of review, we’d need a long track record of error-free performance. When I ask AI proponents if the output was error free before review, the answer is no. Maybe it will be someday.

My description above can be repeated for any tool. The difference between AI in its current state and the usual software tools we regularly use is that the usual tools give in context perfect accuracy. The context is GIGO: garbage in, garbage out. The usual tools are accurate when the input that we are responsible for is accurate. That is where AI breaks the paradigm we’ve been using for software for at least as long as my career: your input prompt for AI can be correct and the results may still be inaccurate.

For the foreseeable future, OSE will be working without AI and experimenting with it in private. And that will change only when we’re sure that there will be no reduction in safety for our projects. For the record, that includes writing as well as analysis and design.

---

The post title is obscure enough that I should probably explain. You Know Me Al is a 1916 novel by Ring Lardner, in the form of a series of letters from a professional baseball player, Jack, to his good friend, Al, back home. Jack is naive, brags, and is manipulated by just about everyone he encounters, and he doesn’t see any of that, but his letters to Al reveal it to the reader. In particular, on occasions where Jack has done something particularly foolish, he often begins his description in a letter with “You know me, Al…”

The ease of typographical puns between “Al” and “AI” obviously played a part in my choice of title; the disconnect between Jack’s view of things and reality may, just may, also have played a part.