Let’s Talk About The Robots…

I’m about a month into coding with Claude Code almost daily. Recall, dear reader, that I don’t know anything and nobody pays me to have opinions. Nevertheless, here are my thoughts on this experience. Back to algorithm-posting tomorrow.

$\textbf{(Probably) Good Things!}$

• Actions Per Minute Are Way Down — I have a limit on the number of things I can do in a day. In this context, my definition of a thing isn’t restricted by size. A thing could be as small as hitting $\textbf{⌘ + tab}$ and using $1/10000^{th}$ of my daily brain-power on locating where e.g. $\texttt{cfgProvider.CompactorMaxPerBlockUploadConcurrency}$ is set. I could even argue that hitting $\textbf{⌘ + tab}$ alone is a thing. Regardless, Claude reduces the number of things I need to do or think about to a coreset. This has been an incredible boon, shedding load is a very effective strategy!

$\textbf{N.B}$ — Shedding load works even better when you’re operating near your limit. See this chart from Improving the QoS of Web Applications across Multiple Virtual Machines in Cloud Computing Environment.

• I’m Working Harder on Testing and Verification — Because I have $\textbf{(1)}$ more free brain-power and $\textbf{(2)}$ strong incentives to not ship any of Claude’s bugs (they are in fact my bugs!), I’m spending more time thinking about testing. This is great! I wrote my first non-trivial property-based tests earlier this month.

• We’re Entering an Age of Thinking Above the Code — I still think “Prompt Engineer” is a silly title, but I’ve found myself putting a good deal of effort into prompting. This means keeping meticulous notes on architecture, specification, and a sort of pseudo-code to pass between Claude Code sessions. I don’t think I’m alone in these newfound habits. If things go right, this will lead to a trend in the industry where we all spend more time on what Leslie Lamport calls *Thinking Above the Code*. See Lamport @ MSFT Research.

$\textbf{N.B}$ — I waffled on this a bit. I am not actually sure which of the following more closely describes how I feel at the moment.

• I think “Prompt Engineer” is still a silly title.
• I still think “Prompt Engineer” is a silly title.

Apologies for the “Real Eyes Realize Real Lies” comment here, but it’s been apparent for years that there is such a thing as skillful and unskillful prompting. I may have said that “Cloud Financial Operations Engineer” was a silly title before I saw some real series D startup cloud bills. I’m sure you get the point here…

• Activation Energy Is Way Down — I only use Claude Code for my job. That said, Claude has made it far easier to work on chores that I’d otherwise be tempted to put off (e.g. writing jsonnet that generates your K8s manifests). A handy, maybe obvious, pattern that I’ve become fond of is:

  1. Describe a problem in as much detail as I can while hedging almost every statement that I’m not totally certain about.

  2. Give Claude an extensive list of files that might be relevant.

  I’m told that one should be tactical with the information you give Claude. However, for around 3 USD, I’ve saved myself hours of half-hearted skimming. Clearly, this is good for me in the short term. Unclear if it’s good for me or my employer in the long term.

$\textbf{(Possibly) Bad Things!}$

• Memetics — In one instance, Claude pitched me on $\$\texttt{SOLUTION}$. I rejected it and later discovered that Claude independently pitched $\$\texttt{SOLUTION}$ to my two human reviewers. While $\$\texttt{SOLUTION}$ was technically valid, it was quite non-obvious. (Others have noticed this! see: Mitchell Hashimoto Tweet).

  I’m noticing a rapid growth in “weird” PRs. PRs that fix a problem in the most non-obvious way (when there ARE obvious ways to do it). I suspect it’s LLM written but non-disclosed. I’m a big fan of code assistance but without an experienced driver the result is some real slop.

  I suspect (pure conjecture) this happened because our repo’s $\texttt{CLAUDE.md}$ strongly discourages breaking certain rules from the Go specification. The obvious solution would probably ruffle Rob Pike’s feathers, and Claude probably internalized those instructions a bit too deeply. We know LLMs have “verbal ticks” (e.g. Why Does ChatGPT “Delve” So Much?), I wonder if we’ll begin to notice a similar uptick in specific code patterns. We’ll see.

• I’m Tossing Too Much Writing Into a Void — Earlier, I said I’m doing more writing. Of the additional $300\%$ (?) more I write per day, almost none (maybe $10\%$) of it makes it to a form usable by other humans. I am trying to be disciplined about this, but I am concerned that this $10\%$ may go sharply negative. We should take care that optimizing on human $\rightleftarrow$ agent and agent $\rightleftarrow$ agent communication doesn’t degrade the quality of human $\rightleftarrow$ human communication.

• It goes without saying, but you should apply a bit of discipline to your usage. It’s not productive to constantly evaluate “Is it worth $2 to pass this operation off to Claude?”, but I did incur $1.95 worth of charges earlier this month for what could have been a single $\texttt{sed}$ call (a somewhat complicated one, but one nonetheless). The 10 minutes of my time is worth more than $2, but that was probably a bit gratuitous.