The second before the first token
A team I worked with spent four months improving answer quality and could not work out why nobody used the product. The model was not the problem. The pause was.
I write down what I measure, mostly so I stop repeating myself in calls. Four pieces so far, all of them about things that broke.
A team I worked with spent four months improving answer quality and could not work out why nobody used the product. The model was not the problem. The pause was.
Three production systems, same three bugs, none of them about chunk size. Including the one I shipped myself and did not notice for six weeks.
It is faster than the API for two of my four workloads and worse at the other two. The split is not where I expected it to be.
Not an argument about frameworks. An argument about who gets paged at 3am.
Not a newsletter, not a course, and not an attempt to build an audience I then sell something to. I get the same six questions from people I work with, and writing the answer once is cheaper than answering it every time.