Articles

  • Why You Need a Thinking Partner (like this Genius)

    You may not have heard of him, but James Clerk Maxwell wrote down what we now think of as four equations that define ALL of electromagnetics. You can get a T-Shirt with them emblazoned on it. If you want to know how radio waves propagate, or how a radio functions, or how anything running on electricity…

    Read Article…



  • Wolfram Alpha and ChatGPT make a power couple

    Originally posted on LinkedIn in March, 2023 It’s happening. Earlier this year I opined about how AI systems would interface with authoritative data sources, based on a Stephen Wolfram blog. Insert your own comments here about the pace of change…but, yeah, it happened. And SW is on it. https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/ This is good news for systems…

    Read Article…



  • Why talking it out helps solve problems: Leveraging latent spaces

    I’m someone who benefits a lot from talking out a problem–often in front of a whiteboard. This is pretty common. But why does it work? We had been previously discussing latent spaces — a kind of internal map connecting different pieces of knowledge in different ways — as part of “understanding” and this is another…

    Read Article…



  • Does Generative AI Understand? Surprisingly: It kind of does

    Dmitri Mendeleev was a smart guy. In 1869 he wrote out a card for each element and started sorting them by increasing atomic weight. He noticed that there were repeating patterns, for instance, the noble gases. Thus was forged the periodic table of the elements. Question: Did Mendeleev “understand” the underlying structure in a way…

    Read Article…



  • LinkedIn Layoffs: free mentoring for affected engineers and managers

    I’m sorry to hear about all the layoffs today. I’ve been through that more than once in my career, and it is crushing. Once the shock wears off, it just feels…depressing. Please take some time and digest the situation, and reflect on your many accomplishments and the many lives who have been brightened by your…

    Read Article…



  • Will LLMs always take millions of dollars to train? Three factors to consider

    Estimated costs to train GPT-4 range from $60 to $100 million USD. But will it always take millions of dollars to train a new foundation language model? There are (at least) three different forces in play, and they are pulling in different directions. (1) The predominant factor is “bigger is better”. The more parameters in…

    Read Article…



  • Will AI Oversaturate? (2000s Calling… They Want Their Hype Cycle Back)

    If the “XML hype cycle” doesn’t spark vivid memories, go ask someone who’s been in IT for 25 years. 🙂 XML was to be the solution to all of the world’s data problems. C-Suite execs who wouldn’t know a parser from a parsnip we getting routinely quoted in press releases, echoing the praises of their…

    Read Article…



  • Hollywood Writers Strike Ends with Surprisingly Reasonable AI Agreement

    It was the longest strike to date, but now it’s over. Here’s a pretty good summary of the agreement. Of course, I like to think that the withering pressure of the threat to replace studio execs with AI had something to do with it. 🙂 As far as AI goes, the basic agreement seems reasonable:…

    Read Article…



  • Unlocking the secrets of AI: LlamaIndex under the hood

    How does this thing really work? The vast majority of tutorials show how to do cool stuff with this great library, but don’t even mention what’s actually happening. What is the LLM actually seeing? What are the intermediate steps? How does it do query planning? Take the basic tutorial listed as a starting point in…

    Read Article…



  • How to think about AI: The Economic Model

    Arguably, present AI systems as well as those on the horizon don’t fundamentally offer new capabilities. They make easier things that could eventually get done by a human–or larger group of humans. (If you have counter-examples, please leave them in the comments below). For example, a code-completion LLM like Github Copilot does what an new-grad…

    Read Article…



  • Explaining Chatbot AI “Temperature” (and Why It’s Terribly Named)

    I’ve seen a few people ask about the “temperature” setting seen in some generative AI apps. Here’s the scoop. Please share this with any confused souls. First of all, know that the term “temperature” comes from the world of AI, not from from a particular app. Which frees me to say that it’s terribly named.…

    Read Article…



π