In this episode of the AI + a16z podcast, a16z General Partner Anjney Midha shares his thoughts on how hardware for artificial intelligence might evolve over the years to come as we place more emphasis on AI inference workloads. Improvements in sensors, chips, models, and more could result in remarkably useful models that are able to run locally, resulting in wearable AI products that evolve the human experience.
Here are some highlights from the episode:
[5:04] “If you go all the way back to the first neural networks in 1958, that’s when we start to see reasoning start to happen through computers. . . And then now we’re at this next phase of massive transformers. . . That’s one lineage, which is the lineage of reasoning and intelligence.
“In parallel, we’ve had this other lineage in computing, which is interfaces. And you started with the command line and keyboards. And then that gave way to the GUI with a mouse . . . And then that has led ultimately to mobile interfaces with touch as an input mechanism. . . The question is: Where are we going next?
“I think we have really good reasons to believe that the next interface will be an AI companion that’s some combination of text, voice, and vision that can understand the world. That’s almost a better predictor of where hardware is going, because the history of computing so far shown that whichever one of those lineages is undergoing a moment of resonance with with customers ends up dominating for the kinds of workloads that then get to scale for the next 10-15 years. And I think we’re in the middle of both a reasoning and an interface shift, and that’s what’s exciting right now.”
[18:55] “I think what we need to be paying attention to on the hardware side is: Are there really low-cost form factors that startups can innovate around because there’s an existing supply chain that an incumbent, like an Apple or a Google has essentially subsidized at scale over the last decade? And that’s what’s so exciting about things like the Apple Vision Pro: When Apple gets into the game, it results in second- and third-order effects of new supply chains showing up for sensors like depth sensors and LIDARs and passthrough mixed reality displays, that then give startups the license to go experiment with new form factors.”
[35:13] “There was just like decades of research at many DARPA- and DOD- and university-funded labs that was the neuroscience-first approach to inventing computers. That has proven to be mostly a distraction. It turns out that just predicting the next token or the next word a model should say is a remarkably useful way to attack intelligence and design computers, instead of . . . getting computers to learn like human beings. . .
“I will say what is now happening is that because transformers are so remarkably effective at what they do, most major industrial labs have doubled down on that architecture. It’s not clear whether that will result in multi-step reasoning of a kind that is essentially unconstrained. It’s not clear that the current architectures will get us all the way to the end goal.”
If you liked this episode, listen to our inaugural two episodes from April 12:
Sign up for our a16z newsletter to get analysis and news covering the latest trends reshaping AI and infrastructure.
Check your inbox for a welcome note.
The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party advertisements; a16z has not reviewed such advertisements and does not endorse any advertising content contained therein.
This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.
Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.
Artificial intelligence is changing everything from art to enterprise IT, and a16z is watching all of it with a close eye. This podcast features discussions with leading AI engineers, founders, and experts, as well as our general partners, about where the technology and industry are heading.