Mushan Yang

Kami + AI = the best thing since sliced bread. Watch our CEO, Hengjie Wang, share some product updates coming soon! Want to learn more? Watch the announcement and book a demo with me!

8

1 Comment

The most robust, reliable, and productive uses of AI are when many models are used in coordination with each other, with "regular" programming mixed in. This leads to more capable, more reliable, and more interpretable AI systems. Most people building with AI already know this. But (unless you work for Google) the main barrier to realizing multi-step AI workloads is an infrastructure one. For the rest of us, we’re left creating an unwieldy mess of chained API calls to multiple providers, with many roundtrips in between. This state of affairs is stifling progress. Nobody wants to deal with more tooling and infrastructure… but everyone would benefit from simple, intuitive interfaces that abstract away a powerful system underneath. Substrate is the first inference API optimized for multi-step AI workloads. With Substrate, you can write less code, run more inference, and build high performance AI applications with zero infrastructure to manage. No tooling, no infrastructure – just elegant abstractions. Excited to support Ben Guo, Rob Cheung and the amazing Substrate team on their mission to democratize how AI applications are built.

40

4 Comments

Today, I found a fascinating way to think about “Tokens” in LLMs. It came from a conversation between Jensen Huang and Patrick Collison at a recent Stripe event. When asked about the future of compute capacity in five years, Jensen smartly dodged that question a bit, but then gave an excellent explanation and analogy about why tokens are the new forces that will power the next decades of humanity. He explained that we're now producing something unprecedented (and at scale): floating point numbers that possess “value”, which we now call tokens. These tokens are valuable because they encapsulate “intelligence”. People are now taking these tokens and transforming them into English, French, images, videos, chemicals, proteins, robotic movements, etc. And many are working hard to expand the range of concepts and ideas we can create with these tokens. He then goes on to made a compelling comparison between tokens and electricity. In the past industrial revolution, we successfully found a way to convert “atoms” into “electrons” (by boiling water to power electricity turbines). And now, we've discovered a way to convert "electrons" into "tokens" (by using energy to power data centers that train and run LLMs). When electricity was first introduced, few people understood its value. Today, paying for kilowatts is routine. The same will happen with tokens. Right now, only early adopters and builders are paying tokens. Soon, everyone will be paying for tokens on a daily basis to supercharge productivity and power new products and services. Many new industries will be born from and built on top of tokens. When I first heard about this way of thinking, I got goosebumps. Perhaps it’s because Jensen has a good storytelling skill that helps him sell what he builds, but I have to admit this way of thinking gave me a brief surge of pride. I'm proud of humanity's collective effort and creativity, which have taken us from living under rocks to building machines that can "think." That's nothing short but a miracle.

18

5 Comments

My cofounder Saumil goes on the Stack Overflow Podcast to deep dive on AI Agents! He goes into detail how we leverage AI agents to power automated workflows to accomplish the most complex tasks. Including agents firing off tasks to other agents to tackle the details. Need an agent to verify your work? No problem. Give it a listen and let us know what you think!

14

The afternoon Keynote @ Stripe Sessions was Fireside Chat between Patrick Collison (CEO of Stripe) and Jensen Huang (CEO of NVIDIA). Patrick was an excellent moderator — asked Jensen really probing, nuanced questions that Jensen answered with his usual sharp wit and unique (and impressively well rationalized) perspective. A couple of quotes among many that stuck with me: - “Current CPU installed market is $1 trillion of hardware, all of which will be upgraded to GPUs over the next 5 years. And I would estimate about another $1 trillion will be spent on top of that during the same time period as companies try to do more with AI. Add those together — and I think at least $2 trillion will be spent on GPUs in the next 5 years.” - “I like to go after Zero-Billion Dollar Markets — nonexistent markets that I envision & can rationalize well, but will be very hard, expensive & time-consuming to achieve.”

1

TL;DR Last Sunday was Father's Day, unfortunately I couldn't be with my dad. But instead I had an incredible opportunity to reconnect with Haska Su. He was the lead developer (maybe the only one in most of the time) of a small game platform that brought joy and impact life of countless players, including myself. Back in high school around 2011, there was a Taiwanese web game hub packed with many types of Flash games—2D shooters, RPGs, pet, chess, etc. The platform drew in around 3000 live players at its peak. It wasn't just a gaming platform; it was a vibrant community, where players shared artworks, discussed gameplay strategies, and some even wrote a novel for a game. It was also the first time I get in touch with an online forum, where players discuss not only game related stuff but as well as academic study, memes, and even political debates. Haska is always there, interacting with us, swiftly solving issues, and fostering a lively atmosphere. It was one of the most memorable times of my life. Haska also introduced a game-changing feature: an in-game development tool. It let us create and publish our own games right on the platform—complete with map builders, flow control, special effects, etc. So many players created in-game games using the tool—there were story-based adventures, MOBAs, board games, memes, anime scenes—basically anything people could imagine. My high school friends and I were exploring it like avid adventurers. It was a revolutionary concept at 2012, similar to Roblox, which have achieved today with millions of daily users and a thriving community. However, the platform eventually declined, which was very sad for me and many others. Especially after going to university, we shifted to play other types of games, and PC games and mobile games started to dominate the market. The platform could no longer withstand it and dissemble later, and Haska maintained the platform alone from there. Unfortunately, another setback occurred later when the server data was accidentally erased, causing nearly everything to be lost. At some point, I had already thought it was all lost, vanished, and mourned its absence. But then three years ago, my friend told me that Haska rebuilt the entire platform and remade several old games using HTML5 and TypeScript, learning and building from scratch. The platform has been reborn, and players can now build games using tools or code directly with built-in libraries. Although it may not go back to its peak, witnessing new players using it again fills me with gratitude. I don't think I've heard a more inspirational story than this, especially since I experienced it personally. And getting to know Haska is even doing this from the very beginning when he had another main job in Canada and a family with multiple children, fills me with even more admiration. Makes me wonder if I can also have a fraction of such skills, mindset, and energy.

7

1 Comment

Don't fight against AI. Help it to get latest info. 👇 is the talk I gave last week regarding Real-Time #RAG in Zilliz's Unstructured Data and #LLM meetup. "Talk is cheap, show me the code" This blog shows the key data flow and code blocks. The full example is available at https://lnkd.in/gUj349hh How it works? Input: Stream stories and comments from #HackerNews API. Pre-process: Retrieve updates and filter for stories/comments via #Bytewax. Retrieve Content: Download the html and parse it into usable text. Thanks to the awesome #Unstructured.io. Vectorize: Create an embedding or list of embeddings for text using #HuggingFace #Transformers. Output: Write the JSON document with #embeddings and other fields to a local #Timeplus #Proton server, into the data streams. Filter/Route: Use streaming SQL to filter/transform JSON data and send output to a local #Kafka server. Forward: Use #Milvus plugin for #KafkaConnect to forward the data to #Zilliz Cloud. Query: Build a #streamlit web application to query the live data with the same embedding model.

18

1 Comment

Today, we’re launching Substrate. We’re also announcing our $8M Series Seed led by Lightspeed. We believe the most robust, reliable, and productive uses of AI are when many models are used in coordination with each other, with regular programming mixed in. Most people building with AI already know this. But the available tools are suboptimal, unless you work for Google. For the rest of us, we’re left creating an unwieldy mess of chained API calls to multiple providers, with many roundtrips in between. We took a hard look at this state of affairs, and recognized how much it is stifling progress. Building large multi-step AI workloads requires sophisticated high-performance tooling and infrastructure. Nobody wants to deal with more tooling and infrastructure… but everyone would benefit from simple, intuitive interfaces that abstract away a powerful system underneath. No tooling, no infrastructure – just elegant abstractions.

25

1 Comment

The best part of investing at the earliest stages of company building is every once in a while you meet founders who can see the future. Ben Guo & Rob Cheung are two of those founders and after a year of building in stealth they're launching ꩜Substrate, the first inference API optimized for multi-step AI workloads. 👉 https://substrate.run With ꩜Substrate, you connect nodes from a curated library that includes optimized ML models, built-in file and vector storage, a code interpreter, and logical control flow. By simply connecting nodes, you describe a graph workflow, which Substrate then analyzes and runs as fast as possible. Entire graphs of many nodes will often run on a single machine, with microsecond communication between tasks. We are thrilled to participate in their 💵 $8M Seed led by Lightspeed (Guru Chahal/Nnamdi Iregbulem) alongside our friends at South Park Commons (Arian Agrawal), Craft Ventures, Red Swan Ventures (Eric Rubin/Sandy Cass), joined by amazing 🧠 founders and 👨‍🏭 operators like Guillermo Rauch, Immad Akhund, William Gaybrick, Chris Best, Gokul Rajaram, Shreyas Doshi, Jana Messerschmidt, Michael Manapat, and Matthew Hartman.

23

1 Comment

In other news: Meta launched Llama 3 last night. 16 hours later, Simplismart has come up with this update. Finest real-time example of what you need in early-stage teams: bias towards action. My guess is this team managed to work all night and come up with this in less than 15 hours. Fastest inference engine in ≤15hrs? Crazy! Simplismart ❌ Simpli-FAST ✅ Amritanshu Jain Devansh Ghatak

21

Last year, I made the hardest decision in my career: I left the company I started with Siqi Chen. Like many of you reading this, I was enamored when ChatGPT launched and felt a deep urgency to be working closely with AI. Very serendipitously, I was introduced to Bret Taylor and Clay Bavor as they were building the initial team for Sierra (forever grateful to you, marius eriksen!). It was a big leap of faith – I had vowed to myself that I would neither join an early stage startup nor join a company where I didn't know anyone. This was both, but I joined anyway. One year in, I know this is the best career decision I've ever made. I'm constantly impressed by Bret and Clay's humility, compassion, and intellect, as well as the caliber of the team they've assembled. I couldn't wish for two better leaders to follow in building in what I believe will be a generational company. While the founders were the biggest reason I joined, I've found myself to be the most motivated I've ever been by the mission. In 5-10 years, I believe we'll have agents everywhere, but the problem of how to design, build, test, monitor, and maintain agents is unsolved and deceptively interesting. We’re on the frontier reinventing the software development lifecycle, and the companies that have innovated the most in the Agent Development Life Cycle will be the ones that create this future. My mission is to make Sierra one of those companies. While I am unsurprisingly posting this on LinkedIn because we are hiring, this is all straight from the heart. If you're remotely curious, reach out and let's chat. What's the worst that can happen?

550

18 Comments

Data is both the foundation of and the reason for code. Programs are written so that data can be sourced, parsed, stored, cleaned, transformed, displayed, bought, sold, shared, used, and more. LLMs may dominate some these categories (parsing and transforming) and will touch all of them, but that's about it. Not to mention areas like infrastructure, dev/ML ops, distributed systems, and embedded systems. LLMs are already insanely powerful and exciting, and it seems like they'll only continue to improve. But to say that the future of programming is making API calls to LLMs strikes me as unrealistically optimistic, and, considering it's coming from the CEO of the company poised to profit the most from this future, borderline disingenuous.

10

Huge congratulations to the LanceDB team, Chang She and Lei Xu, on securing an $8M funding round from CRV! Their mission to build The Database for Multi-modal AI is an exciting one. It's amazing to see how far they've come since Chang's days as a star VP Eng at Tubi. Check out the TechCrunch article for more details on their journey. #AI #technews #fundinground

103

1 Comment

During our time at Coinbase both me and my co-founder Niall O'Higgins had to deal with systems built using MongoDB and PostgreSQL at scale. Hence, I love reading detailed stories about production database migrations like this one from Infisical. In this article they talk about why they decided to migrate from MongoDB to PostgreSQL. Some highlights from the article: 1) The primary goal behind the migration was to decrease the barrier for self-hosting the platform for their users. 2) Lacking support across cloud providers: After MongoDB’s license change to SSPL, many cloud providers opted to offer older versions of MongoDB. As a result, they found it difficult to ensure the availability of features of Infisical for customers running on everything other than the latest stable version(s) of MongoDB. 3) Unsurprisingly, the platform experienced significant performance gains largely attributed to query optimizations with joins.  Having moved to PostgreSQL, they avoided these inefficient operations which also resulted in a 50% cost reduction on our database bill. This mirrors my experience and expectations. Obviously, given me and Niall O'Higgins's backgrounds, this is something we at StepChange.work can help your company with, so don't hesitate to message us. Also Niall O'Higgins has a banger of an article in the works about PostgreSQL problems he's seen at scale building systems used by 100's of millions of people. Reply below if you have specific questions for him or things you would like him to talk about. https://lnkd.in/gziztS_9

32

2 Comments

It’s time for Social Saturday #1 (Stories Beyond Achievements) 🌟 Meet Diksha Gohlyan, a Senior Software Engineer at NVIDIA and Technical Interview Coach at Interview Kickstart, and a former software engineer at Google. 🌟 What inspired you to pursue a career in tech? -> I always wanted to be an engineer growing up, the definition of engineering being building homes. As I grew older, software engineering became a luring field and my interest in computers and exposure to its power led me to pursuing a career in tech. 🤔 How would you describe a typical day at your job? -> A typical day for me would be a few meetings with my team and other stakeholders, a lot of hands-on programming solving problems and resolving bugs, and great discussions with teammates over lunch on any topic really. 🚀 What did you want to be when you were a kid? -> Engineer - Not sure if it was my parents who instilled that on me or the ambition in me wanted to do something different compared to what my family did, but I was clear I wanted to be an engineer, what kind, still unsure. 🌈 Share your favorite childhood memory.  -> This aces all the fun I had with my siblings. We were throwing balloons from our terrace during Holi, and it hit an aunty. She was furious and started ascending up the stairs. My sisters, who were older, jumped over the fence to the other side to hide, but I couldn’t reach and had nowhere to go but to descend down the stairs. The lady caught me and thrashed me. Even with the thrashing its still my favorite moment because it reminds me of all the fun we used to do as kids. 🤔 Your friends describe you as…  -> Hard working, always in a hurry, and sometimes helpful. 🕰️ Describe yourself when you were 22 years old. 🏆 An accomplishment you're particularly proud of? 😓 Reflect on a moment or time when you felt like you were struggling a lot.  🎓 What's the biggest lesson you've learned in your career so far?  ⚖️ How do you balance work and personal life?  🎲 A random fact about yourself. 🎬 What is your favorite movie or series of all time? 💬 A favorite quote or mantra that guides you in your career.  💼 What non-technical skills do you believe are crucial for success in tech? 🚀 A message to young people looking to pursue their career in tech.      Edit: Swipe through the images below to find out more! 👇📸 🙌 A shoutout to your mentor during your early career.  -> Devesh - My ex-manager from Google - I am still in touch with him. Connect with Diksha di by expressing what inspired you most about her journey. And if you're eager for more stories, stay connected! Sharing experiences is how we all grow! 🌟 #YourNewConnection #SocialSaturday #StoriesBeyondAccomplishments #TechInNepal #TechInspiration #NepaleseInTech #WomenInTech

96

On April 18th, just last week, Meta unveiled Llama 3, lauded as "the most capable openly available LLM to date". This remarkable achievement by the Meta team comes hot on the heels of Llama 2's release last July. Notably, the new models surpass their predecessors and competing offerings from other providers in terms of performance. Meta's commitment to open sourcing these powerful models while prioritizing model safety and responsible usage is commendable. (https://lnkd.in/ghv3RJRz) Yesterday, Microsoft introduced Phi-3 Mini, a member of the Phi-3 family of models, promoted as "the most capable and cost-effective small language models available". According to Microsoft, the key to achieve high performance in such a small package lies in the quality of the training data. The model has been open sourced and is now available on Ollama and Hugging Face. (https://lnkd.in/gKgSmrBn)   These compact yet powerful models serve as invaluable tools for creators eager to bring their visions to life or simply experiment with cutting-edge technology, particularly for those with limited access to resources.   Shortly after the release of Llama 3, reports began surfacing of enthusiasts running the 8B model on a Raspberry Pi 5 equipped with just 8GB of RAM—a modest $80 single-board computer. (https://lnkd.in/gbeQPhQ2) While the performance understandably reflects the limitations of such a platform, the fact that the model runs and generates an output is nothing short of magical. Similarly, the Phi-3 Mini model can run on ubiquitous devices such as smartphones.   Reflecting on my own journey, I recall the pivotal role that early access to a low-cost computer played in shaping my passion for coding. The gift of an 8-bit Commodore 64 home computer ignited my curiosity at a young age. Later, with the guidance of an outstanding Computer Science teaching staff at my high school IISS Marconi-Hack Bari, I embarked on ambitious projects like a networked multiplayer version of Battleship written in Turbo Pascal and 8086 Assembly, complete with custom sprites and lo-res graphics. These formative experiences paved the way for my career, eventually leading me to various engineering and leadership roles.   It all began with a simple, low-cost, modestly capable computer used for experimentation—a gateway to discovering my life's passion.   I encourage educators and parents alike to consider setting up similar low-cost experimentation environments. By introducing younger generations to open-source AI technologies, we can nurture their creativity and help them uncover their true passions. It's not just a weekend project—it's an investment in their future and the future of innovation and discovery. #education #youth #AI #LLM #future #innovation

24

Check out our latest blog to see how our Notification Team harnesses the power of Graph Neural Network (GNN) models to create personalized restaurant recommendations for user notifications in our latest blog.

59

April was an exciting month for AI at Meta! We launched MTIA v2 https://lnkd.in/gvC-UgVN, Llama3 https://lnkd.in/gdbfXa-6, presented a tutorial and paper on the PyTorch2 compiler at ASPLOS https://lnkd.in/gsT9DUr5, released PyTorch 2.3 https://lnkd.in/gYn3Vnzm and, to top it off, we launched the PyTorch ecosystem solution for mobile and edge deployments, ExecuTorch Alpha https://lnkd.in/gWv96jpm optimized for Large Language Models. What better than to combine all of these... running Llama3 on an a mobile phone exported with the PT2 Compiler's torch.export, and optimized for mobile deployment. And you can do all of this in an easy-to-use self-service format starting today, for both iPhone and Android as well as many other mobile/edge devices. The video below shows Llama3 running on an iPhone. (Makers will love how well models run on Raspberry Pi 5!) PyTorch is being developed by a multi-disciplinary team comprising ML engineers, accelerator experts, compiler developers, hardware architects, chip designers, HPC developers, mobile developers, and specialists and generalists that are comfortable across many of the layers involved in building end-to-end solutions. Even better -- if you're excited by the possibilities of AI, and solving the system design challenges of making AI run well across all hardware types, we are looking for YOU! The Pytorch team has openings across PyTorch core, compilers, accelerators and HW/SW co-design https://lnkd.in/gu2Y_KpY https://lnkd.in/gQVXBP6P and a broad range of positions that involve PyTorch from model development all the way to hardware deployments https://lnkd.in/gjDdUh5w #PyTorch #ExecuTorch #Llama3 #AICompilers #MTIA #AcceleratedAI #MetaAI #Meta

64