Mike Conover from Databricks was guest speaker at Latent Space. He’s a really smart integration engineer, who understands the trends, and who can use algorithms built by others as tools. Summary:

  • Infrastructure needs for Large Language Models (LLMs).
  • How Dolly was made on Databricks, with 1h of training at cost of $30.
  • What infrastructure was missing, and Mike wished existed, to simplify this kind of LLM development
  • He calls that type of infrastructure “LLMOps”.
  • Example - we’re missing good automation for checking accuracy of question/response interactions. Another model could be built to automate that.
  • Example - computer vision can use thumbnails to quickly summarize large images or videos. With text, there is no notion of thumbnail yet.
  • Example - it would be nice to have infrastructure that runs multiple models in parallel, and displays results in thumbnail format on a spreadsheet (where these are ‘thumbnails for text’, if such a thing could be invented)
  • Enumeration of products that could be built with LLMs
  • The state of the art in closed-source LLMs
  • How open source datasets and models are catching up
  • How companies like HuggingFace and Databricks are successful because they share information, and build things as open source
  • Why there is also a need for closed-source products
  • Why closed-source companies drive the success and excitement around open-source businesses
  • Trends in the next 9 months for LLMs