MLOps: An overview to Machine Learning Demos

Overview

What framework to use, where to host the code, and where to host the model that can be easily loaded into those frameworks.

Model hosting

Gradio

Official website: https://gradio.app/

Gradio(app+deploy) and huggingFace(model hosting)

Pros of Gradio:

  • Quick and easy setup out of the box
  • Runs directly in notebooks
  • Absolutely no knowledge of web development required
  • Apps are easily shared
  • Good selection of built-in UI elements
  • Features such as screenshots or output flagging are very handy for demos

Cons of Gradio:

  • Limited control over the UI layout
  • Not suitable for complex apps (e.g. state management, caching)

Sreamlit

Xgboost: https://medium.com/towards-data-science/deploying-ml-models-using-streamlit-5d6212453bdd

Pros of Streamlit:

  • Quick setup
  • Advanced features such as caching and state management allow for complex apps to be built
  • Large selection of built-in UI widgets
  • Highly customizable UI layout
  • Extensible through support for custom third-party components

Cons of Streamlit:

  • Sharing applications is not as trivial as it is using Gradio
  • Complex apps require some understanding of advanced web development concepts
  • Not compatible with notebooks
  • Lacking some basic built-in features for ML demos (e.g. flagging of unexpected inputs/outputs)

HuggingFace Spaces

Spaces is like a “github page” for models, it creates a repo on huggingface, hosting the codebase serving model. It supports you to choose either Streamlit or Gradio as serving framework.

Check out this video to see how it works:

In this case the model is hosted on huggingface, trained with autoNLP, which still priced quite expensive right now. https://www.youtube.com/watch?v=s-6V6HlwX98&t=0s

Full-stack

Flask

https://flask.palletsprojects.com/en/2.1.x/

https://github.com/flask-restful/flask-restful

  • A back end responsible for loading and serving the model as a REST API
  • A front end to provide UI elements to interact with the back-end

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store