All posts
Editorial
Engineering
Insights
Product
Residencies

Porting a machine learning model from GitHub to RunwayML in 5 minutes šŸ…

User Avatar
Cristobal Valenzuela
/
July 30, 2019
You can take an existing machine learning model, written in any deep learning framework, and add it to Runway with just a few lines of code. Learn how to do it here šŸ‘‡

Update, April 2022: As of now, you will no longer be able to port new models to Runway or update existing models. Models which you have already ported will remain available for the foreseeable future.

For this short tutorial, we will be portingĀ Progressive Growing of GANs (PGAN), an open-source model that creates high-quality images, available in theĀ Pytorch HubĀ and GitHub. The code to run and generate an image with a pre-trained PGAN checkpoint from the Pytorch hub is short and simple, letā€™s review the base code we will be using:

Creating a Runway model

Letā€™s start by creating a file calledĀ runway_model.py, pasting our sample code from above and adding theĀ runway-pythonĀ SDK. We will also addĀ numpyĀ for some minor post-processing. (Be sure to install the Runway python SDK withĀ pip install runway-pythonĀ first!)

Now, letā€™s wrap the loading of the model withĀ Runwayā€™sĀ setupĀ decorator. The setup decorator will only run once, at the beginning of the inference process, andĀ should always return a model:

The setup function is used to load the checkpoints and render any UI elements in Runway that will be part of the initial model configuration. In this case, just by defining aĀ runway.categoryĀ and adding the default PGAN checkpoint as options we get this nice UI in Runway āœØ:

Next, letā€™s wrap the inference part of the model in aĀ Runway command function.Ā A command is a special function that is used to define a way you can interact with a model. This will allow you to define input and output types.

A command decorator functionĀ is built with three options:

  • TheĀ nameĀ of the command: in this caseĀ generateĀ ,
  • TheĀ input type: PGAN generates images from a noise vector of length 512, so we need to provide it with theĀ runway.vectorĀ input type.
  • TheĀ output type:Ā In this case, the output will be anĀ image type.

Defining a command that takes a vector as input creates aĀ vector grid interfaceĀ to navigate the latent space of PGAN and show the resulting image as the output.

Now, we are ready to add our model to Runway! Letā€™s create aĀ runway.ymlĀ file, next to ourĀ runway_model.pyĀ defining how our model should run:

This file provides instructions for defining an environment, installing dependencies, and running your model in a standard and reproducible manner.

Thatā€™s it!Ā Letā€™s create a GitHub repo and push our code to GitHub and get our model in Runway.

Adding the model to Runway via GitHub

Runway can take any existing repository that contains aĀ runway_model.pyĀ andĀ runway.ymlĀ file and create a model that you can use and share in Runway.

Open Runway and click ā€œFrom GitHubā€ in the lower-left corner of the models' directory, select your repository and follow the steps to add a model.

Once the model has been added, Runway will start the build process automatically. You can check the process on theĀ versions tab.Ā Every time you push a commit to GitHub, Runway creates a new version of your model. šŸŽ©

Once the model has been built, you can add it to your workspace and start exploring! šŸ¦„

If you want to learn more about how Runway works, check out the links below. We would love to get your feedback!

Links

Share
Everything you need to make anything you want.
Trusted by the world's top creatives