Versioning Three Decades of Models

• Idea: Show how MLOps can be used to version ML models by applying it to 33 years deep learning models. By doing so, we demonstrate the role of MLOps in making experiment tracking, data versioning and model deployment.

The project is built around a blog post by Andrej Karpathy, in which he recreates Yann Lecun’s 1989 CNN on MNIST, which was a very early breakthrough convolutional neural network. Then with the benefits of 33 years of machine learning knowledge, the Karpathy shows how he can add different features to the model and different data augmentations to improve performance

We can show how MLOps can be used to showcase these changes. We can version models to allow retrieval of models from various eras with different features added/removed, we can do data versioning to show how to manage the processing of handling multiple datasets while maintaining train/val/test splits and we can compare model performances with an experiment tracker.

We can also showcase deployment, demonstrating how you can set up a simple application to which users can submit queries (in the form of images) to any/all models that we’ve trained

As a stretch goal, we can look at if there are any advances in the last three years that can improve the score, or if there are any techniques that would improve the score

• Business Case: Rather than being strictly innovation, this project is about showing off our core compentency, MLOps, in a setting that people are likely to find compelling. We get to explain what MLOps allows us to do, whilst simultaneously showing that we understand machine learning itself.
• The choice of topic also means that we are exploring a subject that has already been shown to be popular https://news.ycombinator.com/item?id=37268610

Tangible Outputs: The tangible outputs of the project are

• A Github repository containing all the code required to replicate our experiments and tooling
• A blog post explaining the decisions made in designing the system and the benefits of MLOps in this case, as well as how it generalises to application in industry
• Social media posts on subject that we know is of interest to industry figures