Abstract. In this talk I give an overview of two important transformer models in NLP.

Abstract. A gentle introduction to graph neural networks (GNN)s

• Attention mechanisms in GNNs
• PyTorch Geometric example: Karate Dataset
• This will be an introductory talk with lots of pictures (in fact, it will almost exclusively be pictures!)

Abstract. A hands on guide to using PyTorch Lightning. Concretely, we grab the GPT-2 model from Huggingface and build a lightning module to train it (more or less from scratch).

Abstract. Models are getting increasingly large, to the point that they don't always fit on a single device! We discuss some techniques to partition models over multiple devices, from plain PyTorch to libraries like deepspeed.

Abstract. In this talk we'll review the GPT-1 paper [Improving Language Understanding by Generative Pre-Training - Radford et al][1]. By way of setting the stage we will give a brief review of the Transformer architecture.

Note: We didn't have time to cover GPT-2 in this talk, but some slides on the topic made it into the deck.

Abstract. In this talk we'll review the 🤗Hugging Face Transformers library. This is an open-source library with the stated goal to "democratize NLP". We'll briefly review some background, explain what problems Huggingface is trying to address, and cover some of the tools and techniques they provide. We will not assume any familiarity with transformers.