Object detection — In recent years, ml community has made huge progress in object detection through models such as Faster R_CNN, Mask R-CNN, YOLO among others. In this Let’s consider system outputs as predicted classes and bounding boxes. Intersection over Union (IOU) : IOU evaluates the overlap between ground-truth bounding box (gt) and…

What is multi task learning ? — In machine learning, the general objective is to learn one model for a task given the dataset corresponding to that task. This can be seen as single task learning. If you learn a single model jointly for multiple tasks it can be termed as multi task learning (MTL). For example…

What is it ? — We are still in early days of AI. The general objective is to find the architecture which gives us the highest accuracy for a given task essentially “buying” stronger results through paying high computational cost. The authors of paper Green AI called this trend as Red AI. …

In this article we describe various flavors of model evaluation in a typical large scale model development lifecycle. We focus on the following stages : Offline training Model processing Recurring training (Post deployment) Continuous evaluation (Post deployment) 1. Offline training : Offline training involves training from scratch. Developers experiment with various modeling paradigms (architecture…

The best thing since sliced bread came around ! Pre-trained word embeddings are an integral part of modern NLP systems, offering significant improvements over embeddings learned from scratch. There are two existing strategies for applying pre-trained language representations to downstream tasks: feature-based and fine-tuning. Feature based approach : Throw in pre-trained representations as additional…

Sync SGD has emerged to be the most popular training paradigm for large scale training. Let’s see why ? Let’s study the algorithm for distributed SGD with all reduce : Distributed : The parameters of the model architecture can be distributed across several machines C1, C2, . . . …

wide and deep model model size — There are two aspects of learning : memorization and generalization. Memorization comes from learning frequent co-occurrence features and exploiting the correlation available in the training data. Generalization comes from learning hidden patterns and correlation between features and labels. Wide & Deep learning is jointly training wide linear models and deep…

Why does it matter ? — Perf matters ! The deep neural network computation mental model should help you to do back of the envelope computations for your components. A typical training iteration contains three steps: forward pass to compute loss, backward pass to compute gradients, and optimizer step to update parameters Forward Pass, Backward Pass, Optimizer step Forward pass refers to…

Why does it matter ? Model size is a key metric to drive optimizations such as : Training system design : Choosing specific sgd optimizer, network bandwidth, hardware (memory and processing). Inference system design : Model file storage optimizations, quantization, model splits for serving. How do we measure it ? While describing model size (in memory units such as GB), we…