Open in app
Home
Notifications
Lists
Stories

Write
Jaideep Ray
Jaideep Ray

Home
About

2 days ago

Check your segment metrics !

ML models are increasingly used in applications impacting a diverse and large population. The models are evaluated based on their primary metric performance on the whole population. For example, a recommendation model evaluation is based on an offline metric such as normalized entropy, ndcg computed on human judgements and online…

Mlops

2 min read


Published in Log-Loss

·Jun 6

Evaluation for perception systems

Object detection — In recent years, ml community has made huge progress in object detection through models such as Faster R_CNN, Mask R-CNN, YOLO among others. In this Let’s consider system outputs as predicted classes and bounding boxes. Intersection over Union (IOU) : IOU evaluates the overlap between ground-truth bounding box (gt) and…

Machine Learning

3 min read

Evaluation for perception systems
Evaluation for perception systems

Published in Better ML

·Jun 2

The benefits of multi-tasking !

What is multi task learning ? — In machine learning, the general objective is to learn one model for a task given the dataset corresponding to that task. This can be seen as single task learning. If you learn a single model jointly for multiple tasks it can be termed as multi task learning (MTL). For example…

Ml Platform

3 min read

The benefits of multi-tasking !
The benefits of multi-tasking !

Published in Better ML

·May 26

Red AI => Green AI

What is it ? — We are still in early days of AI. The general objective is to find the architecture which gives us the highest accuracy for a given task essentially “buying” stronger results through paying high computational cost. The authors of paper Green AI called this trend as Red AI. …

Mlops

2 min read

Red AI => Green AI
Red AI => Green AI

Published in Better ML

·May 4

Flavors of model evaluation

In this article we describe various flavors of model evaluation in a typical large scale model development lifecycle. We focus on the following stages : Offline training Model processing Recurring training (Post deployment) Continuous evaluation (Post deployment) 1. Offline training : Offline training involves training from scratch. Developers experiment with various modeling paradigms (architecture…

Mlops Without Much Ops

3 min read

Flavors of model evaluation
Flavors of model evaluation

Published in Log-Loss

·Apr 30

Pre-trained representations

The best thing since sliced bread came around ! Pre-trained word embeddings are an integral part of modern NLP systems, offering significant improvements over embeddings learned from scratch. There are two existing strategies for applying pre-trained language representations to downstream tasks: feature-based and fine-tuning. Feature based approach : Throw in pre-trained representations as additional…

Machine Learning

2 min read

Pre-trained representations
Pre-trained representations

Published in Log-Loss

·Apr 29

Distributed Sync SGD

Sync SGD has emerged to be the most popular training paradigm for large scale training. Let’s see why ? Let’s study the algorithm for distributed SGD with all reduce : Distributed : The parameters of the model architecture can be distributed across several machines C1, C2, . . . …

Machine Learning

3 min read

Distributed Sync SGD
Distributed Sync SGD

Published in Better ML

·Apr 25

Wide and Deep model training

wide and deep model model size — There are two aspects of learning : memorization and generalization. Memorization comes from learning frequent co-occurrence features and exploiting the correlation available in the training data. Generalization comes from learning hidden patterns and correlation between features and labels. Wide & Deep learning is jointly training wide linear models and deep…

Mlops

4 min read

Wide and Deep model training
Wide and Deep model training

Published in Better ML

·Apr 23

DNN computation mental model

Why does it matter ? — Perf matters ! The deep neural network computation mental model should help you to do back of the envelope computations for your components. A typical training iteration contains three steps: forward pass to compute loss, backward pass to compute gradients, and optimizer step to update parameters Forward Pass, Backward Pass, Optimizer step Forward pass refers to…

Mlops

2 min read

DNN computation mental model
DNN computation mental model

Published in Better ML

·Feb 20

Model size

Why does it matter ? Model size is a key metric to drive optimizations such as : Training system design : Choosing specific sgd optimizer, network bandwidth, hardware (memory and processing). Inference system design : Model file storage optimizations, quantization, model splits for serving. How do we measure it ? While describing model size (in memory units such as GB), we…

Mlops Without Much Ops

2 min read

Model size
Model size
Jaideep Ray

Jaideep Ray

Engineer | ML lifecycle | ML Evaluation | https://www.linkedin.com/in/jaideepray/

Following
  • ODSC - Open Data Science

    ODSC - Open Data Science

  • Robbie Allen

    Robbie Allen

  • Adam Votava

    Adam Votava

  • Manning Publications

    Manning Publications

  • Cassie Kozyrkov

    Cassie Kozyrkov

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Knowable