Artem Ryblov’s Data Science Weekly

Reliable Machine Learning: Applying SRE Principles to ML in Production by Cathy Chen, Niall Richard Murphy, Kranti Parisa, D. Sculley, and Todd Underwood

The book shows data scientists, software engineers, SREs, product managers, and business owners how to run ML reliably, effectively, and accountably within an organization — from model monitoring in production to running a well-tuned model development team.

By applying an SRE mindset to machine learning, the authors and featured guest contributors work through the full ML lifecycle: data collection and analysis, training pipelines, build and validation, quality and performance evaluation, defining and measuring SLOs, launch, and monitoring and feedback loops.

Topics include:
• The ML lifecycle — from data collection through launch and feedback loops
• Data as liability — sensitivity, reliability, durability, version control, and privacy of ML data
• SLOs for ML — defining and measuring service-level objectives for models
• Monitoring and observability — catching model and data drift in production

408 pages, published by O'Reilly Media, October 2022.

Link: Book

Navigational hashtags: #armknowledgesharing #armbooks
General hashtags: #machinelearning #mlsystemdesign

@data_science_weekly

A Short Note on P-Value Hacking by Nassim Nicholas Taleb

The paper works out the exact probability distribution of p-values across repeated trials of the same phenomenon, and shows that an observed p-value of .02 can correspond to a "true" p-value above .1.

P-value hacking is modeled as picking the minimum p-value among m independent tests, which can land considerably lower than the true p-value even in a single trial, owing to the extreme skewness of the underlying meta-distribution. Taleb derives this metadistribution exactly, for small samples (2 < n ≤ 30) and for the large-sample limit, and shows p-values stay highly skewed and volatile regardless of sample size: about 75% of realizations of a true p-value of .05 read below .05, and 60% of a true p-value of .12 read below .05 as well.

Topics include:
• Metadistribution of p-values — exact PDF for small samples and the large-n limit
• P-value hacking — distribution of the minimum p-value across m trials
• Inverse power of a test — how reliable power estimates are as a measure
• Application — what this means for the 5% cutoff and for replication studies

Four pages, free on arXiv, first posted 2015 and revised through January 2018.

Link: Paper

Navigational hashtags: #armknowledgesharing #armarticles
General hashtags: #statistics #math

@data_science_weekly

SQLBolt — Learn SQL with simple, interactive exercises

Every lesson runs a database in the browser, so you can write your first query without installing anything.

SQL lets both technical and non-technical users query, manipulate, and transform data from a relational database. SQLBolt teaches the common standard shared by SQLite, MySQL, Postgres, Oracle and Microsoft SQL Server.

Most people come to SQL to work with a database that already exists, so the lessons start with the parts of a query. Later ones cover altering a table or schema and creating tables from scratch. Each lesson introduces one concept and ends with an exercise.

Table of contents:
• Lessons 1–4 — SELECT queries 101, queries with constraints, filtering and sorting query results
• Lessons 6–8 — multi-table queries with JOINs, OUTER JOINs, a short note on NULLs
• Lessons 9–12 — queries with expressions, queries with aggregates, order of execution of a query
• Lessons 13–15 — inserting, updating and deleting rows
• Lessons 16–18 — creating, altering and dropping tables

Free, and no account required.

Link: Site

Navigational hashtags: #armknowledgesharing #armsites #armtutorials
General hashtags: #sql #databases #interviewpreparation

@data_science_weekly

Machine Learning by Neetcode

Solve ML problems from scratch, from gradient descent to a working GPT clone. No ML background needed, just basic Python. Each problem includes animations, hints, and videos.

Link: Site

Navigational hashtags: #armknowledgesharing #armtutorials
General hashtags: #ml #dl #machinelearning #deeplearning

@data_science_weekly

CS 224V Conversational Virtual Assistants with Deep Learning by Stanford University

Generative AI, and in particular Large Language Models (LLMs), has the potential of assisting knowledge workers, scholars, and researchers in all areas from journalism, history, education, law, medicine, science, and engineering.

This course will focus on the general principles and the latest research on methodologies and tools that can be applied to all domains.

This is a project-oriented course, where students will gain hands-on experience in either methodology research or applying the concepts to create useful assistants for a domain of their choice.

Topics include:
- How to do RAG (retrieval augmented generation) without hallucination?
- How to retrieve information from databases and knowledge graphs using natural language?
- How to perform literature search and write long review articles?
- How to create a non-hallucinating task agent?
- How to answer questions on sets of long documents?
- How to extract a knowledge graph from a large set of documents?
- How to improve reasoning using Satisfiability Modulo Theories (SMT)?
- How to perform complext tasks reliably using the concept of computational thinking?

Link: Course

Navigational hashtags: #armknowledgesharing #armcourses
General hashtags: #llm #genai #rag

A Visual Guide to Quantization

As their name suggests, Large Language Models (LLMs) are often too large to run on consumer hardware. These models may exceed billions of parameters and generally need GPUs with large amounts of VRAM to speed up inference.

As such, more and more research has been focused on making these models smaller through improved training, adapters, etc. One major technique in this field is called quantization.

In this post, author will introduce the field of quantization in the context of language modeling and explore concepts one by one to develop an intuition about the field. He will explain various methodologies, use cases, and the principles behind quantization.

In this visual guide, there are more than 50 custom visuals to help you develop an intuition about quantization!

Link: Site

Navigational hashtags: #armknowledgesharing #armtutorials
General hashtags: #llm #quantization

@data_science_weekly

📊 Most cited sources in A/B Testing

A hand-curated leaderboard from Ron Kohavi — the researcher behind much of the modern A/B testing literature (ex-Microsoft, Amazon, Airbnb) — ranking the most-cited work in A/B testing / Online Controlled Experiments by citations per year, with a strict cutoff of 10+ cites/yr. Scoped deliberately to controlled experiments, not causal inference in general. Each paper's citation count is tracked over time (snapshots back to 2022), with newcomers and fast risers flagged.

🏆 The top of the list — start here:
• Controlled experiments on the web: survey and practical guide — Kohavi, Longbotham, Sommerfield, Henne (2009) — the classic, ~62 cites/yr
• Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing — Kohavi, Tang, Xu (2020) — the field's standard reference book (57k+ ResearchGate reads)
• Online Controlled Experiments and A/B Testing — Kohavi, Longbotham (2016)
• Experimentation and Start-up Performance: Evidence from A/B Testing — Koning, Hasan, Chatterji (2022)

🔬 Deeper cuts worth knowing:
• A causal test of the strength of weak ties — Rajkumar, Saint-Jacques, Bojinov, Brynjolfsson, Aral (2022)
• Exact p-Values for Network Interference — Athey, Eckles, Imbens (2018)
• Design and Analysis of Experiments in Networks: Reducing Bias from Interference — Eckles, Karrer, Ugander (2017)
• The surrogate index: combining short-term proxies to estimate long-term effects — Athey, Chetty, Imbens, Kang (2019)
• Design and analysis of switchback experiments — Bojinov, Simchi-Levi, Zhao (2022)
• Online controlled experiments at large scale — Kohavi, Deng, Frasca, Walker, Xu, Pohlmann (2013)

Link: Google Spreadsheet

Navigational hashtags: #armknowledgesharing #armtutorials
General hashtags: #abtesting #experimentation #datascience #causalinference #statistics

@data_science_weekly

Maths, CS & AI Compendium

Most textbooks bury good ideas under dense notation, skip the intuition, assume you already know half the material, and quickly get outdated in fast-moving fields like AI. This is an open, unconventional textbook covering maths, computing, and artificial intelligence from the ground up. Written for curious practitioners looking to deeply understand the stuff, not just survive an exam/interview.

This repo includes an MCP server that lets any AI assistant (Claude Code, Cursor, VS Code, etc.) use the compendium as a knowledge base. It requires a local clone of the repo. Comes with tools for educational purposes and example implementations.

Table of contents:

🔢 Mathematical foundations

01 — Vectors: spaces, norms, metrics, dot/cross/outer products, basis, duality
02 — Matrices: properties, linear transformations, LU/QR/SVD decompositions
03 — Calculus: derivatives, integrals, Taylor approximation, optimisation, gradient descent
04 — Statistics: sampling, central limit theorem, hypothesis testing, confidence intervals
05 — Probability: counting, distributions, Bayesian methods, information theory

🧠 Machine learning by modality

06 — Machine Learning: classical ML, deep learning, RL, distributed training
07 — Computational Linguistics: NLP, RNNs, attention, transformers, MoE, SSMs, modern LLM architectures
08 — Computer Vision: detection, segmentation, SLAM, ViTs, diffusion, flow matching, VR/AR
09 — Audio & Speech: DSP, ASR, TTS, diarisation, source separation, WaveNet, Conformer
10 — Multimodal Learning: contrastive learning, CLIP, VLMs, video tokenisation, world models
11 — Autonomous Systems: perception, robot learning, VLAs, self-driving, space robots
12 — Graph Neural Networks: geometric deep learning, graph attention, 3D equivariant nets

⚙️ Engineering & systems

13 — Computing & OS: discrete maths, architecture, concurrency, parallelism
14 — Data Structures & Algorithms: Big O, recursion, DP, graphs, sorting, binary search
15 — Production Software Engineering: Linux, Git, CI/CD, Docker, MLOps, working with coding agents
16 — SIMD & GPU Programming: C++ for ML, ARM NEON, x86 AVX, CUDA, Triton, TPUs, WebGPU
17 — AI Inference: quantisation, serving and batching, edge, speculative decoding
18 — ML Systems Design: cloud, feature stores, A/B testing, recsys/search/fraud case studies

Link: GitHub

Navigational hashtags: #armknowledgesharing #armrepo
General hashtags: #math #mathematics #ml #machinelearning #engineering #mlsystemdesign

@data_science_weekly

The Algorithms - GitHub's largest open-source algorithm library

It's not a single repo, but an entire collection: algorithms and data structures implemented in virtually every language — Python, Java, C, C++, JavaScript, Rust, Go, Julia, Fortran, Zig, Nim, and even Mojo.

Every repository is educational by design: sorting, searching, graphs, dynamic programming, mathematics, machine learning, physics. Clean, readable code under open licenses.

The scale speaks for itself: the Python repo alone has 222k stars, Java has 66k — making these some of the most popular educational projects on GitHub, period.

Who it's for:
• Learning algorithms — see the implementation in your language of choice
• Prepping for interviews — the perfect reference
• Want to break into open source — plenty of issues tagged "help wanted"

Free, open, community-maintained.

Link: GitHub

Navigational hashtags: #armknowledgesharing #armrepo
General hashtags: #programming #interviewpreparation #algorithms #github #opensource #programming #learning

@data_science_weekly

CS336: Language Modeling from Scratch by Stanford

Language models serve as the cornerstone of modern natural language processing (NLP) applications and open up a new paradigm of having a single general purpose system address a range of downstream tasks. As the field of artificial intelligence (AI), machine learning (ML), and NLP continues to grow, possessing a deep understanding of language models becomes essential for scientists and engineers alike. This course is designed to provide students with a comprehensive understanding of language models by walking them through the entire process of developing their own. Drawing inspiration from operating systems courses that create an entire operating system from scratch, we will lead students through every aspect of language model creation, including data collection and cleaning for pre-training, transformer model construction, model training, and evaluation before deployment.

Table of contents:
1. Foundations
1.1 Overview
1.2 Tokenization
1.3 PyTorch & einops
1.4 Resource Accounting
- FLOPs
- Memory
- Arithmetic Intensity
2. Transformer Architecture
2.1 Model Architectures
2.2 Hyperparameters
2.3 Attention Alternatives
2.4 Mixture of Experts (MoE)
3. Systems for Training
3.1 GPUs and TPUs
3.2 GPU Kernels
3.3 Triton
3.4 Parallelism
- Part I
- Part II
4. Scaling
4.1 Scaling Laws
- Theory
- Applications
5. Inference
6. Evaluation
7. Data
7.1 Data Sources
7.2 Datasets
7.3 Filtering
7.4 Deduplication
7.5 Data Mixing
7.6 Synthetic Data
8. Post-Training
8.1 Supervised Fine-Tuning (SFT)
8.2 RLHF / Mid-Training
8.3 RLVR
9. Alignment
9.1 Multimodal Alignment
10. Guest Lectures
10.1 Daniel Selsam
10.2 Dan Fu

Links:
• Site
• YouTube

Navigational hashtags: #armknowledgesharing #armcourses
General hashtags: #llm #largelanguagemodels #stanford

@data_science_weekly

LLM Engineering Essentials

Gain the skills to build LLM-powered services that work. Master LLM APIs and self-hosted LLMs as you code, experiment, and create a platform for custom AI-powered NPCs.

1. Understand the fundamentals of LLM APIs and workflows to create a chatbot based on your favorite fantasy character
2. Learn to work with self-hosted LLMs, encoders, and vector stores, and build a RAG system
3. Explore monitoring tools like Prometheus and Grafana. Optimize and fine-tune your LLM-powered service

What you will learn:
• Prompting Strategies
• Inference Parameters
• LLM Workflows
• Agents
• Reasoning
• RAG
• Hugging Face Ecosystem
• Inference Engines
• Quantization
• Evaluation
• Monitoring
• Fine-Tuning
• Reinforcement Learning
• Scaling

Link: Course

Navigational hashtags: #armknowledgesharing #armcourses
General hashtags: #llm #largelanguagemodels #huggingface #agents #rag #promptengineering

@data_science_weekly

Causal Inference: What If by Jamie Robins and Miguel Hernán

This book will be helpful to anyone interested in causal inference, including epidemiologists, statisticians, psychologists, economists, sociologists, political scientists, computer scientists…

The book is divided in three parts of increasing difficulty:
(1) causal inference without models
(2) causal inference with models
(3) causal inference from complex longitudinal data

Link: Book

Navigational hashtags: #armknowledgesharing #armbooks
General hashtags: #causalinference #causal #statistics #stats #math

@data_science_weekly

Recommenders

Recommenders objective is to assist researchers, developers and enthusiasts in prototyping, experimenting with and bringing to production a range of classic and state-of-the-art recommendation systems.

Recommenders is a project under the Linux Foundation of AI and Data.

This repository contains examples and best practices for building recommendation systems, provided as Jupyter notebooks. The examples detail our learnings on five key tasks:
• Prepare Data: Preparing and loading data for each recommendation algorithm.
• Model: Building models using various classical and deep learning recommendation algorithms such as Alternating Least Squares (ALS) or eXtreme Deep Factorization Machines (xDeepFM).
• Evaluate: Evaluating algorithms with offline metrics.
• Model Select and Optimize: Tuning and optimizing hyperparameters for recommendation models.
• Operationalize: Operationalizing models in a production environment on Azure.

Several utilities are provided in recommenders to support common tasks such as loading datasets in the format expected by different algorithms, evaluating model outputs, and splitting training/test data. Implementations of several state-of-the-art algorithms are included for self-study and customization in your own applications. See the Recommenders documentation.

For a more detailed overview of the repository, please see the documents on the wiki page.

For some of the practical scenarios where recommendation systems have been applied, see scenarios.

Link: GitHub

Navigational hashtags: #armknowledgesharing #armrepo
General hashtags: #recsys #recommenders #recommendersystems #recommendation

@data_science_weekly

System Design Interview – Step By Step Guide

Topics mentioned in the video:
• Stages of a typical system design interview: functional requirements (API), non-functional requirements, high-level design, detailed design, bottlenecks and tradeoffs.
• Why requirements clarification is so important.
• What questions to ask the interviewer.
• How to design API.
• Non-functional requirements to consider: scalability, performance, availability, consistency, cost.
• How to define a data model.
• How to scale a SQL database.
• Apache Cassandra high-level architecture.
• Data processing concepts: checkpointing, partitioning, in-memory aggregation, deduplication cache, dead-letter queue, embedded database, state management.
• Data ingestion pipeline concepts: blocking vs non-blocking I/O, buffering and batching, timeouts, retries, exponential backoff and jitter, circuit breaker pattern, software vs hardware load balancing, load balancing algorithms, DNS, health checking and high availability of load balancers, partition strategy, hot partitions, client-side and server-side service discovery, single leader replication and leaderless replication, textual vs binary data formats.
• Data retrieval pipeline concepts: time-series data, data rollup, hot storage, cold storage.
• Types of performance testing: load testing, stress testing, soak testing.
• Health monitoring.
• Audit systems.

Link: YouTube

Navigational hashtags: #armknowledgesharing #armtutorials
General hashtags: #systemdesign

@data_science_weekly

Statistical Rethinking by Richard McElreath

The unfortunate truth about data is that nothing much can be done with it, until we say what caused it.

This course teaches data analysis, but it focuses on scientific models:
• Conceptual, causal models and precise questions about those models
• Bayesian data analysis to connect scientific models to evidence
• Powerful computational tools for coping with high-dimension, imperfect data of the kind that biologists and social scientists face.

Table of Contents
Week 01 — Science Before Statistics / Garden of Forking Data
Week 02 — Geocentric Models / Categories and Curves
Week 03 — Elemental Confounds / Good and Bad Controls
Week 04 — Overfitting / MCMC
Week 05 — Modeling Events / Counts and Confounds
Week 06 — Ordered Categories / Multilevel Models
Week 07 — Multilevel Adventures / Correlated Features
Week 08 — Social Networks / Gaussian Processes
Week 09 — Measurement / Missing Data
Week 10 — Generalized Linear Madness / Horoscopes

Links:
• GitHub
• Book

Navigational hashtags: #armknowledgesharing #armcourses
General hashtags: #math #statistics #stat

@data_science_weekly

ML Systems Design Interview Guide by Patrick Halina

One of the trickiest interview rounds for ML practitioners is ML systems design. If you’re applying to be a Data Scientist, ML Engineer or ML Manager at a big tech company, you’ll probably face an ML Systems design question.

Author recently tackled this question at a few big tech companies on his way to becoming a Staff ML Engineer at Pinterest. In this article he's going to talk about how to approach ML Systems Design interviews, core concepts to know and I’ll provide links to some of the resources he used.

Link: Site

Navigational hashtags: #armknowledgesharing #armarticles
General hashtags: #ml #mlsystemdesign #mlsysdes

@data_science_weekly

Kaggle Tips for Feature Engineering and Selection

Gilberto Titericz explores essential techniques for improving model accuracy, emphasizing exploratory data analysis and human intuition in feature engineering.

Learn strategies for handling high-cardinality categorical features, missing values, and combining variables to maximize performance in competitions.

Link: YouTube

Navigational hashtags: #armknowledgesharing #armyoutube
General hashtags: #ml #featureselection #featureselection #machinelearning

@data_science_weekly

Claude 101

A collection of free guides to master Claude. Yes, it’s free.

🟢 Level 1: Beginner
• Claude For Dummies
• Claude Setup
• Claude Certified

🔵 Level 2: Intermediate
• Claude Cowork
• Claude for Teams
• Claude Design
• Claude Cowork + Projects
• Claude for Slides
• Claude Skills

🟠 Level 3: Advanced
• Avoid Sycophancy
• Claude to Sound Like You
• Stop Hitting Claude Limits
• Stop Prompting
• Stop Sounding Like AI

🔴 Level 4: Expert
• Claude Code
• Claude Computer

Navigational hashtags: #armknowledgesharing #armtutorials
General hashtags: #claudeai #aitools #ailearning #promptengineering #prompts

@data_science_weekly

Practitioner’s Guide to Statistical Tests

The two most essential things in A/B tests are the design of the experiments and accurate analysis of the experiments’ results.

In this article, authors will stick to the most common design and compare various statistical analysis procedures, from the very standard t-test and Mann-Whitney test to state-of-the-art approaches like the reweighted bootstrap.

After reading this article, you will learn how to choose the right statistical test from the many available and run it on your own data.

Links:
• Post
• The code to reproduce everything
• The “sandbox” notebook you can use to play with the tests

Navigational hashtags: #armknowledgesharing #armarticles
General hashtags: #ab #abtesting #statistics

@data_science_weekly

The Complete Mathematics of Neural Networks and Deep Learning by Adam Dhalla

A complete guide to the mathematics behind neural networks and backpropagation.

In this lecture, Adam aims to explain the mathematical phenomena — a combination of linear algebra and optimization — that underlie the most important algorithm in data science today: the feed forward neural network.

Through a plethora of examples, geometrical intuitions, and not-too-tedious proofs, he guides you from understanding how backpropagation works in single neurons to entire networks, and why we need backpropagation anyways.

It's a long lecture, so the author encourages you to segment out your learning time — get a notebook and take some notes, and see if you can prove the theorems yourself.

Adam Dhalla is a high school student from Vancouver, BC, interested in how we can use algorithms from computer science to gain intuition about natural systems and environments.

Link: YouTube

Navigational hashtags: #armknowledgesharing #armcourses
General hashtags: #deeplearning #math #neuralnetworks #nn #optimization #backpropagation

@data_science_weekly