DA @dataxplore
Showing posts older than #1886 · Back to latest
You need to understand what is happening on the host side
The NVIDIA driver in the kernel provides GPU access through device files: /dev/nvidia0, /dev/nvidiactl, and so on
Any application communicates with the GPU exactly through these device files
PyTorch does not directly access the driver
It works through CUDA Runtime (libcudart.so) which is a high-level API that handles allocations, kernel launches, and synchronization
This runtime library is inside your container
The entire stack looks like this:
PyTorch → CUDA Runtime → CUDA Driver → /dev/nvidia0 → kernel → GPU
Runtime lives in the container
Driver lives on the host
Look at the container launch:
Containerd → containerd-shim → OCI-runtime (runc) → container
But if the driver is on the host and the runtime is in the container, how does the application access everything at once?
Answer: OCI hooks
The OCI specification defines hooks (code) that runs at different stages of the container's lifecycle:
prestart/createRuntime
createContainer
startContainer
poststart
poststop
NVIDIA uses these hooks to add GPU support
Before the container starts, the hook does the following:
1. Mounts GPU devices (/dev/nvidia*)
2. Places driver libraries from the host into the container
3. Sets the necessary environment variables
4. Configures device cgroups
Your application has not even started yet
All of this is handled by NVIDIA Container Toolkit. It intercepts container creation and carefully inserts everything needed for GPU operation
Your image remains normal.
GPU capabilities appear at runtime.
1. What is DocETL?
It is a tool for creating and running data pipelines, especially well-suited for complex document processing.
It includes:
- interactive UI playground
- Python package for running pipelines in production
2. DocWrangler
DocWrangler helps gradually build the pipeline:
- trying different prompts and viewing results in real-time
- building the pipeline step by step
- exporting the final configuration for production
3. Python package DocETL
It is used for running pipelines in production. In the example, a pipeline is created that analyzes medical transcripts, identifies drug names, normalizes similar names, and generates summaries of side effects and areas of application.
In such a situation, we can calculate the agreement coefficient, The Agreement. It shows the proportion of matching labels between annotators.
In our case, the average pairwise agreement is 83.9%, which is quite good. However, there is a catch: this coefficient, like accuracy in the case of binary classification, can be misleading in the case of class imbalance. In our dataset, more than 70% of the labels fall on two classes — "Neutral" and "Confusion." Let's use other statistical coefficients to ensure high consistency:
📌 Cohen's Kappa (Cohen's Kappa) : pairwise coefficient. It evaluates the normalized agreement between two annotators.
How to interpret the results:
<0.6 : poor consistency.
0.6...0.8 : good consistency, can be used in practical tasks.
>0.8 : very high consistency.
📌 Fleiss's Kappa (Fleiss's Kappa) — suitable for evaluating consistency between several (more than two) annotators.
In our case, Cohen's Kappa varies from 0.7 to 0.84, indicating high consistency. For additional verification, we took several hundred random examples from the dataset and manually assigned labels. It turned out that in 44% of cases, our labels and annotator labels differed that is, annotators on average consistently assign incorrect labels. That is why even high consistency scores do not guarantee high-quality data annotation.
It provides a whole ecosystem of guides, demos, and learning tracks designed for developers who want to build production AI applications on a reliable data infrastructure.
Two especially useful resources to start with:
1. Basics of vector search in MongoDB: understand how semantic search really works and build a working search pipeline.
2. Building memory agents with MongoDB, Fireworks AI, and LangChain: train an agent to remember past interactions and pull context directly from your operational data.
What makes this library even more interesting is that the content is not limited to AI only. It covers all the supporting components needed to run AI in production, for example:
↳ Storage architectures for AI applications
↳ High-throughput indexing and retrieval
↳ Caching to speed up inference
↳ Best security practices for AI data pipelines
↳ End-to-end examples with real datasets
The flagship is GigaChat 3 Ultra-Preview a 702B-MoEmodel, fully trained from scratch on a corpus of 14 trillion tokens. This is not an adaptation or fine-tuning of foreign weights: the model has its own dataset, its own synthetic pipeline, and a redesigned architecture. On Russian-language and STEM benchmarks, Ultra-Preview confidently outperforms Russian open source counterparts, as well as surpasses DeepSeek V3.1.
Context memory up to 128k tokens.
Also available in open source is the Lightning version a compact 10B-MoE model that competes in inference speed with Qwen3-1.7B and approaches the quality of dense models around 8B. GigaAM-v3 is also open — a set of five models for working with audio. It recognizes speech excellently — showing a −50% WER compared to Whisper-large-v3.
The open GigaChat lineup effectively forms a new open ecosystem for development, generation, and automation and does so as an independent architecture, not a continuation of someone else’s solutions.
This is the first model of this scale where Asynchronous RL is not an experiment but the foundation of training. As a result, the model demonstrates strong performance in mathematics, coding, and reasoning.
The model's focus is on long action chains and agent tasks, not just text generation.
- The model shows top results for its size in mathematics, coding, and reasoning.
- Training was conducted on 512×H200 for about ~2 months.
- Used own stack: PRIME-RL, Verifiers, Environments Hub, and sandbox infrastructure.
- Everything is open: code, environments, tools.
🤖 @DataXplore for Data Science, ML, DL, NN & Big Data Analysis
1️⃣ Take a frozen base model (DINO, CLIP, etc.)
2️⃣ Observe the gradients it produces on real images
3️⃣ Generate synthetic images so that the gradients match
4️⃣ Train a linear classifier - and it works better than training ining ng on thng on the original data
Why this is useful?
— works across models (generated for DINO → works great on CLIP)
— especially strong on fine-grained classifications, where micro-details matter
— helps to see what the model is really looking at: serious correlations, similar clusters, embedding space structure
.fit() in scikit-learn.1️⃣ Linear Algebra:: Language for describing data and models (vectors, matrices, tensors).
2️⃣ Calculus: The toolkit for training and optimization (derivatives, gradients).
3️⃣ Probability Theory: The framework for assessing uncertainty.
From understanding how Backpropagation and SGD work, TO the causes of gradient explosion and choosing the loss function.
Miles offers what is called "True On-Policy." Previously, there was often a divergence between training and inference. Now, thanks to an infrastructural approach, LMSYS has achieved zero divergence. This was made possible by using Flash Attention 3, the DeepGEMM library, and kernels from Thinking Machines Lab, working in conjunction with torch.compile.
The second feature is the use of speculative decoding. Usually, in RL, the draft model is frozen, which prevents it from following the target model's policy. LMSYS added online training of the draft model.
Test results are positive: generation speedup of more than 25%, especially in the later stages of training.
STABILITY.
For enterprise, memory is money. Miles include mechanisms that prevent system crashes due to non-critical OOM errors and fix excessive memory consumption in FSDP.
The project promises support for multimodal training, compatibility with SGLang v2 and extended speculative decoding.
The project setting was truly "red": AMD Instinct GPUs, AMD Pensando network interfaces, and the ROCm software stack.
ZAYA1 turned out to be quite interesting. It has 8.3 billion total parameters, of which only 800 million are active.
Despite its compactness, it performs well in tests. In reasoning, mathematics, and programming, ZAYA1 outperformed Llama-3-8B and OLMoE. And overall, it stands alongside Qwen3-4B and Google's Gemma3-12B.
Training took place on an IBM Cloud cluster, where the model processed 14 trillion tokens. But it’s not just about the hardware; architectural innovations were used in the pipeline:
A new attention mechanism - Compressed Convolutional Attention. It uses convolutions inside the attention block, reducing computational and memory load.
Redesigned MoE router. Instead of the standard linear router, ZAYA1 uses a complex sequence of operations that make the "experts" inside the neural network specialize much better.
Residual Scaling. Trainable scalar gates were added to the residual stream at the outputs of each block, allowing the model to control the degree of forgetting.
Suppose we have an emotion classifier and want to boost its metrics.
☞ Change the training approach : experiment with architecture, pretraining, optimizers.
☞ Work with the data : check the dataset, review the labeling, find noise and errors.
☞ Or completely abandon the classic ML model and try feeding everything to an LLM, hoping for the model's zero/few-shot capabilities.
Most engineers choose the first two options. But, as practice shows, it is precisely the search for errors in datasets and improving data quality that gives the most noticeable gain.
🤖 Data Science, ML & Big Data with @DataXplore
- The DA3 model outperforms previous versions in depth estimation.
- Supports monocular and multi-view depth estimation.
- High-accuracy pose estimation.
- User-friendly interface and export options to various formats.
- Specialized models for metric depth evaluation.
- Uses a simple transformer and a unique depth representation, enabling high performance in depth and pose estimation.
1️⃣ Understand Why its so fast?
Traditional graph databases store connections as linked nodes and traverse them one hop at a time.
But there is a PROBLEM: When you query connections, the database goes through nodes and edges, literally following a map. For huge knowledge graphs powering AI agents, this creates a serious bottleneck.
2️⃣ What if you represent the entire graph as a mathematical structure?
Sparse Matrices: A sparse matrix stores only existing connections. No extra space, no unnecessary data.
And here is the breakthrough:
When your graph is represented as a sparse matrix, you can perform queries using linear algebra instead of traversal. Queries turn into mathematical operations, not step-by-step node transitions.
Mathematics is faster than traversal. Much faster.
Plus, sparse matrices allow incredibly efficient memory usage. You store only what exists, so you can keep huge knowledge graphs in memory without burning resources.
3️⃣ Why not just use Vector Search?
Vector search is fast, but it only captures naive similarity. It can find patterns but does not see structure.
Graphs capture subtle relationships between entities. This ensures the context you bring up for the agent is accurate and relevant, not just similar.
4️⃣ What FalkorDB gives?
↳ Ultra-fast multi-tenant graph database
↳ Efficient storage through sparse matrices
↳ Compatibility with OpenCypher (the same query language as Neo4j)
↳ Specifically designed for LLM applications and agent memory
↳ Runs on top of Redis for easy deployment
1️⃣ Using OpenRouter, the request is sent to several models (currently GPT-5.1, Gemini 3 Pro, Sonnet 4.5, and Grok 4). Each writes its own version of the answer.
2️⃣ Then all models are shown each other's anonymous answers, they check and rank them, leaving their comments.
3️⃣ All this is eventually sent as context to the "LLM chairman," who then compiles them into single Final Answer.