RIML Lab

🔐 ML Security Journal Club

✅ This Week's Presentation:

🔹 Title: Exploiting LLM Quantization

🔸 Presenter: Arian Komaei

🌀 Abstract:

This paper studies the security implications of quantization in large language models (LLMs). While quantization is widely used to reduce memory usage and enable deployment on commodity hardware, its adverse effects from a security perspective have been largely unexplored. The authors reveal that widely used quantization methods can be exploited to produce a harmful quantized LLM, even though the full-precision counterpart appears benign, potentially tricking users into deploying the malicious quantized model. They demonstrate this threat using a three-staged attack framework: (i) first, obtaining a malicious LLM through fine-tuning on an adversarial task; (ii) next, quantizing the malicious model and calculating constraints that characterize all full-precision models that map to the same quantized model; (iii) finally, using projected gradient descent to tune out the poisoned behavior from the full-precision model while ensuring that its weights satisfy the constraints computed in step (ii). This procedure results in an LLM that exhibits benign behavior in full precision but when quantized, it follows the adversarial behavior injected in step (i). Experiments demonstrate the feasibility and severity of such an attack across three diverse scenarios: vulnerable code generation, content injection, and over-refusal attack. In practice, the adversary could host the resulting full-precision model on an LLM community hub such as Hugging Face, exposing millions of users to the threat of deploying its malicious quantized version on their devices.

📄 Paper: Exploiting LLM Quantization

Session Details:
📅 Date: Sunday یک‌شنبه
🕒 Time: 6:00 – 7:00 PM
🌐 Location: Online at vc.sharif.edu/ch/rohban

We look forward to your participation! ✌️

we are looking for Teaching Assistants to join the Multi-Agent Reinforcement Learning (MARL) course team for Fall 2026.

1. Homework TAs
Responsibilities
- Assist in designing, reviewing, and grading homework assignments.
- Prepare and review homework solutions.
Requirements
- Successful completion of both Reinforcement Learning and Game Theory courses.

2. Project TAs
Responsibilities
- Guide students throughout their research projects in MARL.
- Select relevant topics, papers, and materials for different project phases.
- Help students read, understand, and discuss research papers.
- Support students in developing and refining research ideas.
Requirements
- Strong interest in research in Multi-Agent Reinforcement Learning.
- Successful completion of an RL course.
- Previous research experience in RL or related areas.
- Familiarity with Game Theory and/or agentic systems.
- Mandatory attendance at course lectures throughout the semester and continuous engagement with the course material.

3. Quiz TAs
Responsibilities
- Design and prepare quizzes based on the course material.
Requirements
- Strong knowledge of Reinforcement Learning and Game Theory.
- Successful completion of an RL course.
- Mandatory attendance at course lectures throughout the semester and continuous engagement with the course material.
If you are interested, please fill out the application form.
If you have any questions, feel free to contact me.

For course updates and announcements, join the course channel.

🔐 ML Security Journal Club

✅ This Week's Presentation:

🔹 Title: Robust Machine Unlearning for Quantized Neural Networks via Adaptive Gradient Reweighting with Similar Labels

🔸 Presenter: Arian Komaei

🌀 Abstract:

This paper studies machine unlearning (MU) in quantized neural networks, where conventional unlearning methods designed for full-precision models can become significantly less effective. The authors identify two key challenges introduced or amplified by quantization: noise from randomly relabeling forgotten samples and gradient imbalance between forgotten and retained data. Because quantized models operate in a constrained, discrete parameter space, these effects can lead to unstable updates and poorer approximation of a model retrained from scratch.
To address these issues, the authors propose Q-MUL, a quantization-aware unlearning framework with two main components. Similar Labels (SL) replaces random labels for forgotten samples with alternative labels selected according to the model's current output distribution, reducing disruptive noise during unlearning. Adaptive Gradient Reweighting (AGR) dynamically balances the contributions of forgotten and retained data according to their gradient norms.
Experiments on datasets including CIFAR-10, CIFAR-100, SVHN, and Tiny-ImageNet, using quantized ResNet-18 and MobileNetV2, show that Q-MUL generally produces models whose forgetting, retention, test accuracy, and membership-inference behavior are closer to those of models retrained from scratch than existing approximate-unlearning baselines.

📄 Paper: Robust Machine Unlearning for Quantized Neural Networks via Adaptive Gradient Reweighting with Similar Labels

Session Details:
📅 Date: Sunday یک‌شنبه
🕒 Time: 6:00 – 7:00 PM
🌐 Location: Online at vc.sharif.edu/ch/rohban

We look forward to your participation! ✌️

🔘 Open Research Position: Machine Unlearning × Model Quantization
We are looking for motivated students to join a research project exploring the intersection of machine unlearning and neural network quantization.
🔍 Project Description
Machine unlearning aims to remove the influence of specific training data or learned concepts from a trained model without retraining it from scratch. At the same time, quantization is widely used to compress neural networks and enable efficient deployment.
This project investigates the interaction between unlearning and quantization, including questions such as whether quantization can weaken, reverse, or otherwise affect the effectiveness of unlearning methods, and how unlearning techniques can be designed to remain robust under low-precision deployment.
The project is motivated by recent work including:
📄 Catastrophic Failure of LLM Unlearning via Quantization
https://arxiv.org/abs/2410.16454
📄 RAZOR: Ratio-Aware Layer Editing for Targeted Unlearning in Vision Transformers and Diffusion Models
http://arxiv.org/abs/2603.14819
🔹 Requirements
Strong Python programming skills
Solid understanding of deep learning
Hands-on experience with PyTorch
Familiarity with model quantization and/or machine unlearning
Ability to read, understand, and implement ideas from recent research papers
Experience with LLMs, Vision Transformers, diffusion models, or model compression is a plus.
📌 How to Apply
If you are interested, send your CV to @kp_gfe on Telegram.

🔐 ML Security Journal Club
✅ This Week's Presentation:
🔹 Title: Catastrophic Failure of LLM Unlearning via Quantization
🔸 Presenter: Arian Komaei
🌀 Abstract:
The paper identifies a critical limitation in current large language model (LLM) unlearning methods: knowledge that appears to be successfully forgotten in full-precision models can be recovered after quantization. The authors show that existing unlearning approaches often rely on small weight changes to preserve model utility, causing the original and unlearned model weights to be mapped to similar values during low-bit quantization. Extensive experiments across multiple unlearning and quantization methods demonstrate that, for utility-preserving unlearning methods, models retain an average of 21% of the intended forgotten knowledge in full precision, which increases dramatically to 83% after 4-bit quantization. To mitigate this issue, the authors propose SURE, a saliency-based unlearning method with a large learning rate that selectively updates influential model modules, improving robustness against knowledge recovery while aiming to preserve model utility.
📄 Paper: Catastrophic Failure of LLM Unlearning via Quantization
Session Details:
* 📅 Date: Sunday یک‌شنبه
* 🕒 Time: 6:00 - 7:00 PM
* 🌐 Location: Online at vc.sharif.edu/ch/rohban
We look forward to your participation! ✌️

📢 Join the IABI TA Team!

🩻 The Intelligent Analysis of Biomedical Images (IABI) course is looking for motivated Bachelor’s and Master’s students to join its Teaching Assistant team.

🎯 If you’re interested in biomedical image analysis, enjoy helping others learn, and want to gain valuable teaching experience, we encourage you to apply!

📝 Apply here
⏳ Application deadline: 15 September 2026

Call for Research Assistants: A Project on Abductive Reasoning in LLMs

If you are familiar with LLMs, you are invited to join our research project as a research assistant. This project focuses on abductive reasoning in LLMs.
This project focuses on abductive reasoning in LLMs and aims at preparing submission for ICLR.
For an introduction to the topic, you can read:
Wiring the ‘Why’: A Unified Taxonomy and Survey of Abductive Reasoning in LLMs
If you are interested, please complete the following form:
Registration Form

🔐 ML Security Journal Club

✅ This Week's Presentation:

🔹 Title: How Jailbreaks Evade, but Do Not Erase, LLM Safety Mechanisms

🔸 Presenter: Javad Hezareh

🌀 Abstract:
This paper investigates the internal mechanisms of Large Language Models (LLMs) during successful jailbreak attacks. The authors provide mechanistic evidence that jailbreaks do not comprehensively eliminate an LLM's safety features; instead, they selectively suppress specific components to bypass refusal mechanisms, leaving other robust internal safety representations intact. To validate the utility of these mechanistic insights, the authors developed a training-free harmful-content detector. By reading the robust internal activations without any model training, this detector achieves competitive aggregate performance and strong adversarial robustness on safety-eval benchmarks.

📄 Paper: Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models

Session Details:

* 📅 Date: Tuesday, Aug 11
* 🕒 Time: 14:00 - 15:00
* 🌐 Location: Online at vc.sharif.edu/ch/rohban

We look forward to your participation! ✌️

📈 Generative Modeling — Scaling, Multimodality & End-to-End Generation

✅ This Week's Presentation:

🔹 Title: Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation
🔸 Presenter: Amir Qeysarbeigi

🌀 Abstract:
Modern generative models typically handle multimodal distributions by factorizing the generation process into multiple steps, as in autoregressive and diffusion models. While this enables high-quality generation, it creates a mismatch between training and inference and prevents fully end-to-end generation. In this work, we introduce Explorative Modeling (XM), a new paradigm that instead factorizes the training process by exploring multiple candidate generations and training on the best-matching one. This exploration increases generative expressivity, allowing models to capture more modes of multimodal distributions without relying solely on generation factorization.

The paper demonstrates that exploration acts as a third pretraining scaling axis, alongside model parameters and data, improving efficiency across image, video, and language generation. Increasing exploration improves FLOP, sample, and parameter efficiency, with gains that become larger as models and datasets scale. Furthermore, by moving the burden of multimodality from inference-time generation steps to training-time exploration, Explorative Modeling enables end-to-end generative models that can achieve performance comparable to diffusion-based approaches with dramatically fewer inference steps.

📄 Article: *Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation*

Session Details:
We will first review the limitations of conventional reconstructive generative models and introduce the concept of generative expressivity as a fundamental bottleneck in multimodal generation. Then, we will explore how Explorative Modeling replaces generation factorization with training-time exploration, including the Forward and Reverse XM formulations. Finally, we will examine how exploration serves as a new scaling axis, improves efficiency across multiple modalities, and enables end-to-end generation with substantially fewer inference steps.

- 📅 Date: Monday (دوشنبه)
- 🕒 Time: 17:00 - 18:00
- 🌐 **Location: https://vc.sharif.edu/rohban** (Online only)

We look forward to your participation! ✌️

📢 Research Collaboration in Quantitative Finance at RIML

We are seeking motivated students interested in quantitative finance, stochastic modeling, machine learning, and portfolio optimization. Selected researchers will work under the supervision of Dr. Rohban and collaborate with international professors and researchers affiliated with the University of Manchester, the Alan Turing Institute, Virginia Tech, and the Technical University of Munich

🔬 The following four research directions are available:

1️⃣ Reinforcement Learning in High-Frequency Market Making
Study the trade-off between time discretization, learning accuracy, and sample complexity in single- and multi-agent market making, including convergence to continuous-time optimal policies and Nash equilibria.
🔗 Paper: https://arxiv.org/abs/2407.21025

2️⃣ Stochastic Optimal Control for Multi-Asset Market Making
Develop scalable closed-form approximations to the Hamilton–Jacobi equations of multi-asset market-making models, enabling interpretable near-optimal quotes under correlated prices and portfolio-wide inventory risk.
🔗 Paper: https://arxiv.org/abs/1810.04383

3️⃣ Diffolio: A Diffusion Model for Multivariate Probabilistic Financial Time-Series Forecasting and Portfolio Construction Model the conditional joint distribution of future asset returns using hierarchical asset-level and market-level attention, and use the generated scenarios for risk-aware portfolio construction.
🔗 Paper: https://arxiv.org/abs/2511.07014

4️⃣ Structured Filtering for Jump-Diffusion Time Series Forecasting to infer hidden market states from partially observed jump-diffusion data and produce calibrated probabilistic forecasts of continuous movements and abrupt price shocks.
🔗 Paper: https://arxiv.org/abs/2605.24548

✉️ Interested candidates are invited to send their CV to:
alirezanourimath@gmail.com

آزمایشگاه RIML تحت نظارت دکتر رهبان در حال راه‌اندازی یک ژورنال‌کلاب پیرامون یادگیری تقویتی چندعاملی (Multi-Agent RL) بر پایه‌ی کتاب Albrecht با چشم‌انداز حرکت به سمت کار پژوهشی جدی در این حوزه است. در صورت علاقه‌مندی به این مسیر، خواهشمندست این فرم را پر کنید.
*: جلسه‌های ژورنال‌کلاب از این هفته آغاز می‌شود.
در صورتی که پرسش یا ابهامی در این زمینه دارید با شناسه‌ی زیر در تلگرام ارتباط بگیرید:
@Moein_Salimi

🔐 LLM Faithfulness Journal Club

✅ This Week's Presentation:

🔹 Title: Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations

🔸 Presenter: [Farzan Rahmani](https://www.linkedin.com/in/farzan-rahmani-51128b201)

🌀 Abstract:
When large language models explain their decisions, their explanations may sound convincing—but do they faithfully reflect the model's true reasoning? This paper presents a comprehensive counterfactual analysis of self-explanation faithfulness across 75 models from 13 model families. It investigates the tradeoff between concise and comprehensive explanations, introduces two new evaluation metrics (phi-CCT and F-AUROC), and studies how explanation verbosity influences faithfulness measurements. The results reveal a clear scaling trend: larger and more capable language models consistently produce more faithful self-explanations, providing valuable insights into the relationship between model scale, explanation quality, and AI safety.

📄 Paper: [Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations](https://arxiv.org/abs/2503.13445)

Session Details:

* 📅 Date: Wednesday (چهارشنبه)
* 🕑 Time: 2:00 - 3:00 PM
* 🌐 Location: Online at http://vc.sharif.edu/ch/rohban

We look forward to your participation! ✌️

🤖 RL Journal Club

✅ This Week's Presentation:

🔹 Title: Is Reinforcement Learning Really Harder Than Bandits?
🔸 Presenter: Arshia Gharooni

🌀 Abstract:
Episodic reinforcement learning, despite having longer planning horizons, presents little additional sample complexity difficulty compared to contextual bandits, with the proposed Monotonic Value Propagation (MVP) algorithm achieving near-optimal regret bounds. The MVP algorithm utilizes a simplified, variance-aware bonus to achieve superior performance, offering an exponential improvement in horizon dependency and sample efficiency over previous state-of-the-art methods.

📄 Paper: Is Reinforcement Learning More Difficult Than Bandits? A Near-optimal Algorithm Escaping the Curse of Horizon

Session Details:

- 📅 Date: Wednesday (چهارشنبه)
- 🕒 Time: 17:00 - 18:00
- 🌐 Location: Online at http://vc.sharif.edu/ch/rohban

We look forward to your participation! ✌️

🤖 RL Journal Club

✅ This Week's Presentation:

🔹 Title: Test Time Exploration to Achieve Generalization in Zero-Shot RL
🔸 Presenter: Alireza Farajtabrizi

🌀 Abstract:
This paper studies zero-shot generalization in reinforcement learning, where an agent is trained on a set of tasks but must perform well on unseen test environments. The authors argue that standard reward-maximizing RL agents can overfit to training tasks, especially in environments where simple invariance-based methods fail. Their key insight is that exploration behavior is harder to memorize than reward-seeking behavior and can therefore generalize better.

To build on this idea, the paper introduces Explore to Generalize (ExpGen), an algorithm that combines a maximum-entropy exploration policy with an ensemble of reward-seeking agents. At test time, when the ensemble agrees on an action, the agent exploits that decision; when the ensemble is uncertain, the agent switches to the exploration policy to reach new parts of the state space. Experiments on ProcGen show strong improvements on challenging tasks such as Maze and Heist, setting new state-of-the-art results in several zero-shot RL settings.

📄 Paper: Explore to Generalize in Zero-Shot RL (NeurIPS 2023)

Session Details:

* 📅 Date: Tuesday سه‌شنبه
* 🕒 Time: 15:30 - 16:30
* 🌐 Location: Online at vc.sharif.edu/ch/rohban (http://vc.sharif.edu/ch/rohban)

We look forward to your participation! ✌️

🚀 Open Research Position: Visual Reasoning in Large Vision-Language Models (LVLMs)

We are looking for motivated students to join our research on visual reasoning in Large Vision-Language Models (LVLMs) at RIML Lab.

🔍 Project Description

Large Vision-Language Models have achieved remarkable performance across a wide range of multimodal tasks. However, their ability to perform complex visual reasoning remains an open challenge. This research focuses on understanding, evaluating, and improving the reasoning capabilities of LVLMs, including multi-step reasoning, visual grounding, and reasoning over complex visual scenes.

📄 Relevant Papers

Question Aware Vision Transformer for Multimodal Reasoning
Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning

🔹 Must-Have Requirements

Strong Python programming skills
Knowledge of deep learning and machine learning fundamentals
Hands-on experience with PyTorch
Familiarity with Vision-Language Models or Large Language Models
Strong research interest and willingness to learn
Ready to start immediately

⏳ Workload

Commitment: At least 20 hours per week

📌 Note: Filling out this form does not guarantee acceptance. Only shortlisted candidates will be contacted via email.

🔗 Apply here: Form

💬 Telegram: @Arianaghamohseni

@RIMLLab

#research_position #ML_research #VisionLanguageModels #MultimodalAI #VisualReasoning #DeepLearning

🔐 LLM Faithfulness Journal Club

✅ This Week's Presentation:

🔹 Title: Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety

🔸 Presenter: [Farzan Rahmani](https://www.linkedin.com/in/farzan-rahmani-51128b201)

🌀 Abstract:
AI systems that “think” in human language offer a unique opportunity for AI safety: we can monitor their chains of thought (CoT) for the intent to misbehave. Like all other known AI oversight methods, CoT monitoring is imperfect and allows some misbehavior to go unnoticed. Nevertheless, it shows promise and we recommend further research into CoT monitorability and investment in CoT monitoring alongside existing safety methods. Because CoT monitorability may be fragile, we recommend that frontier model developers consider the impact of development decisions on CoT monitorability.

📄 Paper: [Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety](https://arxiv.org/abs/2507.11473v2)

Session Details:

- 📅 Date: Wednesday (چهارشنبه)
- 🕒 Time: 10:00 - 11:00 AM
- 🌐 Location: Online at http://vc.sharif.edu/ch/rohban

We look forward to your participation! ✌️

🔐 ML Security Journal Club

✅ This Week's Presentation:

🔹 Title: A Machine Unlearning Approach to Safety Alignment

🔸 Presenter: Arian Komaei

🌀 Abstract:
The paper identifies a fundamental limitation in current vision language model (VLM) alignment called the "safety mirage." Traditional supervised safety fine-tuning often reinforces superficial textual patterns rather than deep harm mitigation, leaving models vulnerable to simple one-word attacks and causing "over-prudence" (unnecessary rejections of benign queries). To address this, the authors propose Machine Unlearning (MU) as a superior alternative. Unlike standard fine-tuning, MU directly removes harmful knowledge and avoids biased feature-label mappings. Extensive evaluations show that MU-based alignment reduces attack success rates by up to 60.27% and cuts unnecessary rejections by over 84.20%, all while preserving the model's general capabilities.

📄 Paper: Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearning

Session Details:

* 📅 Date: Thursday پنج شنبه
* 🕒 Time: 9:00 - 10:00 AM
* 🌐 Location: Online at vc.sharif.edu/ch/rohban
We look forward to your participation! ✌️

🔘 Open Research Positions: Shortcut Learning and Spurious Correlation
We are looking for motivated students to join our research projects.


🔍 Project Description
Shortcut learning occurs when models rely on spurious or overly simple patterns instead of learning the intended underlying task. Our research aims to understand why and how shortcuts emerge, and how they can be identified, analyzed, and mitigated.
We offer three research assistant positions aligned with the following directions, co-advised by Dr. Rohban and Dr. Soleymani:

🔹 1. Understanding the Implicit Effects of Inductive Biases on Shortcut Learning

This project investigates how training-related inductive biases, such as batch size, learning rate, and loss function, influence the emergence of shortcut learning and group robustness.

[1] The Silent Helper: How Implicit Regularization Enhances Group Robustness (HiLD Workshop ICLR 2025)
[2] On the Role of Implicit Regularization of Stochastic Gradient Descent in Group Robustness (ICLR 2026)

🔗 Apply here: Google Form



🔹 2. Understanding Shortcut Learning Through the Lens of Task Arithmetic

It has been observed that shortcut features are often learned early during training. By analyzing the trajectory of weight evolution, we aim to identify shortcut task directions in parameter space and leverage this understanding to mitigate shortcut reliance.

[3] Editing Models with Task Arithmetic (ICLR 2023)

🔗 Apply here: Google Form



🔹 3. Investigating Shortcut Learning in Large-Scale Models

Shortcut-based generalization failures were first studied in linear models and simple settings. In this project, we aim to understand how these biases propagate to large-scale models, including language models, and explore strategies to mitigate them.

[4] Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models (EMNLP 2024)
[5] Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful (NeurIPS 2025)

🔗 Apply here: Google Form



📌 Important Notes:
Please carefully read the Internship Ethics and Guidelines before submitting your application.
Submitting the application form does not guarantee acceptance.
Only shortlisted candidates will be contacted via email by March 15.

دانشجویانی که علاقه مند به دستیاری آموزشی درس یادگیری تقویتی دکتر رهبان در نیم سال جاری هستند لطفا فرم زیر را پر کنند.


https://docs.google.com/forms/d/e/1FAIpQLSe2rPGlxxTQDzDnt6PB_hRAtBa64_OLmWLE_dUnPqQlAfpUqQ/viewform?usp=dialog

We invite interested collaborators to join an ongoing research project that aims to redefine attention mechanisms in large language models, drawing inspiration from the concept of consciousness. The goal of this work is to produce results suitable for submission to NeurIPS 2026. This research is conducted under the supervision of Dr. Rohban and is led by his PhD student, Hamidreza Akbari. Motivated undergraduate students with a strong background or interest in deep learning/LLMs and related fields are encouraged to reach out to @hamidrakbari to explore potential collaboration opportunities.