TGViewer
ReverseEngineering ReverseEngineering @reverseengine · 1.33K subscribers
Post #1789 47
ReverseEngineering Trap و ورود از User Mode به Kernel Mode تا اینجا فهمیدیم برنامه ها توی User Mode اجرا میشن و وقتی بخوان از سیستم عامل یه کار حساس بخوان از System Call استفاده میکنن ولی یه سوال مهم داریم CPU دقیقا چطوری از اجرای عادی برنامه توی User Mode میره داخل Kernel…
Trap

and Entering from User Mode to Kernel Mode

So far we have understood that programs run in User Mode and when they ask the operating system for a sensitive task, they use System Call

But we have an important question

How exactly does the CPU go from normal program execution in User Mode to Kernel Mode

Here the concept of Trap comes into play

Trap

means a controlled transition from normal program execution to Kernel, meaning the program is running in User Mode

User Mode
│
│ Service Request
▼
Trap
│
▼
Kernel Mode

After the Kernel has done its job, control returns to the program

What is the difference between System Call and Trap

These two are not the same

System Call

Actually, the program requests the operating system

For example, the program says I want to read from a file

or write something

or get memory

For example:

read()
write()
mmap()

But Trap is a mechanism that causes the CPU to transfer execution control to the Kernel

So very simply
Program
│
│ System Call
▼
Trap / System call entry
│
▼
Kernel

That is, the System Call tells what service I want

Trap

or the entry path causes the program to enter the Kernel

Example x86 64:

In Linux on the x86 64 architecture, a System Call is usually executed with the
syscall

command

Before executing this command, the program places the necessary information into specific registers

Concept:

Registers
│
├── System Call Number

├── Argument 1
├── Argument 2
└──
│
▼
syscall
│
▼
Kernel

Kernel

When it enters, it checks the System Call number to find out exactly what the program requested. For example, the program may request to read information from a file. So the Kernel needs to know which System Call this request is related to.

How does the Kernel know what System Call was requested? Here, one of the important things is the System Call Number. Each System Call in Linux has a specific number. Of course, these numbers depend on the architecture and ABI. There are
That is, we should not think that the number of a System Call is the same on all systems

Conceptually:

System Call Number
+
Arguments
│
▼
syscall
│
▼
Kernel


Kernel

Using this information, it finds the path to that request

Kernel

What does it do when it enters
When the CPU enters the Kernel, the Kernel must check the program request

Simply
User Code
│
▼
syscall
│
▼
Kernel Entry
│
▼
Check the request
│
▼
Execution of the operation
│
▼
Return Value
│
▼
User Mode

Kernel

It may check things like access permissions and the validity of some information

For example, a User Space program should not be able to manipulate any memory address it wants without control
The operating system plays the role of a watchdog here because if any program could do anything with system resources, the operating system would practically become useless

Trap

Is not just for System Calls
Here is an important point We have

When we say Trap, we should not think that it is only used for System Call
In the operating system and CPU architecture, control transfer to the Kernel can occur for various reasons

For example:

System Call

Exception

Some hardware events via Interrupt, of course, these concepts are not exactly the same and should be examined separately

What is Exception

Exception

It happens when the CPU encounters a special condition while executing an instruction

For example:

Program
│
▼
Execution of instruction
│
▼
Special conditions
│
▼
Exception
│
▼
Kernel Handler

One of its famous examples is

Page Fault

Suppose the program accesses a memory page, but that page does not have the necessary conditions for access at that moment

The CPU
creates a Page Fault

Then the Kernel enters the story and checks what should be done

An important point:

Page Fault

does not necessarily mean Crash

The operating system may be able to handle it completely normally

For example, it can prepare the desired page and the program can continue again Give

Interrupt

Usually generated by hardware or hardware controller, for example, a device wants to notify the CPU of an event

Simply:
More from @reverseengine
  1. Oct 15, 2026Reverse Engineer Anything v6.5.0 A tool for reverse engineering that allows AI agents to a…
  2. Oct 15, 2026Someone Used AI to Reverse-Engineer PS5 Security in Weeks: What Is REA? https://pub.toward…
  3. Oct 15, 2026How Dangerous Is REA (Reverse Engineer Anything) for Your App? https://21zerixpm.medium.co…
  4. Oct 15, 2026REA Is Impressive. But Reverse Engineering Was Never the New Part https://crizantlai.mediu…
  5. Oct 15, 2026QuarkslaB Dynamic binary Instrumentation https://qbdi.quarkslab.com/introduction @reversee…
  6. Oct 15, 2026angr’s documentation https://docs.angr.io/en/latest/search.html?q=Document&check_keywords=…
Threads Profile ViewerView any public Threads profile without an account.Open ThreadLook →Writing with AI? Make it sound human.Metric37 rewrites AI drafts so they read naturally. Free AI detector, 1,500 words free.Try Metric37 →