How Do CPU Threads Work? AMD SMT vs. Intel Hyperthreading

At the conceptual level, both SMT and hyperthreading are the same things.

However, SMT is the technology/architecture concept. Hyper-Threading is Intel’s brand name for its implementation of SMT. Intel historically called its equivalent technology Hyper-Threading Technology, or HTT, or HT. On the other hand, AMD calls it simply SMT.

Both allow two hardware threads to share the resources of one physical CPU core on the supported processors. The goal is to utilize a core’s resources more efficiently. Basically, the operating system is allowed to see a single core as two logical processors, allowing it to schedule two threads on that core. Two threads have separate architectural states but share the same core’s resources, including cache and execution units. The resulting performance improvements will depend mainly on the workload.

Both these terms are talked about as competitors, but it is slightly misleading. The difference is only in how AMD and Intel implement that concept in CPU architectures. So, if you ask if AMD SMT is better than Intel Hyper-Threading or vice versa, there is no simple answer. However, I have done some research on both and will reveal the secrets, working principles, and differences in this article.

Instructions and Processes and Their Relation to Threads

Before we jump directly to the technology, it is important to understand the fundamentals. A process is a running program. For example, Google Chrome or GTA-V. Each process has its own memory space, files, handles, and one or more threads.

So, every program becomes a process utilizing some memory space. It contains one or more threads. Each thread is responsible for executing instructions or CPU commands. The instructions run on the CPU cores, which are the actual hardware execution units.

It is the operating system’s responsibility to schedule threads. Threads are all software components. They are then assigned to logical processors. A logical processor is a virtual or simulated processing unit created through SMT or hyper-threading. Think of a logical processor like an artificial processor that is seen and managed by the operating system. Logical processors share the internal parts of a single physical core

The core can then execute the instructions from both the threads. However, it doesn’t necessarily happen always at the same time. A modern CPU has multiple execution resources and threads, which helps to share, not duplicate them.

For example, an instruction may need an integer unit, floating-point unit, etc. At the same time, another instruction may be unable to proceed because it is waiting for data from a cache or memory or because it depends on an earlier instruction. With SMT, the core doesn’t necessarily have to sit around doing nothing. It can work on two different threads if they want to use different available execution resources.

So, basically, we are sharing the same resources over two different tasks, which increases the core utilization (not doubling it). Intel says that in its Intel Xeon processors, with hyper-threading, the CPU utilization can increase anywhere from 10 to 30 percent. However, in the AMD EPYC processors, they found the performance between 30 and 50 percent.

What is SMT?

SMT stands for Simultaneous Multithreading. To be very simple, this technology allows one physical CPU core to work on two instruction streams at the same time. However, because they are sharing the same hardware, two threads can’t use the same resources on the same core at the same time. To be more precise, SMT allows for the different core resources to be used by different threads at the same time.

Consider an 8-core AMD processor with SMT.

It will still have 8 cores but 16 logical processors. However, it is possible that SMT doesn’t double the number of logical processors from the cores. For example, there can be CPUs with 8 cores and 12 threads. But that is the topic for another day.

For now, each physical core is maintaining two hardware threads. The operating system therefore sees 16 logical processors, even though the CPU has 8 physical cores.

But what does SMT actually do?

Our computer software resides on the operating system. Whenever we launch a program, the application creates software threads. These threads contain streams of instructions that need to be executed. Now, there is an operating system’s scheduler that decides which available local processor should run each software thread.

The scheduler basically says that this thread needs CPU time, so I’ll run it on the available logical processor. It looks at the logical processor rather than the physical cores. Now, the CPU’s hardware maps those logical processors into the physical cores and manages the actual execution. The relevant CPU parts responsible for the mapping are the CPU’s hardware thread contexts, instruction fetch/decode logic, scheduler/dispatch logic, and execution units.

Software threads and logical processors are two different things here.

The application generally creates a software thread by calling an OS threading API. It is the OS kernel that creates and manages the scheduling object.

On the other hand, the logical processor isn’t created by the operating system. The CPU hardware exposes them. When the system boots, the CPU provides the information about its topology. The firmware/BIOS and operating system discover this information during boot.

So, where does the SMT reside and operate?

If you ask where SMT is in all this, it is inside the CPU hardware. It sits between the OS’s logical processor and the physical core’s execution resources. If the CPU physically contains 8 cores, SMT gives each core two hardware-thread contexts so that the CPU exposes 16 logical processors to the OS.

Here, SMT is the CPU technology that makes the Core 0 capable of presenting itself as two logical processors (T1 and T2) while sharing the same core’s physical resources. So, SMT is not the software thread created by an application. It is also not the Windows scheduler or the logical processor itself. Nor is it another physical CPU core at all.

MT is simply the CPU microarchitectural mechanism that allows one physical core to maintain and execute multiple hardware threads concurrently.

AMD’s SMT (Simultaneous Multi-Threading)

AMD introduced 2-way SMT with the original Zen architecture in 2017. Before this, they were using Clustered Multithreading (CMT) in the Bulldozer-family CPUs. AMD has continued the basic 2-ways SMT approach through its Zen 2, Zen 3, Zen 4, and Zen 5 architectures. Essentially, the technologies work almost the same in both CPUs, but the companies add their own tweaks, algorithms, and strategies to improve resource usage. Let’s talk a little about how AMD is implementing the improvements.

AMD’s partitioning and watermarking in SMT

In its official article, AMD says that in-order queues are statically partitioned. while parts of the out-of-order queues and branch-prediction resources are watermarked for each thread. The remaining resources are competitively shared.

Watermarking here simply means that AMD sets limits so that one thread cannot consumer the entire resource pool of shared resources from a core. This makes the other thread unable to make progress.

So, if one thread is stalled for a long time, AMD doesn’t want the other thread to consume everything and leave no resources for the first task when it wakes up again. AMD will reserve some capacity while still giving resources to the active task.

I wasn’t able to find the exact resource-sharing mechanisms, but they are doing competitive sharing in areas like execution units, schedulers, register files, decode resources, caches, and µOP caches.

According to Mike Clark, Zen 5’s redesigned front end has two decoders, allowing the two SMT threads to each receive a decoder when operating in two-thread mode. The two threads therefore don’t simply have to compete for a single decoder in the same way older Zen designs did.

He also said that caches and TLBs remain competitively shared, while resources such as the retire queue, reorder buffer, store queue, and µOP queue are statically partitioned.

Intel Hyper-Threading

Intel first introduced the Hyper-Threading Technology (HTT) in the Xeon processor family in 2002. Intel kept the definition simple by saying that the technology allows one physical processor to appear to the OS as two logical processors.

Intel’s documentation for later Core processors describes this arrangement more deeply. It says that each logical processor has its own architectural state, including its own general-purpose and control registers, while resources such as execution units, caches, and buses are shared.

In another documentation, Intel declares that the execution engine is shared by two logical processors. The Rapid Execution Engine takes instructions from the instruction queues and sends them to the available execution units. They say that the majority of these execution resources are shared between two threads.

Intel specifically describes the Fetch and Deliver engine and Reorder and Retire block as partitioning some resources between the two threads, with some resources alternating between them. The execution engine itself is largely shared.

In Intel’s hyperthreading technology, a logical processor can borrow resources from its sibling logical processor when that sibling is stalled. In the workloads where most of the core’s resources are consumed, the second thread has much less to contribute.

However, if we talk about the fundamental difference between SMT and hyperthreading, you will hardly find any differences.

Intel’s hybrid design is where the game changes a little

Intel’s recent hybrid architecture makes the traditional CPU = Core x 2 threads rule unreliable. The 12th generation and later hybrid processors from Intel combine P-cores and E-cores. Historically, the P-cores supported hyper-threading, while the E-cores did not.

P-cores (Performance-cores) are designed for high single-thread performance and demanding workloads. They are larger in size and more complex with stronger out-of-order executions. Not to mention, they use hyperthreading.

The E-cores (Efficient-cores) are designed to provide more CPU throughput per unit of power and physical space. They are smaller in size and more power efficient. However, they generally support one hardware thread per core. They are mainly utilized for background workloads or parallel tasks. Basically, everywhere where having more physical cores is more valuable than maximizing performance.

So, a modern Intel CPU could have the core and threads configuration something like this:

8 P-cores × 2 threads = 16 logical processors
8 E-cores × 1 thread = 8 logical processors

Total = 16 cores / 24 threads

So, the E-cores are therefore not exactly “half and Hyperthread core.” They are actual physical cores that simply provide one hardware thread each.

And now, in their Core Ultra Series 2 processors, they are continuing without Hyper-Threading design, meaning their physical cores count equals their hardware-thread count. This is a significant current distinction from AMD, where Zen 4 and Zen 5 continue to use 2-way SMT on their supported cores.

Key differences and similarities between SMT and HT

Conclusion

Simultaneous Multithreading is an interesting topic, and as we do really deep dives, it reveals how smart we are when it comes to utilizing the free resources using the same hardware. People are confused between AMD and Intel’s implementation of the same technology. But they are essentially the same. And I don’t think a CPU purchase decision should be made by focusing on this technology rather than other things like raw core count, TDP, clock speed, and IPC etc.