When you run Python code, it appears to be multi-threaded. However, Python uses GIL (Global Interpreter Lock), which is an exclusion lock that prevents non-thread-safe code (such as C libraries) from being shared with other threads, thus limiting the number of threads that can run simultaneously to one Comment: This is not a good solution for Node. Comment: This is the same as Node.js. I think improvements are urgently needed.

https://christina04.hatenablog.com/entry/fastapi-def-vs-async-def

 2022 04 12

Should FastAPI use def or async def for Path Operation?

Python   FastAPI

background

In FastAPI, you can set the path of the HTTP server by using the decorator function as shown below, which is called Path Operation.

from fastapi import FastAPI

app = FastAPI ()

@ app.get ( "/" )
 async  def  read_root ():
     return { "Hello" : "World" }

@ app.get ( "/ items / {item_id}" )
 async  def  read_item (item_id: int , q: str = None ):
     return { "item_id" : item_id, "q" : q}

defI will explain the policy of whether to use or should use in this Path Operation async def.

environment

• Python  3.9.11
• FastAPI 0.74.1

policy

From the conclusion, let's implement it with the following policy.

• awaitasync defUse if there is
• defUse otherwise

cause

The reason is as follows.

• Even if it is CPU bound processing, it does not async defchangedef
• defFor IO bound in synchronous processing, it is better to use an external thread pool
• awaitAsynchronous processing is async defrequired

I will explain one by one.

FastAPI has a main async loop and an external thread pool

First of all, as a prerequisite, FastAPI has a main async loop and an external thread pool.

• def→ External thread pool
• async def→ Main a sync loop

Will be executed in.

The external thread pool looks multithreaded but is effectively single threaded

In the figure above, the external thread pool looks like multithreading.
However , Python uses the GIL (Global Interpreter Lock), which is an exclusive lock to prevent sharing unthread-safe code ( such as C language libraries) with other threads, so threads that can be executed at the same time . Is limited to one .

So there is no difference in CPU bound processing whether it is an external thread pool or a thread in the main async loop.

External thread pool can switch to another thread during synchronous IO

However, in the case of IO bound synchronization processing, it is advantageous to have multiple threads because it does not use the CPU and is simply blocked by waiting time.

In the main async loop, processing is blocked, so only three processes can be executed, whereas in an external thread pool with multiple threads, threads can be switched while waiting to process many requests .

awaitIn asynchronous processingasync def

awaitAsynchronous processing with is async defonly usable within the first place.

You can only use await inside of functions created with async def.

ref:  Concurrency and async / await --FastAPI

I think the following past articles will be helpful for distinguishing between synchronous and asynchronous.

christina04.hatenablog.com

Since the main async loop that was blocked earlier is an asynchronous IO, it is not blocked, and like a multithreaded synchronous IO, it will be in the next event during preparation, and when preparation is completed, processing of that event will be completed.

benchmark

Below are the actual benchmark results. From the left

1. defPerform synchronization processing with
2. Asynchronous processing is async defexecuted by
3. async defPerform synchronization processing with

It has become.

ref:  Are async path operations supposed to be exclusively single-threaded? · Issue # 4265 · tiangolo / fastapi · GitHub

You can see that the IO bound synchronization process is async defblocked inside and the performance is poor as mentioned above .

On the other hand, in the following benchmarks , CPU-bound processing has async defbetter performance .

ysk24ok.github.io

Since the context switch occurs when switching threads, it seems that the performance was affected by that amount.
However, it is costly to implement and review to always be aware of which function is CPU bound, so basically it defis better to give priority to the same level of performance.

summary

• If it is CPU bound processing, async def and def do not change much
• In case of IO bound in synchronous processing, def using external thread pool is excellent
• Async def is required for asynchronous processing with await

From that

• Use async def if you have await
• Otherwise use def

You should judge that.

reference

• Are async path operations supposed to be exclusively single-threaded? · Issue # 4265 · tiangolo / fastapi · GitHub
• What is the Global Interpreter Lock (GIL)?
• [FastAPI] Getting Started with Python's ASGI Web Framework FastAPI-Qiita

Junpei Tsuji  ( id: quoll00 )