You can tell by looking at the page. It is 283,645 per second in JSON serialization.
However, that number has little practical meaning.
First, a typical Web server performs processes such as user authentication, database access, and HTML creation, so the processing time is much longer. It simply means that Sanic's processing time is negligible.
Also, there are not many cases where a web application needs more than 1000 processes per second. First, a web server such as Niginx can process and cache static files. If you do so, it is unlikely that your company's internal system of 10,000 employees will need to process 1000 jobs per second.
Even now, Dgango and Flask are mainly used because they are generally sufficient in many cases. If you have a problem, it's usually better to use the Go language, which is faster and has more parallel processing capabilities than using Sanic in Python. Note that this does not mean that Sanic is not used, but that if the number of processes per second becomes very large, Python itself is often not suitable for that process. .
Views : 80 times · Requested by Masahiro Ishizuka
Thank you for your answer.
> In general, it is often better to use the Go language, which is faster and has more parallel processing functions than using Sanic in Python.
> In general, it is often better to use the Go language, which is faster and has more parallel processing functions than using Sanic in Python.
I see, together
Conclusion
- The GIL has ensured that multithreading does not benefit from multicore.
If this blog is right, and we consider it together, Python is a programming language that doesn't take advantage of multi-core CPUs. If you don't need AI (artificial intelligence) or machine learning libraries, Go lang is better than Python. We hope that either the framework or the formal support of multi-core CPUs will be resolved in the future. There is also an AI, machine learning and numerical analysis library developed by Google called TensorFlow for Go lang . This API can be used in Go lang, but it is still only Python that can build and learn models, so I hope that it will be improved in the future.
Original authorWed
In terms of GIL, it is usually not a problem because heavy processing can be eliminated by writing it in C / C ++ or using a library written in C / C ++. However, if the number of operations per second becomes very large, processing collisions cannot be avoided, and Python becomes inefficient. So I only recommend using Go. If it's not large, developing with Python alone is not a problem. For large scales, it may be necessary to calculate and test whether Python alone can withstand it.
As for machine learning, training uses a lot of data to do a lot of calculations. In this regard, Python is good at Go and not good at Go, so Go will never be able to build and learn machine learning models.
On the other hand, in the process using the learned result, the calculation is light because only the result is used. If you have to deal with a lot of processing here, there is an API for Go because Go is more suitable for the reasons mentioned above.
Both TesorFlow and Go have been developed by Google, but it's a good idea to consider the characteristics of Python and Go, so it won't be unified.
Show question
Introduction of respondents
It is verification reading the blog of Python3.
Maybe
In Python2 & 3, because it is a fake multi-core & multi-thread, it is verified that it is different from a true multi-core & multi-thread like JAVA or Go lang (Go language).
True multi-core & multi-thread support with uvloop?
It may be possible to say.
aon CEO Masahiro Ishizuka
tel: 042-559-8638
iPhone: 070-3861-5011
Find out why asyncio is using POSIX threads
Shared.
Tokibito-sensei ( id: nullpobug ) called out to the office to play, so we went to the open collector. I had a chat with Mr. aodag ( id: aodag ), but I remembered what I was worried about before.
I was worried
While preparing a presentation for a study session, I prepared a code example that sends an HTTP request to a server using asyncio and aiohttp.
import aiohttp import asyncio async def fetch (l, url): async with aiohttp.ClientSession (loop = l) as session: async with session.get (url) as response: return await response.text () async def main (l, url, num): tasks = [fetch (l, url) for _ in range (num)] return await asyncio.gather (* tasks) if __name__ == '__main__' : loop = asyncio.get_event_loop () results = loop.run_until_complete (main (loop, 'http: // localhost: 8000' , 3 )) for r in results: print (r)
PyCharm has a function to display a "Concurrency Diagram", and you can check the movement of threads and processes. The behavior of threads when this code is executed is as follows.
For some reason,
concurrent.futures.ThreadPoolExecutor is appearing. I tried to find out what was written in the document, but because I couldn't find any such descriptions, I gave up and asked Aodag and Tokibito.
socket.getaddrinfo
He found the cause in tens of minutes.
socket.getaddrinfo is executed synchronously, it socket.getaddrinfo that the cpython implementation does not execute this asynchronously, but executes concurrent.futures.ThreadPoolExecutor multiple threads.- https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/asyncio/base_events.py#L666-L673
- https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/asyncio/base_events.py#L627-L636
A sample using aioredis was prepared as a code sample in which getaddrinfo was not used. Connect to a locally built Redis server with a UNIX domain socket.
import asyncio import aioredis async def connection_example (key): conn = await aioredis.create_connection ( '/tmp/redis.sock' ) return await conn.execute ( 'GET' , key) async def main (num): tasks = [connection_example ( 'my-key' ) for _ in range (num)] return await asyncio.gather (* tasks) if __name__ == '__main__' : loop = asyncio.get_event_loop () results = loop.run_until_complete (main ( 3 )) for r in results: print (r)
By the way, redis config is ↓.
daemonize no pidfile /var/run/redis.pid unixsocket /tmp/redis.sock unixsocketperm 700 logfile "" databases 1
Looking at the Concurrency Diagram at this time,
Certainly no threads have been created.
Access to an external Redis server
On the other hand, if you prepare a Redis server outside and connect it (this time I used arukas.io ),
import asyncio import aioredis async def connection_example (key): conn = await aioredis.create_connection ( ( 'seaof-xxx-xxx.arukascloud.io' , 311390 ), db = 0 , password = 'xxxxxxxxxxxxxxx' ) return await conn.execute ( 'GET' , key) async def main (num): tasks = [connection_example ( 'my-key' ) for _ in range (num)] return await asyncio.gather (* tasks) if __name__ == '__main__' : loop = asyncio.get_event_loop () results = loop.run_until_complete (main ( 3 )) for r in results: print (r)
When executed,
It seems that a worker thread is created after all.
How many worker threads are created for ThreadPoolExecutor?
The Concurrency Diagram when the number of simultaneous executions in Semaphore is limited to 3 is as follows.
Apparently, Thread in ThreadPoolExecutor is not reused. The extent to which it was created was described in the documentation under ThreadPoolExecutor.
If max_workers is None or not specified, the default value is the number of processors on the machine multiplied by 5.17.4. Concurrent.futures – Concurrent task execution — Python 3.6.1 documentation
I send about 30 requests for a trial (Semaphore is 3).
Up to 20 were generated, and it was confirmed that they were reused thereafter. Although the upper limit of the worker thread cannot be changed due to the implementation, it is unlikely that there will be any problems.
uvloop
As for the uvloop, it was confirmed that they worked hard without using POSIX threads.
import uvloop # Omitted loop = uvloop.new_event_loop () asyncio.set_event_loop (loop)
When executed,
Oh, really.
in conclusion
Aodag-sensei and tokibito-sensei are amazing ... I was reading and leaving the document humid. I'm also a member of society from tomorrow, so I'm going to work hard for two people.
An open collector with Mr. aodag and Mr. tokibito seems to be looking for a job. Thank you very much!
0 コメント:
コメントを投稿