在用 iris
和 gin
做压测,压测结果不怎么理想。发现两个程序都有以下的日志输出:
1
2
3
[HTTP Server] http: Accept error: accept tcp [::]:9000: accept4: too many open files; retrying in 5ms
[HTTP Server] http: Accept error: accept tcp [::]:9000: accept4: too many open files; retrying in 10ms
[HTTP Server] http: Accept error: accept tcp [::]:9000: accept4: too many open files; retrying in 5ms
看日志是说有 server 进程打开的文件太多,那么进程应该打开了最大限制量的文件描述符,无法再打开新的,在这里即网络连接。
想到压测是用 5000 的并发量测试的,查看系统限制:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
[root@Uphie ~]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 62790
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 62790
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[root@Uphie ~]# cat /proc/18178/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 10485760 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 62790 62790 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 62790 62790 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
可以看到系统限制了 server 进程可打开的文件描述符的数量为 1024(软限制),最大不能超过 4096(硬限制),限制是默认的没被修改过。
而我在别的机器上也做了相同并发的测试,没有出现上面的错误,查看其进程的文件描述符软硬限制已经被改为了102400,完全够了。
从系统层面修改限制
1
[root@Uphie ~]# vim /etc/security/limits.conf
在末尾修改:
1
2
* soft nofile 102400
* hard nofile 102400
重新登录,启动程序测试,问题解决。