Nginx stops sending request body when upstream server consuming body while returning data

Hi there,

I use Nginx as a reverse proxy. My client sends 20M data to Nginx, and Nginx received all of them (I can view 20M file in /var/lib/nginx). But when Nginx tries to proxy the 20M data to upstream server, only half of them are sent. My server's function is reading a file while returning the encoded file. Server starts to reply before it receives all of the request body for efficiency purpose.

The debug_log shows: writev() not ready (11: Resource temporarily unavailable) while writing to upstream server
Following log is where Nginx stops sending data to upstream server:
2019/09/09 11:02:47 [debug] 15570#0: *1 http body new buf t:1 f:0 00005654FB2DE420, pos 00005654FB2DE420, size: 8192 file: 0, size: 0
2019/09/09 11:02:47 [debug] 15570#0: *1 chain writer buf fl:1 s:8192
2019/09/09 11:02:47 [debug] 15570#0: *1 chain writer in: 00005654FB28D800
2019/09/09 11:02:47 [debug] 15570#0: *1 writev: 8192 of 8192
2019/09/09 11:02:47 [debug] 15570#0: *1 chain writer out: 0000000000000000
2019/09/09 11:02:47 [debug] 15570#0: *1 http read client request body
2019/09/09 11:02:47 [debug] 15570#0: *1 recv: eof:0, avail:1
2019/09/09 11:02:47 [debug] 15570#0: *1 recv: fd:3 8192 of 8192
2019/09/09 11:02:47 [debug] 15570#0: *1 http client request body recv 8192
2019/09/09 11:02:47 [debug] 15570#0: *1 http body new buf t:1 f:0 00005654FB2DE420, pos 00005654FB2DE420, size: 8192 file: 0, size: 0
2019/09/09 11:02:47 [debug] 15570#0: *1 chain writer buf fl:1 s:8192
2019/09/09 11:02:47 [debug] 15570#0: *1 chain writer in: 00005654FB28D800
2019/09/09 11:02:47 [debug] 15570#0: *1 writev: 8192 of 8192
2019/09/09 11:02:47 [debug] 15570#0: *1 chain writer out: 0000000000000000
2019/09/09 11:02:47 [debug] 15570#0: *1 http read client request body
2019/09/09 11:02:47 [debug] 15570#0: *1 recv: eof:0, avail:1
2019/09/09 11:02:47 [debug] 15570#0: *1 recv: fd:3 8192 of 8192
2019/09/09 11:02:47 [debug] 15570#0: *1 http client request body recv 8192
2019/09/09 11:02:47 [debug] 15570#0: *1 http body new buf t:1 f:0 00005654FB2DE420, pos 00005654FB2DE420, size: 8192 file: 0, size: 0
2019/09/09 11:02:47 [debug] 15570#0: *1 chain writer buf fl:1 s:8192
2019/09/09 11:02:47 [debug] 15570#0: *1 chain writer in: 00005654FB28D800
2019/09/09 11:02:47 [debug] 15570#0: *1 writev: 4800 of 8192
2019/09/09 11:02:47 [debug] 15570#0: *1 writev: -1 of 3392
2019/09/09 11:02:47 [debug] 15570#0: *1 writev() not ready (11: Resource temporarily unavailable)
2019/09/09 11:02:47 [debug] 15570#0: *1 chain writer out: 00005654FB28D800
2019/09/09 11:02:47 [debug] 15570#0: *1 http read client request body
2019/09/09 11:02:47 [debug] 15570#0: *1 event timer del: 10: 1044869538
2019/09/09 11:02:47 [debug] 15570#0: *1 event timer add: 10: 60000:1044879558
2019/09/09 11:02:47 [debug] 15570#0: timer delta: 9736
2019/09/09 11:02:47 [debug] 15570#0: worker cycle
2019/09/09 11:02:47 [debug] 15570#0: epoll timer: 60000
2019/09/09 11:02:47 [debug] 15570#0: epoll: fd:3 ev:0005 d:00007FD0DDF9C1E0
2019/09/09 11:02:47 [debug] 15570#0: *1 http run request: "/dbsec/encodeData?"
2019/09/09 11:02:47 [debug] 15570#0: *1 http upstream read request handler
2019/09/09 11:02:47 [debug] 15570#0: *1 http upstream send request
2019/09/09 11:02:47 [debug] 15570#0: *1 http upstream send request body
2019/09/09 11:02:47 [debug] 15570#0: *1 http read client request body
2019/09/09 11:02:47 [debug] 15570#0: *1 event timer: 10, old: 1044879558, new: 1044879622
2019/09/09 11:02:47 [debug] 15570#0: *1 http run request: "/dbsec/encodeData?"
2019/09/09 11:02:47 [debug] 15570#0: *1 http upstream check client, write event:1, "/dbsec/encodeData"
2019/09/09 11:02:47 [debug] 15570#0: epoll: fd:10 ev:0001 d:00007FD0DDF9C2C8
2019/09/09 11:02:47 [debug] 15570#0: *1 http upstream request: "/dbsec/encodeData?"
2019/09/09 11:02:47 [debug] 15570#0: *1 http upstream process header
2019/09/09 11:02:47 [debug] 15570#0: *1 malloc: 00005654FB28DE80:4096
2019/09/09 11:02:47 [debug] 15570#0: *1 recv: eof:0, avail:1
2019/09/09 11:02:47 [debug] 15570#0: *1 recv: fd:10 4096 of 4096
2019/09/09 11:02:47 [debug] 15570#0: *1 http proxy status 200 "200 OK"
2019/09/09 11:02:47 [debug] 15570#0: *1 http proxy header: "Date: Mon, 09 Sep 2019 15:02:37 GMT"
2019/09/09 11:02:47 [debug] 15570#0: *1 http proxy header: "Content-Type: application/octet-stream"
2019/09/09 11:02:47 [debug] 15570#0: *1 http proxy header: "Transfer-Encoding: chunked"
2019/09/09 11:02:47 [debug] 15570#0: *1 http proxy header: "Server: Jetty(9.4.8.v20171121)"
2019/09/09 11:02:47 [debug] 15570#0: *1 http proxy header done
2019/09/09 11:02:47 [debug] 15570#0: *1 HTTP/1.1 200 OK
Server: nginx/1.16.1
Date: Mon, 09 Sep 2019 15:02:47 GMT
Content-Type: application/octet-stream
Transfer-Encoding: chunked
Connection: keep-alive


Does someone know what happens here and why write stops? Thanks in advance!