Browse Source

Don't close raw data tempfiles until the response gets GC'd

Closing the raw data tempfiles immediately on connection reuse caused any response reading to fail with an I/O error if another request started on the same connection in the meantime. Delaying the closing until the response object falls out of scope and gets GC'd ensures that as long as there is a reference to that object, it can be read from, at the expense of a possibly larger memory overhead.
tags/v0.2.3
JustAnotherArchivist 4 years ago
parent
commit
4ff8b260a1
1 changed files with 5 additions and 2 deletions
  1. +5
    -2
      qwarc/aiohttp.py

+ 5
- 2
qwarc/aiohttp.py View File

@@ -45,8 +45,6 @@ class ResponseHandler(aiohttp.client_proto.ResponseHandler):
self.rawData.responseData.write(data)

def reset_raw_data(self):
if self.rawData:
self.rawData.close()
self.rawData = RawData()


@@ -170,6 +168,11 @@ class ClientResponse(aiohttp.client_reqrep.ClientResponse):
self.connection.reset_raw_data()
await super().release()

def __del__(self):
if self._rawData:
self._rawData.close()
super().__del__()


class Payload:
# A class implementing the minimal subset used by the HttpPayloadParser to retrieve the data


Loading…
Cancel
Save