Sfoglia il codice sorgente

Disable decoding the response content

chardet can be very slow (https://github.com/chardet/chardet/issues/29 https://github.com/psf/requests/issues/2359) and the decoding may be unnecessary if it's binary content.
tags/v0.2.0
JustAnotherArchivist 4 anni fa
parent
commit
46c95e2157
1 ha cambiato i file con 1 aggiunte e 1 eliminazioni
  1. +1
    -1
      qwarc/__init__.py

+ 1
- 1
qwarc/__init__.py Vedi File

@@ -65,7 +65,7 @@ class Item:
logging.info('Fetching {}'.format(url))
response = await self.session.request(method, url, data = data, headers = headers, allow_redirects = False)
try:
ret = await response.text(errors = 'surrogateescape')
ret = await response.read()
except:
# No calling the handleResponse callback here because this is really bad. The not-so-bad exceptions (e.g. an error during reading the response) will be caught further down.
response.close()


Caricamento…
Annulla
Salva