48 Révisions (a812cb5fc291e8e22c41a65a15bc965aa8155577)
 

Auteur SHA1 Message Date
  JustAnotherArchivist a812cb5fc2 More snscrape helper tools il y a 5 ans
  JustAnotherArchivist 3ee3ffc340 Generate commands for Blogspot il y a 5 ans
  JustAnotherArchivist 5090a8ad02 Enumerate users on a Mastodon instance il y a 5 ans
  JustAnotherArchivist 0000d8ffd9 Add script to queue derive on IA il y a 5 ans
  JustAnotherArchivist 6dc711c54e Further helper scripts for snscrape: normalising usernames and extracting them from a list of URLs il y a 5 ans
  JustAnotherArchivist e3a37455ba Add uniqify il y a 5 ans
  JustAnotherArchivist 321067819c Proper script for tracking size of uploaded data il y a 5 ans
  JustAnotherArchivist 5c654cb16b Split out size formatting il y a 5 ans
  JustAnotherArchivist de2cdc0aae curl with ArchiveBot UA il y a 5 ans
  JustAnotherArchivist 89ccd68b59 Helper tools for snscrape and the wiki pages il y a 5 ans
  JustAnotherArchivist f2e836d2e9 Add support for differently formatted digests il y a 5 ans
  JustAnotherArchivist 94c4f76570 Fix crash when a digest is missing from a record il y a 5 ans
  JustAnotherArchivist ef78a3318c Colour only the header field names but not the values il y a 5 ans
  JustAnotherArchivist 9ce4653094 Document colouring and usage il y a 5 ans
  JustAnotherArchivist e7c5d82254 Coloured WARCs?! il y a 5 ans
  JustAnotherArchivist 70b413f5c1 Better events: include raw WARC header data and separate HTTP requests into headers and body il y a 5 ans
  JustAnotherArchivist 641bc7a207 Fix infinite loop at end of WARC il y a 5 ans
  JustAnotherArchivist a700e8e2fe Add tcp-closer command il y a 5 ans
  JustAnotherArchivist 859c75a591 Add tool for WARC verification and extraction il y a 5 ans
  JustAnotherArchivist e867a2327f Replace urlencoded @ symbol il y a 5 ans
  JustAnotherArchivist cbd952024b Workaround for hash no longer needed with current transfer.sh code il y a 5 ans
  JustAnotherArchivist 61431c2054 Add VK scraping helper il y a 5 ans
  JustAnotherArchivist d6ff566c4d Instagram always uses lower-case usernames il y a 5 ans
  JustAnotherArchivist 138c2a2d39 Get rid of post-processing now that snscrape (dev version) has clean URLs il y a 5 ans
  JustAnotherArchivist 27b0d2da75 Better username capitalisation extraction method il y a 5 ans
  JustAnotherArchivist 3aa828a0ac transfer.kiska.pw -> transfer.notkiska.pw il y a 5 ans
  JustAnotherArchivist 63f4a8b3d3 transfer.sh -> transfer.kiska.pw il y a 5 ans
  JustAnotherArchivist 0168d50f62 Automatically fix capitalisation of Facebook and Twitter usernames il y a 5 ans
  JustAnotherArchivist db0104b3c8 Get correct capitalisation for a Facebook username il y a 5 ans
  JustAnotherArchivist 4a1a9a10e0 Allow overriding the "remote filename" il y a 5 ans
  JustAnotherArchivist 769f95808e Add ix.io upload script il y a 5 ans
  JustAnotherArchivist c79721337b +x il y a 5 ans
  JustAnotherArchivist c30dcf5985 Finding outdated Mastodon instances il y a 5 ans
  JustAnotherArchivist 1748a6b607 Better workaround for the 5000 results limit; works for FoolFuuka 2.0.1 and up il y a 5 ans
  JustAnotherArchivist fd680551df Add Bing, Reddit/Pushshift, and FoolFuuka scrapers il y a 5 ans
  JustAnotherArchivist ede77ad142 Filter Twitter hashtag scrapes based on account scrapes il y a 5 ans
  JustAnotherArchivist 57ef544c6c Fix line endings il y a 5 ans
  JustAnotherArchivist 07c3e7baaa Add snscrape helpers il y a 5 ans
  JustAnotherArchivist b7e3a703d8 Monitor how a pipeline's wget processes are faring il y a 5 ans
  JustAnotherArchivist 168f61b39a Quote filename so it works with any weird characters in the paths il y a 5 ans
  JustAnotherArchivist 8f77c8c72a xargs -r flag to not run the second find if the first produces no results (GNU extension) il y a 5 ans
  JustAnotherArchivist 9d7a4096f9 Pipe into second find directly il y a 5 ans
  JustAnotherArchivist e3a4bf6a47 Replace slow lsof with procfs access il y a 5 ans
  JustAnotherArchivist 4a83a54616 Print host for each stuck request il y a 5 ans
  JustAnotherArchivist 2b2c65f034 Print PID il y a 5 ans
  JustAnotherArchivist fadb70e297 Fixed version which handles multiple roots correctly il y a 5 ans
  JustAnotherArchivist d10a1d3675 First set of little things il y a 5 ans
  JustAnotherArchivist a00607f28e Initial commit il y a 5 ans