48 Commits (a812cb5fc291e8e22c41a65a15bc965aa8155577)
 

Author SHA1 Message Date
  JustAnotherArchivist a812cb5fc2 More snscrape helper tools 5 years ago
  JustAnotherArchivist 3ee3ffc340 Generate commands for Blogspot 5 years ago
  JustAnotherArchivist 5090a8ad02 Enumerate users on a Mastodon instance 5 years ago
  JustAnotherArchivist 0000d8ffd9 Add script to queue derive on IA 5 years ago
  JustAnotherArchivist 6dc711c54e Further helper scripts for snscrape: normalising usernames and extracting them from a list of URLs 5 years ago
  JustAnotherArchivist e3a37455ba Add uniqify 5 years ago
  JustAnotherArchivist 321067819c Proper script for tracking size of uploaded data 5 years ago
  JustAnotherArchivist 5c654cb16b Split out size formatting 5 years ago
  JustAnotherArchivist de2cdc0aae curl with ArchiveBot UA 5 years ago
  JustAnotherArchivist 89ccd68b59 Helper tools for snscrape and the wiki pages 5 years ago
  JustAnotherArchivist f2e836d2e9 Add support for differently formatted digests 5 years ago
  JustAnotherArchivist 94c4f76570 Fix crash when a digest is missing from a record 5 years ago
  JustAnotherArchivist ef78a3318c Colour only the header field names but not the values 5 years ago
  JustAnotherArchivist 9ce4653094 Document colouring and usage 5 years ago
  JustAnotherArchivist e7c5d82254 Coloured WARCs?! 5 years ago
  JustAnotherArchivist 70b413f5c1 Better events: include raw WARC header data and separate HTTP requests into headers and body 5 years ago
  JustAnotherArchivist 641bc7a207 Fix infinite loop at end of WARC 5 years ago
  JustAnotherArchivist a700e8e2fe Add tcp-closer command 5 years ago
  JustAnotherArchivist 859c75a591 Add tool for WARC verification and extraction 5 years ago
  JustAnotherArchivist e867a2327f Replace urlencoded @ symbol 5 years ago
  JustAnotherArchivist cbd952024b Workaround for hash no longer needed with current transfer.sh code 5 years ago
  JustAnotherArchivist 61431c2054 Add VK scraping helper 5 years ago
  JustAnotherArchivist d6ff566c4d Instagram always uses lower-case usernames 5 years ago
  JustAnotherArchivist 138c2a2d39 Get rid of post-processing now that snscrape (dev version) has clean URLs 5 years ago
  JustAnotherArchivist 27b0d2da75 Better username capitalisation extraction method 5 years ago
  JustAnotherArchivist 3aa828a0ac transfer.kiska.pw -> transfer.notkiska.pw 5 years ago
  JustAnotherArchivist 63f4a8b3d3 transfer.sh -> transfer.kiska.pw 5 years ago
  JustAnotherArchivist 0168d50f62 Automatically fix capitalisation of Facebook and Twitter usernames 5 years ago
  JustAnotherArchivist db0104b3c8 Get correct capitalisation for a Facebook username 5 years ago
  JustAnotherArchivist 4a1a9a10e0 Allow overriding the "remote filename" 5 years ago
  JustAnotherArchivist 769f95808e Add ix.io upload script 5 years ago
  JustAnotherArchivist c79721337b +x 5 years ago
  JustAnotherArchivist c30dcf5985 Finding outdated Mastodon instances 5 years ago
  JustAnotherArchivist 1748a6b607 Better workaround for the 5000 results limit; works for FoolFuuka 2.0.1 and up 5 years ago
  JustAnotherArchivist fd680551df Add Bing, Reddit/Pushshift, and FoolFuuka scrapers 5 years ago
  JustAnotherArchivist ede77ad142 Filter Twitter hashtag scrapes based on account scrapes 5 years ago
  JustAnotherArchivist 57ef544c6c Fix line endings 5 years ago
  JustAnotherArchivist 07c3e7baaa Add snscrape helpers 5 years ago
  JustAnotherArchivist b7e3a703d8 Monitor how a pipeline's wget processes are faring 5 years ago
  JustAnotherArchivist 168f61b39a Quote filename so it works with any weird characters in the paths 5 years ago
  JustAnotherArchivist 8f77c8c72a xargs -r flag to not run the second find if the first produces no results (GNU extension) 5 years ago
  JustAnotherArchivist 9d7a4096f9 Pipe into second find directly 5 years ago
  JustAnotherArchivist e3a4bf6a47 Replace slow lsof with procfs access 5 years ago
  JustAnotherArchivist 4a83a54616 Print host for each stuck request 5 years ago
  JustAnotherArchivist 2b2c65f034 Print PID 5 years ago
  JustAnotherArchivist fadb70e297 Fixed version which handles multiple roots correctly 5 years ago
  JustAnotherArchivist d10a1d3675 First set of little things 5 years ago
  JustAnotherArchivist a00607f28e Initial commit 5 years ago