little-things

Commit Graph

Автор	SHA1	Съобщение	Дата
JustAnotherArchivist	a812cb5fc2	More snscrape helper tools	преди 5 години
JustAnotherArchivist	3ee3ffc340	Generate commands for Blogspot	преди 5 години
JustAnotherArchivist	5090a8ad02	Enumerate users on a Mastodon instance	преди 5 години
JustAnotherArchivist	0000d8ffd9	Add script to queue derive on IA	преди 5 години
JustAnotherArchivist	6dc711c54e	Further helper scripts for snscrape: normalising usernames and extracting them from a list of URLs	преди 5 години
JustAnotherArchivist	e3a37455ba	Add uniqify	преди 5 години
JustAnotherArchivist	321067819c	Proper script for tracking size of uploaded data	преди 5 години
JustAnotherArchivist	5c654cb16b	Split out size formatting	преди 5 години
JustAnotherArchivist	de2cdc0aae	curl with ArchiveBot UA	преди 5 години
JustAnotherArchivist	89ccd68b59	Helper tools for snscrape and the wiki pages	преди 5 години
JustAnotherArchivist	f2e836d2e9	Add support for differently formatted digests	преди 5 години
JustAnotherArchivist	94c4f76570	Fix crash when a digest is missing from a record	преди 5 години
JustAnotherArchivist	ef78a3318c	Colour only the header field names but not the values	преди 5 години
JustAnotherArchivist	9ce4653094	Document colouring and usage	преди 5 години
JustAnotherArchivist	e7c5d82254	Coloured WARCs?!	преди 5 години
JustAnotherArchivist	70b413f5c1	Better events: include raw WARC header data and separate HTTP requests into headers and body	преди 5 години
JustAnotherArchivist	641bc7a207	Fix infinite loop at end of WARC	преди 5 години
JustAnotherArchivist	a700e8e2fe	Add tcp-closer command	преди 5 години
JustAnotherArchivist	859c75a591	Add tool for WARC verification and extraction	преди 5 години
JustAnotherArchivist	e867a2327f	Replace urlencoded @ symbol The fix for https://github.com/dutchcoders/transfer.sh/issues/215 led to @ being encoded as %40 in filenames in the URL returned, which is awkward when working with social media scrapes since ArchiveBot normalises it to @ again.	преди 5 години
JustAnotherArchivist	cbd952024b	Workaround for hash no longer needed with current transfer.sh code	преди 5 години
JustAnotherArchivist	61431c2054	Add VK scraping helper	преди 5 години
JustAnotherArchivist	d6ff566c4d	Instagram always uses lower-case usernames	преди 5 години
JustAnotherArchivist	138c2a2d39	Get rid of post-processing now that snscrape (dev version) has clean URLs Keep the dirty URLs on Instagram because they're not that dirty and are linked from the profile pages. I usually throw it into ArchiveBot anyway such that it grabs the non-"taken-by" URLs as well.	преди 5 години
JustAnotherArchivist	27b0d2da75	Better username capitalisation extraction method	преди 5 години
JustAnotherArchivist	3aa828a0ac	transfer.kiska.pw -> transfer.notkiska.pw	преди 5 години
JustAnotherArchivist	63f4a8b3d3	transfer.sh -> transfer.kiska.pw	преди 5 години
JustAnotherArchivist	0168d50f62	Automatically fix capitalisation of Facebook and Twitter usernames	преди 5 години
JustAnotherArchivist	db0104b3c8	Get correct capitalisation for a Facebook username	преди 5 години
JustAnotherArchivist	4a1a9a10e0	Allow overriding the "remote filename"	преди 5 години
JustAnotherArchivist	769f95808e	Add ix.io upload script	преди 5 години
JustAnotherArchivist	c79721337b	+x	преди 5 години
JustAnotherArchivist	c30dcf5985	Finding outdated Mastodon instances	преди 5 години
JustAnotherArchivist	1748a6b607	Better workaround for the 5000 results limit; works for FoolFuuka 2.0.1 and up	преди 5 години
JustAnotherArchivist	fd680551df	Add Bing, Reddit/Pushshift, and FoolFuuka scrapers	преди 5 години
JustAnotherArchivist	ede77ad142	Filter Twitter hashtag scrapes based on account scrapes	преди 5 години
JustAnotherArchivist	57ef544c6c	Fix line endings	преди 5 години
JustAnotherArchivist	07c3e7baaa	Add snscrape helpers	преди 5 години
JustAnotherArchivist	b7e3a703d8	Monitor how a pipeline's wget processes are faring	преди 5 години
JustAnotherArchivist	168f61b39a	Quote filename so it works with any weird characters in the paths (Last reconstructed commit from text file full of different versions)	преди 5 години
JustAnotherArchivist	8f77c8c72a	xargs -r flag to not run the second find if the first produces no results (GNU extension)	преди 5 години
JustAnotherArchivist	9d7a4096f9	Pipe into second find directly	преди 5 години
JustAnotherArchivist	e3a4bf6a47	Replace slow lsof with procfs access	преди 5 години
JustAnotherArchivist	4a83a54616	Print host for each stuck request	преди 5 години
JustAnotherArchivist	2b2c65f034	Print PID	преди 5 години
JustAnotherArchivist	fadb70e297	Fixed version which handles multiple roots correctly	преди 5 години
JustAnotherArchivist	d10a1d3675	First set of little things	преди 5 години
JustAnotherArchivist	a00607f28e	Initial commit	преди 5 години

48 Ревизии (a812cb5fc291e8e22c41a65a15bc965aa8155577) Всички клонове Търсене

48 Ревизии (a812cb5fc291e8e22c41a65a15bc965aa8155577)

Всички клонове