This website works better with JavaScript.
Home
Explore
Help
Sign In
JustAnotherArchivist
/
little-things
Watch
2
Star
0
Fork
0
Code
Issues
0
Pull Requests
0
Releases
0
Wiki
Activity
Browse Source
Fix deduplication within each section processing
master
JustAnotherArchivist
4 years ago
parent
98d77ecc96
commit
256a94443e
1 changed files
with
5 additions
and
0 deletions
Unified View
Diff Options
Show Stats
Download Patch File
Download Diff File
+5
-0
wiki-recursive-extract-normalise
+ 5
- 0
wiki-recursive-extract-normalise
View File
@@ -51,6 +51,11 @@ do
url="${url%% | *}"
url="${url%% | *}"
fi
fi
if [[ "${sectionUrls[${url}]}" ]]
then
# Processed already, skip
continue
fi
sectionUrls["${url}"]=1
sectionUrls["${url}"]=1
toProcess=("${url}")
toProcess=("${url}")
while [[ ${#toProcess[@]} -gt 0 ]]
while [[ ${#toProcess[@]} -gt 0 ]]
Write
Preview
Loading…
Cancel
Save