Browse Source

Make 25GB chunks for better uploading.

master
Alard 11 years ago
parent
commit
a1e5b5be25
1 changed files with 1 additions and 1 deletions
  1. +1
    -1
      chunker

+ 1
- 1
chunker View File

@@ -10,7 +10,7 @@

PATH_TO_UPLOADS=$1 # /home/archiveteam/uploads
PATH_TO_TARGET=$2 # /home/archiveteam/processed
MAX_MEGABYTES=$((1024*50))
MAX_MEGABYTES=$((1024*25))

# find every .warc.gz in the upload directory
find $PATH_TO_UPLOADS -type f -name "*.warc.gz" \


Loading…
Cancel
Save