Multiprocess directory iteration via os.scandir() with progress indicator via tqdm bars.
Alternatives To Iterfilesystem
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Fs_extra1662113522 months ago9February 03, 202334mitRust
Expanding opportunities standard library std::fs and std::io
7 years ago1September 12, 2016Go
Serve a JSON file as a file system over 9P
Node S3 Proxy10
10 years agoJavaScript
an example of streaming multipart uploads directly to s3, displaying a real-time progress bar
Dtype Fs7
3 years ago3gpl-3.0Vue
Filesystem app based on dType
4 years ago1mitGo
A tool to scan directories and report on the space that could be saved by hardlinking identical files. It can also perform the linking. Written in Go.
Go Finder4
9 years agoGo
Find files in your filesystem with Go. Work in progress, I'm just testing the language
Ceph Sync2
2 years agoJavaScript
Sync tool between LOCAL file system and REMOTE object storage.
3 years ago1otherPython
Multiprocess directory iteration via os.scandir() with progress indicator via tqdm bars.
6 years agomitPython
Python Interface Disk and Filesystem Utils
Convertvideos Electron1
a year ago17JavaScript
This is a project written in electron to select video(s) from the file system, convert them to a different video format, view the converted file by clicking on a button within the application, view the progress of the video conversion on the application, and add more videos while a video is converting.
Alternatives To Iterfilesystem
Select To Compare

Alternative Project Comparisons


Multiprocess directory iteration via os.scandir()

Who's this Lib for?

You want to process a large number of files and/or a few very big files and give feedback to the user on how long it will take.


  • Progress indicator:
    • Immediately after start: process files and indication of progress via multiprocess
    • process bars via tqdm
    • Estimated time based on file count and size
  • Easy to implement extra process bar for big file processing.
  • Skip directories and file name via fnmatch.

How it works:

The main process starts statistic processes in background via Python multiprocess and starts directly with the work.

There are two background statistic processes collects information for the process bars:

  • Count up all directories and files.
  • Accumulates the sizes of all files.

Why two processes?

Because collect only the count of all filesystem items via os.scandir() is very fast. This is the fastest way to predict a processing time.

Use os.DirEntry.stat() to get the file size is significantly slower: It requires another system call.

OK, but why two processed?

Use only the total count of all DirEntry may result in bad estimated time Progress indication. It depends on what the actual work is about: When processing the contents of large files, it is good to know how much total data to be processed.

That's why we used two ways: the DirEntry count to forecast a processing time very quickly and the size to improve the predicted time.


  • Python 3.6 or newer.
  • tqdm for process bars
  • psutils for setting process priority
  • For dev.: Pipenv. Packages and virtual environment manager


Please: try, fork and contribute! ;)

Build Status on
Build Status on
Coverage Status on
Coverage Status on
Requirements Status on


Use example CLI, e.g.:

~$ git clone
~$ cd IterFilesystem
~/IterFilesystem$ pipenv install
~/IterFilesystem$ pipenv shell
(IterFilesystem) ~/IterFilesystem$ print_fs_stats --help
(IterFilesystem) ~/IterFilesystem$ pip install -e .
Successfully installed iterfilesystem

~/IterFilesystem$ $ poetry run print_fs_stats --help
usage: [-h] [-v] [--debug] [--path PATH]
                         [--skip_dir_patterns [SKIP_DIR_PATTERNS [SKIP_DIR_PATTERNS ...]]]
                         [--skip_file_patterns [SKIP_FILE_PATTERNS [SKIP_FILE_PATTERNS ...]]]

Scan filesystem and print some information

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  --debug               enable DEBUG
  --path PATH           The file path that should be scanned e.g.: "~/foobar/"
                        default is "~"
  --skip_dir_patterns [SKIP_DIR_PATTERNS [SKIP_DIR_PATTERNS ...]]
                        Directory names to exclude from scan.
  --skip_file_patterns [SKIP_FILE_PATTERNS [SKIP_FILE_PATTERNS ...]]
                        File names to ignore.

example output looks like this:

(IterFilesystem) ~/IterFilesystem$ $ print_fs_stats --path ~/IterFilesystem --skip_dir_patterns ".*" "*.egg-info" --skip_file_patterns ".*"
Read/process: '~/IterFilesystem'...
Skip directory patterns:
	* .*
	* *.egg-info

Skip file patterns:
	* .*

Filesystem items..:Read/process: '~/IterFilesystem'...


Filesystem items..: 100%||135/135 13737.14entries/s [00:00<00:00, 13737.14entries/s]
File sizes........: 100%||843k/843k [00:00<00:00, 88.5MBytes/s]
Average progress..: 100%||00:00<00:00
Current File......:, /home/jens/repos/IterFilesystem/Pipfile

Processed 135 filesystem items in 0.02 sec
SHA515 hash calculated over all file content: 10f9475b21977f5aea1d4657a0e09ad153a594ab30abc2383bf107dbc60c430928596e368ebefab3e78ede61dcc101cb638a845348fe908786cb8754393439ef
File count: 109
Total file size: 843.5 KB
6 directories skipped.
6 files skipped.




Popular Progress Projects
Popular Filesystem Projects
Popular User Interface Components Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.