Nodejs get all files in directory recursively

I have a little problem with my function. I would like to get all files in many directories. Currently, I can retrieve the files in the file passed in parameters. I would like to retrieve the html files of each folder in the folder passed as a parameter. I will explain if I put in parameter "test" I retrieve the files in "test" but I would like to retrieve "test / 1 / *. Html", "test / 2 / . /.html ":

var srcpath2 = path.join['.', 'diapo', result];
function getDirectories[srcpath2] {
                return fs.readdirSync[srcpath2].filter[function [file] {
                    return fs.statSync[path.join[srcpath2, file]].isDirectory[];
                }];
            }

The result : [1,2,3]

thanks !

asked Jan 4, 2017 at 11:22

It looks like the glob npm package would help you. Here is an example of how to use it:

File hierarchy:

test
├── one.html
└── test-nested
    └── two.html

JS code:

const glob = require["glob"];

var getDirectories = function [src, callback] {
  glob[src + '/**/*', callback];
};
getDirectories['test', function [err, res] {
  if [err] {
    console.log['Error', err];
  } else {
    console.log[res];
  }
}];

which displays:

[ 'test/one.html',
  'test/test-nested',
  'test/test-nested/two.html' ]

NearHuscarl

47.6k12 gold badges188 silver badges172 bronze badges

answered Jan 4, 2017 at 11:32

Paul MougelPaul Mougel

16.3k5 gold badges54 silver badges63 bronze badges

3

I've seen many very long answers, and it's kinda a waste of memory space. Some also use packages like glob, but if you don't want to depend on any package, here's my solution.

const Path = require["path"];
const FS   = require["fs"];
let Files  = [];

function ThroughDirectory[Directory] {
    FS.readdirSync[Directory].forEach[File => {
        const Absolute = Path.join[Directory, File];
        if [FS.statSync[Absolute].isDirectory[]] return ThroughDirectory[Absolute];
        else return Files.push[Absolute];
    }];
}

ThroughDirectory["./input/directory/"];

It's pretty self-explanatory. There's an input directory, and it iterates through that. If one of the items is also a directory, go through that and so on. If it's a file, add the absolute path to the array.

Hope this helped :]

answered Jul 27, 2020 at 8:25

SmallySmally

1,2161 gold badge7 silver badges22 bronze badges

1

Using ES6 yield

const fs = require['fs'];
const path = require['path'];

function *walkSync[dir] {
  const files = fs.readdirSync[dir, { withFileTypes: true }];
  for [const file of files] {
    if [file.isDirectory[]] {
      yield* walkSync[path.join[dir, file.name]];
    } else {
      yield path.join[dir, file.name];
    }
  }
}

for [const filePath of walkSync[__dirname]] {
  console.log[filePath];
}

answered Feb 6, 2021 at 23:31

StephenStephen

3,2181 gold badge21 silver badges20 bronze badges

2

I really liked Smally's Solution but didn't like the Syntax.

Same solution but slightly easier to read:

const fs = require["fs"];
const path = require["path"];
let files = [];

const getFilesRecursively = [directory] => {
  const filesInDirectory = fs.readdirSync[directory];
  for [const file of filesInDirectory] {
    const absolute = path.join[directory, file];
    if [fs.statSync[absolute].isDirectory[]] {
        getFilesRecursively[absolute];
    } else {
        files.push[absolute];
    }
  }
};

answered Feb 13, 2021 at 16:07

arthurDentarthurDent

6257 silver badges10 bronze badges

1

Here's mine. Like all good answers it's hard to understand:

const isDirectory = path => statSync[path].isDirectory[];
const getDirectories = path =>
    readdirSync[path].map[name => join[path, name]].filter[isDirectory];

const isFile = path => statSync[path].isFile[];  
const getFiles = path =>
    readdirSync[path].map[name => join[path, name]].filter[isFile];

const getFilesRecursively = [path] => {
    let dirs = getDirectories[path];
    let files = dirs
        .map[dir => getFilesRecursively[dir]] // go through each directory
        .reduce[[a,b] => a.concat[b], []];    // map returns a 2d array [array of file arrays] so flatten
    return files.concat[getFiles[path]];
};

answered Nov 26, 2017 at 2:45

user875234user875234

2,2536 gold badges28 silver badges45 bronze badges

3

With modern JavaScript [NodeJs 10] you can use async generator function and loop through them using for-await...of

// ES modules syntax that is included by default in NodeJS 14.
// For earlier versions, use `--experimental-modules` flag
import fs from "fs/promises"

// or, without ES modules, use this:
// const fs = require['fs'].promises

async function run[] {
  for await [const file of getFiles[]] {
    console.log[file.path]
  }
}

async function* getFiles[path = `./`] {
  const entries = await fs.readdir[path, { withFileTypes: true }]

  for [let file of entries] {
    if [file.isDirectory[]] {
      yield* getFiles[`${path}${file.name}/`]
    } else {
      yield { ...file, path: path + file.name }
    }
  }
}

run[]

answered Oct 16, 2020 at 8:22

mikabytesmikabytes

1,7282 gold badges17 silver badges29 bronze badges

2

Packed into library: //www.npmjs.com/package/node-recursive-directory

//github.com/vvmspace/node-recursive-directory

List of files:

const getFiles = require['node-recursive-directory'];

[async [] => {
    const files = await getFiles['/home'];
    console.log[files];
}][]

List of files with parsed data:

const getFiles = require['node-resursive-directory'];
 
[async [] => {
    const files = await getFiles['/home', true]; // add true
    console.log[files];
}][]

You will get something like that:

  [
      ...,
      {
        fullpath: '/home/vvm/Downloads/images/Some/Some Image.jpg',
        filepath: '/home/vvm/Downloads/images/Some/',
        filename: 'Some Image.jpg',
        dirname: 'Some'
    },
  ]

Max

6,4563 gold badges41 silver badges57 bronze badges

answered Jun 24, 2020 at 14:42

2

You can also write your own code like below to traverse the directory as shown below :

var fs = require['fs'];
function traverseDirectory[dirname, callback] {
  var directory = [];
  fs.readdir[dirname, function[err, list] {
    dirname = fs.realpathSync[dirname];
    if [err] {
      return callback[err];
    }
    var listlength = list.length;
    list.forEach[function[file] {
      file = dirname + '\\' + file;
      fs.stat[file, function[err, stat] {
        directory.push[file];
 if [stat && stat.isDirectory[]] {
          traverseDirectory[file, function[err, parsed] {
     directory = directory.concat[parsed];
     if [!--listlength] {
       callback[null, directory];
     }
   }];
 } else {
     if [!--listlength] {
       callback[null, directory];
     }
          }
      }];
    }];
  }];
}
traverseDirectory[__dirname, function[err, result] {
  if [err] {
    console.log[err];
  }
  console.log[result];
}];

You can check more information about it here : //www.codingdefined.com/2014/09/how-to-navigate-through-directories-in.html

answered Jan 5, 2017 at 11:13

CodingDefinedCodingDefined

1,9621 gold badge15 silver badges13 bronze badges

2

I needed to so something similar, in an Electron app: get all subfolders in a given base folder, using TypeScript, and came up with this:

import { readdirSync, statSync, existsSync } from "fs";
import * as path from "path";

// recursive synchronous "walk" through a folder structure, with the given base path
getAllSubFolders = [baseFolder, folderList = []] => {

    let folders:string[] = readdirSync[baseFolder].filter[file => statSync[path.join[baseFolder, file]].isDirectory[]];
    folders.forEach[folder => {
        folderList.push[path.join[baseFolder,folder]];
        this.getAllSubFolders[path.join[baseFolder,folder], folderList];
    }];
}

answered Nov 7, 2017 at 15:09

BenBen

1,1292 gold badges17 silver badges26 bronze badges

const fs = require['fs'];
const path = require['path'];
var filesCollection = [];
const directoriesToSkip = ['bower_components', 'node_modules', 'www', 'platforms'];

function readDirectorySynchronously[directory] {
    var currentDirectorypath = path.join[__dirname + directory];

    var currentDirectory = fs.readdirSync[currentDirectorypath, 'utf8'];

    currentDirectory.forEach[file => {
        var fileShouldBeSkipped = directoriesToSkip.indexOf[file] > -1;
        var pathOfCurrentItem = path.join[__dirname + directory + '/' + file];
        if [!fileShouldBeSkipped && fs.statSync[pathOfCurrentItem].isFile[]] {
            filesCollection.push[pathOfCurrentItem];
        }
        else if [!fileShouldBeSkipped] {
            var directorypath = path.join[directory + '\\' + file];
            readDirectorySynchronously[directorypath];
        }
    }];
}

readDirectorySynchronously[''];

This will fill filesCollection with all the files in the directory and its subdirectories [it's recursive]. You have the option to skip some directory names in the directoriesToSkip array.

answered Mar 19, 2018 at 12:54

vvn050vvn050

1191 silver badge3 bronze badges

Speaking of npm packages - another short option is to use fs-readdir-recursive:

const read = require["fs-readdir-recursive"];
const foundFiles = read["test"];
console.log[foundFiles];

Output:

[ 'one.html', 'test-nested/some_text.txt', 'test-nested/two.html' ]

If you're interested only in files with specific extension [like .html mentioned in the question] you can filter them using .endsWith[]:

const filteredFiles = read["test"].filter[item => item.endsWith[".html"]];

answered Aug 11, 2021 at 10:05

ZukuZuku

80010 silver badges20 bronze badges

If you rather work synchronously with glob, use the glob.sync[] function as mentioned in their documentation. Here's the equivalent example provided by @Paul Mougel but written synchronously:

const glob = require["glob"];

var getDirectories = function [src] {
  return glob.sync[src + '/**/*'];
};
var rest = getDirectories['test'];
console.log[res];

answered May 1, 2021 at 16:56

You can use loop through all the files and directories of the root folder, if it's a directory, then get inside it and repeat the process. Consider the code below:

const fs = require['fs'];
const path = require['path'];

const target = './'; // choose the directory to target
var result = []
var filePaths = []
var tempFolder = []
const targetPath = fs.readdirSync[target];




function hit[mainPath = targetPath] {

  mainPath.forEach[[file] => {

    let check = fs.statSync[file];

    if [!check.isDirectory[]] {
      filePaths.push[file]
    }
    else {
      if [file[0] != '.'] {
        tempFolder.push[file]
      }
    }
  }];

  // get files from folder
  if [tempFolder.length > 0] {
    tempFolder.forEach[[dir] => {
      getFiles[dir]
    }]
  } 
   // filePaths contains path to every file

}




function getFiles[dir] {

  var paths = fs.readdirSync[dir];
  var files = [];

  paths.forEach[function [file] {
    var fullPath = dir + '/' + file;
    files.push[fullPath];
  }];

  files.forEach[[tempFile] => {
    let check = fs.statSync[tempFile];

    if [check.isDirectory[]] {
      getFiles[tempFile]
    } else {
      filePaths.push[tempFile]
    }
  }]
}



hit[]; // main function

answered Oct 16, 2020 at 9:32

A solution with Promises based on globby:

import { globby } from 'globby';

[async [] => {
  const path = '/path/to/dir';
  const files = await globby[[`${path}/**/*`]];

  console.log[files];
  // [ 
  //   '/path/to/dir/file1.txt', 
  //   '/path/to/dir/subdir/file2.txt', 
  //   ... 
  // ]
}][]

answered Jul 23, 2021 at 17:11

Although not perfect in some scenarios, it must be helpful in many.

const getAllFilePath = [path: string] => {
    const addData = [_paths: string[]] => {
        const newFoldersToScrape: string[] = [];

        _paths.forEach[_path => {
            fs.readdirSync[_path].forEach[[file: string] => {
                if [file.indexOf["."] === -1] {
                    newFoldersToScrape.push[`${_path}/${file}`];
                } else {
                    filePaths.push[`${_path}/${file}`];
                }
            }];
        }];

        foldersToScrape = newFoldersToScrape;
    };

    const baseDirPath = `/${path}`;
    let foldersToScrape: string[] = [];
    const filePaths: string[] = [];

    addData[[baseDirPath]];

    while [foldersToScrape.length !== 0] {
        addData[foldersToScrape];
    }

    return filePaths;
};

answered Feb 12 at 11:19

Here is a compact pure function that returns all the paths [relatives] in the directory.

import path from 'path'

const getFilesPathsRecursively = [directory: string, origin?: string]: string[] =>
  fs.readdirSync[directory].reduce[[files, file] => {
    const absolute = path.join[directory, file]
    return [
      ...files,
      ...[fs.statSync[absolute].isDirectory[]
        ? getFilesPathsRecursively[absolute, origin || directory]
        : [path.relative[origin || directory, absolute]]],
    ]
  }, []]

answered Jun 17, 2021 at 13:07

2

I did mine with typescript works well fairly easy to understand

    import * as fs from 'fs';
    import * as path from 'path';

    export const getAllSubFolders = [
      baseFolder: string,
      folderList: string[] = []
    ] => {
      const folders: string[] = fs
        .readdirSync[baseFolder]
        .filter[file => fs.statSync[path.join[baseFolder, file]].isDirectory[]];
      folders.forEach[folder => {
        folderList.push[path.join[baseFolder, folder]];
        getAllSubFolders[path.join[baseFolder, folder], folderList];
      }];
      return folderList;
    };
    export const getFilesInFolder = [rootPath: string] => {
      return fs
        .readdirSync[rootPath]
        .filter[
          filePath => !fs.statSync[path.join[rootPath, filePath]].isDirectory[]
        ]
        .map[filePath => path.normalize[path.join[rootPath, filePath]]];
    };
    export const getFilesRecursively = [rootPath: string] => {
      const subFolders: string[] = getAllSubFolders[rootPath];
      const allFiles: string[][] = subFolders.map[folder =>
        getFilesInFolder[folder]
      ];
      return [].concat.apply[[], allFiles];
    };

answered Nov 3, 2018 at 17:11

1