Can python have multiple files open?

Since Python 3.3, you can use the class ExitStack from the contextlib module to safely
open an arbitrary number of files.

It can manage a dynamic number of context-aware objects, which means that it will prove especially useful if you don't know how many files you are going to handle.

In fact, the canonical use-case that is mentioned in the documentation is managing a dynamic number of files.

with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # All opened files will automatically be closed at the end of
    # the with statement, even if attempts to open files later
    # in the list raise an exception

If you are interested in the details, here is a generic example in order to explain how ExitStack operates:

from contextlib import ExitStack

class X:
    num = 1

    def __init__(self):
        self.num = X.num
        X.num += 1

    def __repr__(self):
        cls = type(self)
        return '{cls.__name__}{self.num}'.format(cls=cls, self=self)

    def __enter__(self):
        print('enter {!r}'.format(self))
        return self.num

    def __exit__(self, exc_type, exc_value, traceback):
        print('exit {!r}'.format(self))
        return True

xs = [X() for _ in range(3)]

with ExitStack() as stack:
    print(len(stack._exit_callbacks)) # number of callbacks called on exit
    nums = [stack.enter_context(x) for x in xs]
    print(len(stack._exit_callbacks))

print(len(stack._exit_callbacks))
print(nums)

Output:

0
enter X1
enter X2
enter X3
3
exit X3
exit X2
exit X1
0
[1, 2, 3]

• Problem Formulation and Solution Overview
• Preparation & Overview
• Method 1: Open Multiple Text Files with open()
• Method 2: Open Files Across Multiple Lines with open() and Backslash (\)
• Method 3: Open Multiple Text Files using Parenthesized Context Managers and open()
• Method 4: Open Multiple Text Files using the os library and open()
• Summary
• Programmer Humor

Problem Formulation and Solution Overview

In this article, you’ll learn how to open multiple files in Python.

💬 Question: How would we write Python code to open multiple files?

We can accomplish this task by one of the following options:

• Method 1: Open Multiple Text Files using open()
• Method 2:Open Multiple Text Files using open() and backslash (\)
• Method 3: Open Multiple Text Files using Parenthesized Context Managers and open()
• Method 4: Open Multiple Text Files using the os library and open()

To make it more fun, we have the following running scenario:

You have been contacted by the Finxter Academy to write code that opens multiple files and writes the contents from one file to another.

Preparation & Overview

If your script opens files, it is a good practice to check for errors, such as:

• No Such File Or Directory
• Not Readable
• Not Writable

In this regard, all examples in this article will be wrapped in a try/except statement to catch these errors.

Contents of orig_file.txt

To follow along with this article, create orig_file.txt with the contents below and place it in the current working directory.

30022425,Oliver,Grandmaster,2476
30022437,Cindy,Intermediate,1569
30022450,Leon,Authority,1967

Add the following code to the top of each code snippet. This snippet will allow the code in this article to run error-free.

import logging

This allows us to log any error message that may occur when handling the files.

Method 1: Open Multiple Text Files with open()

This method opens multiple text files simultaneously on one line using the open() statement separated by the comma (,) character.

try:
    with open('orig_file.txt', 'r') as fp1, open('new_file.txt', 'w') as fp2:
        fp2.write(fp1.read())
except OSError as error:
    logging.error("This Error Occurred: %s", error)

This code snippet is wrapped in a try/except statement to catch errors. When this runs, it falls inside the try statement and executes as follows:

• Opens two (2) text files on one line of code.
  • The file for reading, which contains a comma (,) character at the end to let Python know there is another file to open: open('orig_file.txt', 'r') as fp1
  • The second is for writing: open('new_file.txt', 'w') as fp2
• Then, the entire contents of orig_file.txt writes to new_file.txt.

If an error occurs during the above process, the code falls to the except statement, which retrieves and outputs the error message to the terminal.

💡 Note: The strip() function removes any trailing space characters. The newline character (\n) is added to place each iteration on its own line.

If successful, the contents of orig_file.txt and new_file.txt are identical.

Output of new_file.txt

30022425,Oliver,Grandmaster,2476
30022437,Cindy,Intermediate,1569
30022450,Leon,Authority,1967

Python open() Function – An 80/20 Guide by Example

Method 2: Open Files Across Multiple Lines with open() and Backslash (\)

Before Python version 3.10, and in use today, opening multiple files on one line could be awkward. To circumvent this, use the backslash (\) character as shown below to place the open() statements on separate lines.

try:
    with open('orig_file.txt', 'r') as fp1, \
         open('new_file3.txt', 'w')  as fp2:
            fp2.write(fp1.read())
except OSError as error:
    logging.error("This Error Occurred: %s", error)

This code snippet is wrapped in a try/except statement to catch errors. When this runs, it falls inside the try statement and executes as follows:

• Opens the first file using open('orig_file.txt', 'r') for reading and contains a backslash (\) character to let Python know there is another file to open.
• Opens the second file using open('new_file.txt', 'w') for writing.
• Then, the entire contents of orig_file.txt writes to new_file.txt.

If an error occurs during the above process, the code falls to the except statement, which retrieves and outputs the error message to the terminal.

💡 Note: The strip() function removes any trailing space characters. The newline character (\n) is added to place each iteration on its own line.

If successful, the contents of orig_file.txt and new_file.txt are identical.

Method 3: Open Multiple Text Files using Parenthesized Context Managers and open()

In Python version 3.10, Parenthesized Context Managers were added. This fixes a bug found in version 3.9, which did not support the use of parentheses across multiple lines of code. How Pythonic!

Here’s how they look in a short example:

try:
    with (
        open('orig_file.txt', 'r')  as fp1,
        open('new_file.txt',  'w') as fp2
    ):
        fp2.write(fp1.read())
except OSError as error:
    logging.error("This Error Occurred: %s", error)

This code snippet is wrapped in a try/except statement to catch errors. When this runs, it falls inside the try statement and executes as follows:

• Declare the opening of with and the opening bracket (with ()).
  • Opens orig_file.txt (open('orig_file.txt', 'r') as fp1,) for reading with a comma (,) to let Python know to expect another file.
  • Open new_file.txt (open('new_file.txt', 'w') as fp2) for writing.
• Closes the with statement by using ):.
• Then, the entire contents of orig_file.txt writes to new_file.txt.

If an error occurs during the above process, the code falls to the except statement, which retrieves and outputs the error message to the terminal.

If successful, the contents of orig_file.txt and new_file.txt are identical.

Method 4: Open Multiple Text Files using the os library and open()

This method calls in the os library (import os) which provides functionality to work with the Operating System. Specifically, for this example, file and folder manipulation.

import os 

os.chdir('files')
filelist = os.listdir(os.getcwd()) 

for i in filelist: 
    try: 
        with open(i, 'r') as fp: 
            for line in fp.readlines(): 
                print(line) 
    except OSError as error: 
        print('error %s', error) 

💡 Note: In this example, two (2) files are read in and output to the terminal.

This code snippet imports the os library to access the required functions.

For this example, we have two (2) text files located in our files directory:
file1.txt and file2.txt. To access and work with these files, we call os.chdir('files') to change to this folder (directory).

Next, we retrieve a list of all files residing in the current working directory
(os.listdir(os.getcwd()) and save the results to filelist.

IF we output the contents of filelist to the terminal, we would have the following: a list of all files in the current working directory in a List format.

['file1.txt', 'file2.txt']

This code snippet is wrapped in a try/except statement to catch errors. When this runs, it falls inside the try statement and executes as follows:

• Instantiates a for loop to traverse through each file in the List and does the following:
  • Opens the current file for reading.
  • Reads in this file one line at a time and output to the terminal.

If an error occurs during the above process, the code falls to the except statement, which retrieves and outputs the error message to the terminal.

Output

The contents of the two (2) files are:

Contents of File1.
Contents of File2.

Summary

These four (4) methods of how to multiple files should give you enough information to select the best one for your coding requirements.

Good Luck & Happy Coding!

Programmer Humor

❓ Question: How did the programmer die in the shower? ☠️

❗ Answer: They read the shampoo bottle instructions:
Lather. Rinse. Repeat.

At university, I found my love of writing and coding. Both of which I was able to use in my career.

During the past 15 years, I have held a number of positions such as:

In-house Corporate Technical Writer for various software programs such as Navision and Microsoft CRM
Corporate Trainer (staff of 30+)
Programming Instructor
Implementation Specialist for Navision and Microsoft CRM
Senior PHP Coder

How many files can Python open at once?

How do I read multiple files at a time in Python?

How do I run 3 Python files?

How do I open multiple files at once?