Practical spreadsheets and data
Compare two files in Python: check bytes, not names and dates
An original same-name, same-size, same-timestamp counterexample and a small local Python check with distinct equality, difference and error results.
Start here
Two copies named invoice.txt can have the same size and modification time yet contain different numbers. Compare their bytes before deciding whether they are interchangeable. This short check reads two local files and reports IDENTICAL, DIFFERENT or ERROR; it does not rewrite or delete them.
This guide is for people who already run Python from a terminal. The example was tested with Python 3.12.3 on Linux. Windows and macOS execution has not been tested for this article.
A difference that a file listing cannot show
Our original test used these files in separate folders:
original/invoice.txt
text
amount=100
11 bytes, including a final LF newline; both files have the same modification time.
copy/invoice.txt
text
amount=900
11 bytes, including a final LF newline; both files have the same modification time.
Why matching metadata is insufficient
We deliberately gave them matching timestamps. That creates a controlled counterexample; it is not evidence about how often copies become damaged.
Python's filecmp.cmp() normally allows a shortcut based on file type, size and modification time. In this fixture, the default returned True, while shallow=False returned False. This matches the documented behavior; it is not a Python defect.
Read the contents
Save the following as compare_files.py in a working folder. It reads binary data in chunks, so a different text encoding or newline style is not silently normalized.
compare_files.py
python
import argparse
from pathlib import Path
import sys
def equal_bytes(left, right):
for path in (left, right):
if not Path(path).is_file():
raise OSError(f"Not a readable regular file: {path}")
with open(left, "rb") as a, open(right, "rb") as b:
while True:
left_chunk = a.read(1024 * 1024)
right_chunk = b.read(1024 * 1024)
if left_chunk != right_chunk:
return False
if not left_chunk:
return True
def main():
parser = argparse.ArgumentParser(description="Compare two local files by bytes.")
parser.add_argument("left")
parser.add_argument("right")
args = parser.parse_args()
try:
same = equal_bytes(args.left, args.right)
except (OSError, ValueError) as error:
print(f"ERROR: {error}", file=sys.stderr)
return 2
print("IDENTICAL" if same else "DIFFERENT")
return 0 if same else 1
if __name__ == "__main__":
sys.exit(main())
Prepare the two files
Close applications that may edit either file, and let copying or synchronization finish. Make locally available copies first if a cloud client currently shows only placeholders. Then run the script against the two files you intend to compare:
Run the comparison
sh
python3 compare_files.py "original/invoice.txt" "copy/invoice.txt"
Check the output
Use your actual paths and keep the quotes when paths contain spaces. The script prints DIFFERENT for our two invoices. After copying the original's exact bytes into the second test file, it prints IDENTICAL, even when the modification times differ.
Interpret the result
- IDENTICAL: the byte sequences read in this comparison matched. Exit status is 0.
- DIFFERENT: the sequences differed. Exit status is 1. Keep both files until you know why; the result does not identify which version is correct.
- ERROR: a required file could not be checked. Exit status is 2. Fix the path or access problem and rerun; this result says nothing about equality.
Equal bytes do not prove a usable file
Two empty files are identical to one another. That does not make either a useful invoice. Likewise, byte-for-byte equality between two damaged files does not prove that either file opens correctly. After a copy check, open the document in its usual application and check the content that matters to you.
When “different” is expected
Our checks treated LF and CRLF line endings as different, and adding a UTF-8 byte-order mark also changed the result. These files can display similar text while containing different bytes. If your actual question is “does this PDF say the same thing?” or “do these CSV tables contain the same records?”, use the appropriate document or table comparison as a separate step. Do not automatically discard a file merely because its byte comparison differs.
The boundary of this check
This compares the normal file byte stream. It does not check permissions, extended attributes, folder completeness or whether a backup can be restored. It also does not freeze files while another program writes to them: compare stable files, or use an appropriate snapshot workflow for changing data.
A symbolic link is followed to its target. Giving the same file twice can also produce IDENTICAL; neither case proves that you have an independent second copy. Keep the two intended locations clear.
Our local experiment covered 14 cases, including a changed byte beyond two 1,048,576-byte chunks, binary bytes, spaces and Cyrillic filenames, empty files, a missing path, a directory and a named pipe. The script preserved input bytes, sizes and modification times in every completed case. The experiment ran with networking isolated and only a loopback interface. These are technical checks on artificial fixtures, not an evaluation of every filesystem or a guarantee of backup quality.