Python File I/O: Your Ultimate Guide to Reading and Writing Files
Welcome to Module 9 of our comprehensive Python tutorial series! In this module, we'll dive deep into File Input/Output (I/O), a fundamental skill for any Python programmer. Whether you're building a data analysis tool, a web application, or a simple script, the ability to interact with files is crucial. We'll cover everything from the basics of opening and closing files to working with popular file formats like CSV and JSON. Get ready to master file handling in Python!
Reading and Writing Files in Python
At its core, file I/O involves getting data from a file (reading) or putting data into a file (writing). Python provides a straightforward way to achieve this through built-in functions and methods.
Opening Files: open() Function
The open() function is your gateway to interacting with files. It takes at least two arguments: the filename and the mode. The mode specifies how you intend to interact with the file (read, write, append, etc.).
Note: Python file handling, open() function explained, file modes in Python.
Here are the most common modes you'll encounter:
'r' (Read Mode):
This is the default mode.
Used to read content from an existing file.
If the file doesn't exist, it will raise a FileNotFoundError.
Code Examples:
Beginner-Friendly: Reading a simple text file.
Python
# beginner_read_file.py
# This example shows how to open a file in read mode and print its content.
try:
file = open("my_first_file.txt", "r") # Open the file in read mode
content = file.read() # Read the entire content of the file
print(content) # Print the content to the console
file.close() # Always close the file when done!
except FileNotFoundError:
print("Error: 'my_first_file.txt' not found. Please create it first.")
Intermediate: Reading a specific number of characters.
Python
# intermediate_read_chars.py
# This example demonstrates reading a specific number of characters from a file.
try:
file = open("sample_data.txt", "r")
partial_content = file.read(10) # Read only the first 10 characters
print(f"First 10 characters: {partial_content}")
file.close()
except FileNotFoundError:
print("Error: 'sample_data.txt' not found.")
Advanced: Iterating through a file line by line (more efficient for large files).
Python
# advanced_read_lines_efficient.py
# This is a more memory-efficient way to read large files.
try:
with open("large_log.txt", "r") as file: # Using 'with' statement for automatic closing
for line_number, line in enumerate(file, 1):
print(f"Line {line_number}: {line.strip()}") # .strip() removes leading/trailing whitespace including newlines
except FileNotFoundError:
print("Error: 'large_log.txt' not found.")
Advanced: Reading and processing data based on a condition.
Python
# advanced_read_process_data.py
# This example reads a file and processes lines that meet a certain condition.
data_points = []
try:
with open("sensor_data.txt", "r") as file:
for line in file:
if "ERROR" in line:
print(f"Found error log: {line.strip()}")
else:
# Assuming each line is a number
try:
data_points.append(float(line.strip()))
except ValueError:
print(f"Skipping non-numeric line: {line.strip()}")
print(f"Processed {len(data_points)} valid data points.")
except FileNotFoundError:
print("Error: 'sensor_data.txt' not found.")
Expert: Reading a file and using regular expressions to extract specific patterns.
Python
# expert_read_regex.py
# This example demonstrates using regular expressions to extract specific patterns from a file.
import re
email_pattern = r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"
found_emails = []
try:
with open("user_data.txt", "r") as file:
content = file.read()
matches = re.findall(email_pattern, content)
for email in matches:
found_emails.append(email)
print("Found emails:")
for email in set(found_emails): # Use set to get unique emails
print(email)
except FileNotFoundError:
print("Error: 'user_data.txt' not found.")
'w' (Write Mode):
Used to write content to a file.
Caution: If the file exists, it will be truncated (emptied) before writing.
If the file doesn't exist, it will create a new file.
Note: Python write to file, open('w') mode, overwriting files.
Code Examples:
Beginner-Friendly: Writing a simple message to a new file.
Python
# beginner_write_file.py
# This example shows how to open a file in write mode and write text to it.
file = open("my_output.txt", "w") # Open the file in write mode
file.write("Hello, Python File I/O!\n") # Write a string to the file
file.write("This is a new line.") # Add another line
file.close() # Close the file to save changes
print("Content written to 'my_output.txt'")
Intermediate: Writing a list of items to a file, one item per line.
Python
# intermediate_write_list.py
# This example writes each item from a list to a new line in a file.
shopping_list = ["Apples", "Bananas", "Milk", "Bread"]
with open("shopping_list.txt", "w") as file: # Using 'with' for auto-closing
for item in shopping_list:
file.write(item + "\n") # Write each item followed by a newline character
print("Shopping list saved to 'shopping_list.txt'")
Advanced: Generating and writing structured data to a file.
Python
# advanced_write_structured_data.py
# This example generates and writes structured sensor data to a file.
import random
sensor_readings = []
for i in range(5):
temperature = round(20 + (random.random() * 10), 2) # Simulate temperature
humidity = round(40 + (random.random() * 20), 2) # Simulate humidity
sensor_readings.append(f"Reading {i+1}: Temp={temperature}°C, Humidity={humidity}%\n")
with open("sensor_log.txt", "w") as file:
file.writelines(sensor_readings) # Use writelines to write a list of strings
print("Sensor data logged to 'sensor_log.txt'")
Advanced: Creating a configuration file from a dictionary.
Python
# advanced_write_config.py
# This example shows how to write configuration settings from a dictionary to a file.
config_settings = {
"database_host": "localhost",
"database_port": "5432",
"username": "admin",
"debug_mode": "True"
}
with open("app_config.ini", "w") as file:
for key, value in config_settings.items():
file.write(f"{key}={value}\n")
print("Configuration saved to 'app_config.ini'")
Expert: Implementing a simple custom logger that writes messages with timestamps.
Python
# expert_custom_logger.py
# This example demonstrates a basic custom logger that appends timestamped messages.
from datetime import datetime
def log_message(message, log_file="application.log"):
"""Appends a timestamped message to a specified log file."""
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
with open(log_file, "a") as file: # 'a' mode for appending
file.write(f"[{timestamp}] {message}\n")
log_message("Application started.")
log_message("User 'john_doe' logged in.")
log_message("Data processing complete.", log_file="data_process.log")
print("Messages logged to respective files.")
'a' (Append Mode):
Used to add new content to the end of an existing file.
If the file doesn't exist, it will create a new file.
Existing content is preserved.
Note: Python append to file, add content to file, open('a') mode.
Code Examples:
Beginner-Friendly: Appending a new line to an existing file.
Python
# beginner_append_file.py
# This example shows how to open a file in append mode and add new content.
# Ensure 'my_output.txt' exists from a previous example or create it.
try:
with open("my_output.txt", "a") as file: # Open in append mode
file.write("This is an appended line.\n")
print("Content appended to 'my_output.txt'")
except FileNotFoundError:
print("Error: 'my_output.txt' not found. Please run the write example first.")
Intermediate: Logging events with timestamps.
Python
# intermediate_log_events.py
# This example appends timestamped event logs to a file.
from datetime import datetime
def log_event(event_description):
current_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
with open("event_log.txt", "a") as log:
log.write(f"[{current_time}] {event_description}\n")
log_event("User clicked 'submit' button.")
log_event("Data validation failed for form.")
print("Events logged to 'event_log.txt'")
Advanced: Appending unique entries to a list in a file.
Python
# advanced_append_unique.py
# This example appends unique user IDs to a file, avoiding duplicates.
def add_unique_user(user_id, filename="registered_users.txt"):
existing_users = set()
try:
with open(filename, "r") as f:
for line in f:
existing_users.add(line.strip())
except FileNotFoundError:
pass # File doesn't exist yet, no existing users
if user_id not in existing_users:
with open(filename, "a") as f:
f.write(user_id + "\n")
print(f"User '{user_id}' added.")
else:
print(f"User '{user_id}' already exists.")
add_unique_user("user123")
add_unique_user("user456")
add_unique_user("user123") # This should indicate it already exists
print("User registration complete.")
Advanced: Maintaining a running total in a file.
Python
# advanced_running_total.py
# This example demonstrates appending to a file to maintain a running total.
def update_total(amount, filename="daily_sales.txt"):
try:
with open(filename, "r") as f:
current_total = float(f.read().strip() or "0") # Read current total, default to 0
except (FileNotFoundError, ValueError):
current_total = 0.0 # Handle case where file is empty or not found
new_total = current_total + amount
with open(filename, "w") as f: # Overwrite with the new total
f.write(str(new_total))
print(f"Daily sales updated: +{amount}. New total: {new_total}")
update_total(100.50)
update_total(50.25)
update_total(75.00)
print("Daily sales recorded.")
Expert: Implementing a simple chat log for a game or application.
Python
# expert_chat_log.py
# This example simulates a simple chat log by appending messages to a file.
from datetime import datetime
def record_chat_message(username, message, log_file="chat_history.log"):
timestamp = datetime.now().strftime("[%H:%M:%S]")
with open(log_file, "a") as f:
f.write(f"{timestamp} <{username}> {message}\n")
record_chat_message("Alice", "Hello everyone!")
record_chat_message("Bob", "Hi Alice!")
record_chat_message("Charlie", "What's up?")
print("Chat messages recorded.")
'x' (Exclusive Creation Mode):
Used to create a new file, but only if it doesn't already exist.
If the file already exists, it will raise a FileExistsError.
This is useful when you want to ensure you're creating a truly new file and not overwriting an existing one.
Note: Python create new file, open('x') mode, avoid overwriting.
Code Examples:
Beginner-Friendly: Creating a new file exclusively.
Python
# beginner_exclusive_create.py
# This example attempts to create a new file using 'x' mode.
try:
with open("my_exclusive_file.txt", "x") as file:
file.write("This file was created exclusively.\n")
print("'my_exclusive_file.txt' created successfully.")
except FileExistsError:
print("Error: 'my_exclusive_file.txt' already exists. Cannot create exclusively.")
Intermediate: Generating a unique log file name to prevent conflicts.
Python
# intermediate_unique_log.py
# This example generates a unique log file name using a timestamp and 'x' mode.
from datetime import datetime
import os
def create_unique_log_file():
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
filename = f"log_{timestamp}.txt"
try:
with open(filename, "x") as file:
file.write(f"Log file created at {timestamp}.\n")
print(f"Unique log file '{filename}' created.")
except FileExistsError:
print(f"Error: A log file with name '{filename}' already exists (highly unlikely).")
create_unique_log_file()
print("Unique log file creation attempt finished.")
Advanced: Ensuring a configuration file is initialized only once.
Python
# advanced_init_config.py
# This example ensures a default configuration file is created only if it doesn't exist.
DEFAULT_CONFIG = """
[Settings]
api_key=YOUR_API_KEY
timeout=30
"""
def initialize_config(config_path="config.ini"):
try:
with open(config_path, "x") as file:
file.write(DEFAULT_CONFIG.strip())
print(f"Default config file '{config_path}' created.")
except FileExistsError:
print(f"Config file '{config_path}' already exists. Skipping initialization.")
initialize_config()
# You can call it again to see the FileExistsError message
initialize_config()
print("Config initialization attempts finished.")
Advanced: Creating a lock file to prevent multiple instances of a script.
Python
# advanced_lock_file.py
# This example uses 'x' mode to create a lock file, preventing multiple script instances.
import os
import time
LOCK_FILE = "script.lock"
def acquire_lock():
try:
with open(LOCK_FILE, "x") as f:
f.write(str(os.getpid())) # Write the process ID to the lock file
print("Lock acquired. Script is running.")
return True
except FileExistsError:
print("Another instance of the script is already running. Exiting.")
return False
def release_lock():
try:
os.remove(LOCK_FILE)
print("Lock released. Script finished.")
except OSError as e:
print(f"Error releasing lock: {e}")
if __name__ == "__main__":
if acquire_lock():
try:
# Simulate some work being done by the script
print("Doing important work...")
time.sleep(5)
finally:
release_lock()
else:
pass # Script exited because lock couldn't be acquired
Expert: Implementing a simple file-based semaphore using 'x' mode.
Python
# expert_file_semaphore.py
# This example demonstrates a basic file-based semaphore using 'x' mode for resource control.
import os
import time
SEMAPHORE_FILE = "resource.sem"
def acquire_semaphore(timeout=10):
start_time = time.time()
while time.time() - start_time < timeout:
try:
with open(SEMAPHORE_FILE, "x") as f:
f.write("locked")
print("Semaphore acquired.")
return True
except FileExistsError:
print("Waiting for semaphore...")
time.sleep(0.5) # Wait a bit before retrying
print("Failed to acquire semaphore within timeout.")
return False
def release_semaphore():
try:
os.remove(SEMAPHORE_FILE)
print("Semaphore released.")
except OSError as e:
print(f"Error releasing semaphore: {e}")
if __name__ == "__main__":
# Example usage:
if acquire_semaphore(timeout=3):
try:
print("Critical section: Accessing shared resource...")
time.sleep(2) # Simulate work on shared resource
finally:
release_semaphore()
else:
print("Could not access resource.")
'rb' (Read Binary Mode):
Used to read non-text files (images, audio, executables).
Content is read as bytes, not strings.
Note: Python read binary file, open('rb') for images, byte data.
Code Examples:
Beginner-Friendly: Reading a small image file's bytes.
Python
# beginner_read_binary_image.py
# This example reads a small image file in binary mode.
# Create a dummy 'tiny_image.png' or use an existing small image for testing.
try:
with open("tiny_image.png", "rb") as file:
image_bytes = file.read()
print(f"Read {len(image_bytes)} bytes from 'tiny_image.png'.")
# print(image_bytes[:20]) # Print first 20 bytes to see the raw data
except FileNotFoundError:
print("Error: 'tiny_image.png' not found. Please provide a small image file.")
Intermediate: Reading and displaying header bytes of a common file type.
Python
# intermediate_read_file_header.py
# This example reads the first few bytes of a file to identify its type (e.g., PDF, JPEG).
def identify_file_type(filepath):
try:
with open(filepath, "rb") as f:
header = f.read(4) # Read the first 4 bytes
if header == b'%PDF':
print(f"'{filepath}' is likely a PDF file.")
elif header.startswith(b'\xFF\xD8\xFF'): # JPEG magic number
print(f"'{filepath}' is likely a JPEG image.")
else:
print(f"Cannot identify type for '{filepath}' from header: {header}")
except FileNotFoundError:
print(f"Error: '{filepath}' not found.")
# Create dummy files for testing or use existing ones
# with open("dummy.pdf", "wb") as f: f.write(b'%PDF-1.4\n')
# with open("dummy.jpg", "wb") as f: f.write(b'\xFF\xD8\xFF\xE0\x00\x10JFIF')
identify_file_type("dummy.pdf")
identify_file_type("dummy.jpg")
identify_file_type("my_output.txt") # A text file
Advanced: Copying a binary file block by block.
Python
# advanced_copy_binary.py
# This example efficiently copies a binary file by reading and writing in chunks.
def copy_binary_file(source_path, destination_path, chunk_size=4096):
try:
with open(source_path, "rb") as source_file:
with open(destination_path, "wb") as destination_file:
while True:
chunk = source_file.read(chunk_size)
if not chunk: # End of file
break
destination_file.write(chunk)
print(f"Successfully copied '{source_path}' to '{destination_path}'.")
except FileNotFoundError:
print(f"Error: Source file '{source_path}' not found.")
except Exception as e:
print(f"An error occurred during copying: {e}")
# Create a dummy binary file for testing (e.g., a small image or audio file)
# Or copy an existing file from your system.
# with open("source_binary.bin", "wb") as f: f.write(b'\x00\x01\x02\x03' * 1000)
copy_binary_file("source_binary.bin", "destination_binary_copy.bin")
Advanced: Calculating the MD5 checksum of a binary file.
Python
# advanced_md5_checksum.py
# This example calculates the MD5 hash of a binary file to verify its integrity.
import hashlib
def calculate_md5(filepath, chunk_size=4096):
md5_hash = hashlib.md5()
try:
with open(filepath, "rb") as f:
while chunk := f.read(chunk_size): # Python 3.8+ walrus operator
md5_hash.update(chunk)
return md5_hash.hexdigest()
except FileNotFoundError:
print(f"Error: '{filepath}' not found.")
return None
except Exception as e:
print(f"An error occurred: {e}")
return None
file_to_check = "source_binary.bin" # Use the file from the previous example
checksum = calculate_md5(file_to_check)
if checksum:
print(f"MD5 checksum of '{file_to_check}': {checksum}")
Expert: Reading specific byte ranges from a binary file (e.g., for parsing custom file formats).
Python
# expert_read_byte_range.py
# This example reads specific byte ranges from a binary file, useful for parsing headers/footers.
# Simulate a custom file format with a header and data section
# Header: 4 bytes (version), Data: remaining bytes
def create_dummy_custom_file(filename="custom_data.bin"):
with open(filename, "wb") as f:
f.write(b'\x01\x02\x03\x04') # Version bytes
f.write(b'This is the actual data content.') # Data content
print(f"Dummy custom file '{filename}' created.")
def read_custom_file(filepath):
try:
with open(filepath, "rb") as f:
# Read the version header (first 4 bytes)
version_bytes = f.read(4)
print(f"Version bytes: {version_bytes.hex()}")
# Read the rest of the file as data
data_bytes = f.read()
print(f"Data content: {data_bytes.decode('utf-8')}") # Decode if it's text
except FileNotFoundError:
print(f"Error: '{filepath}' not found.")
except Exception as e:
print(f"An error occurred: {e}")
create_dummy_custom_file()
read_custom_file("custom_data.bin")
'wb' (Write Binary Mode):
Used to write non-text files (images, audio, executables).
Content must be provided as bytes.
Similar to 'w', it will truncate the file if it exists.
Note: Python write binary file, save image data, open('wb') for bytes.
Code Examples:
Beginner-Friendly: Writing a few bytes to a binary file.
Python
# beginner_write_binary.py
# This example writes a simple sequence of bytes to a binary file.
binary_data = b'\x01\x02\x03\x04\x05' # Bytes object
with open("simple_binary.bin", "wb") as file:
file.write(binary_data)
print("Binary data written to 'simple_binary.bin'")
Intermediate: Creating a small, blank image file.
Python
# intermediate_create_blank_image.py
# This example creates a very basic (and usually unviewable) blank image file.
# This is highly simplified and not a real image format, just for demonstration.
# Real image formats (like BMP, PNG, JPEG) have complex headers.
dummy_image_data = b'\x42\x4D' # Simple BMP magic number (start of a BMP file)
dummy_image_data += b'\x00\x00\x00\x00' * 10 # Some more dummy bytes
with open("blank_image.bmp", "wb") as file:
file.write(dummy_image_data)
print("Dummy blank_image.bmp created.")
Advanced: Saving received network data (e.g., from an API) directly to a binary file.
Python
# advanced_save_network_data.py
# This example simulates saving binary data (e.g., an image) received over a network.
# In a real scenario, you'd fetch this using 'requests' library or similar.
import requests # You might need to 'pip install requests'
def download_image(url, filename):
try:
response = requests.get(url, stream=True) # stream=True for large files
response.raise_for_status() # Raise HTTPError for bad responses (4xx or 5xx)
with open(filename, "wb") as file:
for chunk in response.iter_content(chunk_size=8192): # Iterate in chunks
if chunk: # Filter out keep-alive new chunks
file.write(chunk)
print(f"Successfully downloaded '{url}' to '{filename}'.")
except requests.exceptions.RequestException as e:
print(f"Error downloading image: {e}")
except Exception as e:
print(f"An unexpected error occurred: {e}")
# Example: Download a public domain image
image_url = "https://www.python.org/static/img/python-logo.png"
download_image(image_url, "python_logo.png")
Advanced: Creating a simple archive file by concatenating multiple files.
Python
# advanced_create_archive.py
# This example concatenates several text files into a single binary "archive".
# This is very basic and doesn't include metadata for extraction.
# Create some dummy files first:
# with open("file1.txt", "w") as f: f.write("Content of file 1.")
# with open("file2.txt", "w") as f: f.write("Content of file 2.")
# with open("file3.txt", "w") as f: f.write("Content of file 3.")
files_to_archive = ["file1.txt", "file2.txt", "file3.txt"]
archive_name = "my_archive.bin"
with open(archive_name, "wb") as archive_file:
for filepath in files_to_archive:
try:
with open(filepath, "rb") as current_file:
content = current_file.read()
# Simple delimiter (e.g., ---FILE_END---) - not robust for real archives
archive_file.write(content + b'\n---END_OF_FILE---\n')
print(f"Added '{filepath}' to archive.")
except FileNotFoundError:
print(f"Warning: '{filepath}' not found, skipping.")
print(f"Archive '{archive_name}' created.")
Expert: Writing raw pixel data to create a custom uncompressed image format.
Python
# expert_write_raw_pixels.py
# This example writes raw pixel data to create a very simple (and likely unviewable by default)
# uncompressed image file. This would be interpreted by specific image viewers.
# Let's create a 3x3 simple RGB image (red, green, blue pixels)
# Each pixel is 3 bytes (R, G, B)
# Total bytes = width * height * 3
width = 3
height = 3
pixel_data = bytearray()
# Row 1: Red, Green, Blue
pixel_data.extend(b'\xFF\x00\x00') # Red
pixel_data.extend(b'\x00\xFF\x00') # Green
pixel_data.extend(b'\x00\x00\xFF') # Blue
# Row 2: Cyan, Magenta, Yellow
pixel_data.extend(b'\x00\xFF\xFF') # Cyan
pixel_data.extend(b'\xFF\x00\xFF') # Magenta
pixel_data.extend(b'\xFF\xFF\x00') # Yellow
# Row 3: Black, White, Gray
pixel_data.extend(b'\x00\x00\x00') # Black
pixel_data.extend(b'\xFF\xFF\xFF') # White
pixel_data.extend(b'\x80\x80\x80') # Gray
output_filename = "simple_rgb_image.raw"
with open(output_filename, "wb") as f:
f.write(pixel_data)
print(f"Raw RGB image data written to '{output_filename}'.")
print(f"This file contains {width}x{height} pixels, 3 bytes per pixel.")
print("You would need a specific viewer that understands raw RGB data to open it.")
Closing Files: close() Method
After you're done with file operations, it's crucial to close the file using the close() method. This releases the file resource back to the operating system and ensures that any buffered writes are actually saved to the disk. Failing to close files can lead to data loss or resource leaks.
Note: Close file Python, file.close() importance, release file resources.
Code Examples:
Beginner-Friendly: Explicitly closing a file after writing.
Python
# beginner_close_file.py
# This example demonstrates the importance of explicitly closing a file.
file = open("explicit_close.txt", "w")
file.write("This content needs to be saved by closing the file.\n")
print("Content written to buffer.")
file.close() # Crucial: Flushes the buffer and closes the file
print("File 'explicit_close.txt' closed.")
Intermediate: Handling potential errors during file operations and ensuring closure.
Python
# intermediate_close_with_error_handling.py
# This example shows how to ensure a file is closed even if an error occurs.
file = None # Initialize to None
try:
file = open("error_prone.txt", "w")
file.write("Line 1\n")
# Simulate an error
# result = 1 / 0
file.write("Line 2\n") # This line might not be written if error above
except ZeroDivisionError:
print("An error occurred (division by zero).")
finally:
if file: # Check if the file object was successfully created
file.close()
print("File 'error_prone.txt' closed in finally block.")
else:
print("File could not be opened.")
Advanced: Counting lines in a large file and ensuring it's closed.
Python
# advanced_count_lines_close.py
# This example counts lines in a file, ensuring the file is closed afterward.
def count_lines(filepath):
line_count = 0
file_obj = None # Initialize to None for error handling
try:
file_obj = open(filepath, "r")
for _ in file_obj:
line_count += 1
print(f"'{filepath}' has {line_count} lines.")
return line_count
except FileNotFoundError:
print(f"Error: '{filepath}' not found.")
return -1
finally:
if file_obj:
file_obj.close()
print(f"File '{filepath}' explicitly closed.")
# Create a dummy file for testing
with open("lines_to_count.txt", "w") as f:
f.write("Line A\nLine B\nLine C\n")
count_lines("lines_to_count.txt")
count_lines("non_existent_file.txt") # Test with a file that doesn't exist
Advanced: Implementing a custom file writer function that always closes.
Python
# advanced_custom_writer_close.py
# This example wraps file writing logic in a function that guarantees file closure.
def write_to_log(message, filename="app_audit.log"):
file_handle = None
try:
file_handle = open(filename, "a")
file_handle.write(f"[{datetime.now().strftime('%Y-%m-%d %H:%M:%S')}] {message}\n")
print(f"Message logged to '{filename}'.")
except IOError as e:
print(f"Error writing to log file: {e}")
finally:
if file_handle:
file_handle.close()
# print("Log file handle closed.") # Can add for debug if needed
from datetime import datetime
write_to_log("User 'alice' accessed dashboard.")
write_to_log("Attempted to connect to database.")
Expert: Analyzing a corrupt file, reading parts, and ensuring proper closure.
Python
# expert_analyze_corrupt_file_close.py
# This example simulates analyzing a partially written or corrupt file, ensuring closure.
import os
def create_partially_written_file(filename="corrupt_data.txt"):
with open(filename, "w") as f:
f.write("Line 1\n")
f.write("Line 2\n")
# Simulate a crash here - file is not properly closed or fully written
print(f"Created '{filename}' (simulated corrupt/incomplete file).")
def analyze_file_robustly(filepath):
file_object = None
try:
file_object = open(filepath, "r")
lines = file_object.readlines()
print(f"Read {len(lines)} lines from '{filepath}'.")
for i, line in enumerate(lines):
print(f" Line {i+1}: {line.strip()}")
except FileNotFoundError:
print(f"Error: '{filepath}' not found.")
except IOError as e:
print(f"An I/O error occurred while reading '{filepath}': {e}")
finally:
if file_object:
file_object.close()
print(f"File '{filepath}' robustly closed.")
else:
print(f"File '{filepath}' could not be opened for analysis.")
create_partially_written_file()
analyze_file_robustly("corrupt_data.txt")
The with Statement (Context Manager)
The with statement is the recommended way to handle file operations in Python. It's a context manager that ensures files are properly closed automatically, even if errors occur. This eliminates the need for explicit try...finally blocks with close(). It makes your code cleaner, safer, and more Pythonic.
Note: Python with statement, context manager files, automatic file closing, best practice file I/O.
Code Examples:
Beginner-Friendly: Writing a file using with statement.
Python
# beginner_with_statement_write.py
# This example shows the simplest way to write a file using the 'with' statement.
with open("with_example.txt", "w") as file:
file.write("This line is written using the 'with' statement.\n")
file.write("The file will automatically close when this block exits.")
print("File 'with_example.txt' written and automatically closed.")
Intermediate: Reading a file using with statement and iterating lines.
Python
# intermediate_with_statement_read.py
# This example demonstrates reading a file line by line using 'with'.
try:
with open("with_example.txt", "r") as f:
print("Content of 'with_example.txt':")
for line_num, line in enumerate(f, 1):
print(f"{line_num}: {line.strip()}")
except FileNotFoundError:
print("Error: 'with_example.txt' not found. Run the write example first.")
Advanced: Chaining multiple with statements for file copying.
Python
# advanced_with_chaining.py
# This example efficiently copies content from one file to another using chained 'with' statements.
def copy_file_with_chaining(source, destination):
try:
with open(source, "r") as src_file, open(destination, "w") as dest_file:
content = src_file.read()
dest_file.write(content)
print(f"File '{source}' copied to '{destination}' successfully.")
except FileNotFoundError:
print(f"Error: Source file '{source}' not found.")
except Exception as e:
print(f"An error occurred during copy: {e}")
# Create a source file for testing
with open("source_for_copy.txt", "w") as f:
f.write("Hello from source!\nLine 2 from source.")
copy_file_with_chaining("source_for_copy.txt", "copied_file.txt")
Advanced: Processing data from a file and writing results to another within with.
Python
# advanced_process_and_write_with.py
# This example reads numbers from one file, doubles them, and writes to another.
# Create input file
with open("numbers.txt", "w") as f:
f.write("10\n25\n5\n")
try:
with open("numbers.txt", "r") as infile, open("doubled_numbers.txt", "w") as outfile:
for line in infile:
try:
number = int(line.strip())
outfile.write(str(number * 2) + "\n")
except ValueError:
print(f"Skipping invalid number: {line.strip()}")
print("Numbers processed and doubled_numbers.txt created.")
except FileNotFoundError:
print("Error: 'numbers.txt' not found.")
except Exception as e:
print(f"An error occurred: {e}")
Expert: Implementing a custom context manager for resource management (beyond files).
Python
# expert_custom_context_manager.py
# This example demonstrates how to create your own context manager using a class,
# mimicking the behavior of 'with open()'.
class ResourceManager:
def __init__(self, resource_name):
self.resource_name = resource_name
self.is_open = False
print(f"Initializing ResourceManager for '{self.resource_name}'")
def __enter__(self):
print(f"Entering context: Opening '{self.resource_name}'")
# Simulate resource acquisition (e.g., database connection, network socket)
self.is_open = True
return self # Return the instance itself, which is assigned to 'as var'
def __exit__(self, exc_type, exc_val, exc_tb):
print(f"Exiting context: Closing '{self.resource_name}'")
# Simulate resource release
self.is_open = False
if exc_type:
print(f"An exception occurred: {exc_type.__name__}: {exc_val}")
return False # Let Python propagate the exception if one occurred
# Usage of the custom context manager
with ResourceManager("Database Connection") as db:
print(f"Inside context: Is DB open? {db.is_open}")
# Simulate some work
# raise ValueError("Simulating an error inside context")
print(f"Outside context: Is DB open? {db.is_open}")
print("Custom context manager demonstration complete.")
Reading File Content
Once a file is opened, you have several methods to read its content. The choice depends on whether you want to read the entire file, one line at a time, or all lines into a list.
Note: Read file content Python, read() vs readline() vs readlines(), Python file reading methods.
read()
Reads the entire content of the file as a single string (in text mode) or bytes (in binary mode).
You can also specify an optional argument to read a certain number of characters/bytes.
Code Examples:
Beginner-Friendly: Reading the entire content of a small text file.
Python
# beginner_read_all.py
# Create a dummy file for reading
with open("full_text.txt", "w") as f:
f.write("This is the first line.\n")
f.write("This is the second line of text.\n")
f.write("And the final line.")
with open("full_text.txt", "r") as file:
content = file.read() # Reads everything
print("Entire file content:")
print(content)
print("Reading with read() complete.")
Intermediate: Reading a specific number of characters from the beginning of a file.
Python
# intermediate_read_n_chars.py
with open("full_text.txt", "r") as file:
first_20_chars = file.read(20) # Read only the first 20 characters
print(f"First 20 characters: '{first_20_chars}'")
# The file pointer is now at the 20th character
remaining_content = file.read() # Reads from the 21st character to the end
print(f"Remaining content: '{remaining_content}'")
print("Reading specified characters with read() complete.")
Advanced: Reading an entire configuration file and parsing it (simple example).
Python
# advanced_read_config_file.py
# Create a dummy config file
with open("settings.conf", "w") as f:
f.write("DEBUG=True\n")
f.write("HOST=localhost\n")
f.write("PORT=8080\n")
config = {}
try:
with open("settings.conf", "r") as file:
full_config_string = file.read()
for line in full_config_string.split('\n'):
line = line.strip()
if line and '=' in line:
key, value = line.split('=', 1)
config[key.strip()] = value.strip()
print("Parsed configuration:")
for key, value in config.items():
print(f" {key}: {value}")
except FileNotFoundError:
print("Error: 'settings.conf' not found.")
print("Configuration reading with read() complete.")
Advanced: Processing a large CSV file by reading it entirely (caution for very large files).
Python
# advanced_read_large_csv_whole.py
# This approach is suitable for moderately large files that fit into memory.
# For very large files, iterate line by line or use the csv module.
# Create a dummy CSV file
with open("products.csv", "w") as f:
f.write("ID,Name,Price\n")
f.write("1,Laptop,1200.00\n")
f.write("2,Mouse,25.50\n")
f.write("3,Keyboard,75.00\n")
products = []
try:
with open("products.csv", "r") as file:
content = file.read()
lines = content.strip().split('\n')
header = lines[0].split(',')
for line in lines[1:]:
values = line.split(',')
if len(values) == len(header):
product = dict(zip(header, values))
products.append(product)
print("Products from CSV (using read()):")
for product in products:
print(product)
except FileNotFoundError:
print("Error: 'products.csv' not found.")
print("CSV reading with read() complete.")
Expert: Implementing a simple templating engine by reading a template file.
Python
# expert_simple_templating.py
# This example reads a template file and replaces placeholders.
# Create a template file
with open("template.html", "w") as f:
f.write("<h1>Welcome, {{username}}!</h1>\n")
f.write("<p>Your email is: {{email}}.</p>\n")
f.write("<p>Current date: {{date}}.</p>")
def render_template(template_path, context):
try:
with open(template_path, "r") as file:
template_content = file.read()
rendered_content = template_content
for key, value in context.items():
placeholder = "{{" + key + "}}"
rendered_content = rendered_content.replace(placeholder, str(value))
return rendered_content
except FileNotFoundError:
return f"Error: Template '{template_path}' not found."
context_data = {
"username": "Alice Smith",
"email": "alice@example.com",
"date": datetime.now().strftime("%Y-%m-%d")
}
from datetime import datetime
html_output = render_template("template.html", context_data)
print("\nRendered HTML:")
print(html_output)
with open("output.html", "w") as f:
f.write(html_output)
print("\nRendered HTML saved to 'output.html'.")
readline()
Reads a single line from the file.
Returns an empty string ('') when the end of the file is reached.
Includes the newline character (\n) if present in the line.
Code Examples:
Beginner-Friendly: Reading a file line by line using a loop.
Python
# beginner_readline_loop.py
# Create a dummy file for reading
with open("line_by_line.txt", "w") as f:
f.write("First entry\n")
f.write("Second entry\n")
f.write("Third entry\n")
with open("line_by_line.txt", "r") as file:
print("Reading line by line with readline():")
line1 = file.readline()
print(f"Line 1: {line1.strip()}") # .strip() removes newline
line2 = file.readline()
print(f"Line 2: {line2.strip()}")
line3 = file.readline()
print(f"Line 3: {line3.strip()}")
line4 = file.readline() # Will be an empty string at EOF
print(f"Line 4 (EOF check): '{line4}' (empty string means end of file)")
print("Reading with readline() complete.")
Intermediate: Processing a log file, skipping comments and empty lines.
Python
# intermediate_process_log_readline.py
# Create a dummy log file
with open("app_log.log", "w") as f:
f.write("# This is a comment\n")
f.write("INFO: User 'admin' logged in.\n")
f.write("\n") # Empty line
f.write("WARNING: Low disk space.\n")
f.write("ERROR: Database connection failed.\n")
print("\nProcessing app_log.log:")
with open("app_log.log", "r") as log_file:
while True:
line = log_file.readline()
if not line: # Check for end of file
break
clean_line = line.strip()
if clean_line and not clean_line.startswith('#'): # Skip empty lines and comments
print(f"Processed: {clean_line}")
print("Log file processing with readline() complete.")
Advanced: Implementing a custom "head" command to read the first N lines.
Python
# advanced_head_command.py
def head(filepath, num_lines=10):
"""Reads and prints the first num_lines from a file."""
print(f"\n--- Head of '{filepath}' (first {num_lines} lines) ---")
try:
with open(filepath, "r") as file:
for i in range(num_lines):
line = file.readline()
if not line:
break # Reached end of file before num_lines
print(line.strip())
print("--------------------------------------------------")
except FileNotFoundError:
print(f"Error: '{filepath}' not found.")
# Create a longer file for testing
with open("long_text.txt", "w") as f:
for i in range(20):
f.write(f"This is line number {i+1}.\n")
head("long_text.txt", 5)
head("non_existent.txt")
head("long_text.txt", 15)
print("Head command simulation with readline() complete.")
Advanced: Reading delimited data line by line for efficient parsing.
Python
# advanced_read_delimited_readline.py
# Create a dummy file with pipe-delimited data
with open("data.psv", "w") as f:
f.write("John|Doe|30|New York\n")
f.write("Jane|Smith|25|London\n")
f.write("Peter|Jones|40|Paris\n")
users = []
print("\nParsing pipe-separated data with readline():")
try:
with open("data.psv", "r") as file:
while True:
line = file.readline()
if not line:
break
parts = line.strip().split('|')
if len(parts) == 4:
user = {
"first_name": parts[0],
"last_name": parts[1],
"age": int(parts[2]),
"city": parts[3]
}
users.append(user)
for user in users:
print(user)
except FileNotFoundError:
print("Error: 'data.psv' not found.")
print("Delimited data reading with readline() complete.")
Expert: Implementing a custom file iterator using readline() and __next__ for lazy loading.
Python
# expert_custom_file_iterator.py
# This class makes a file object behave like a custom iterator, useful for
# handling very large files without loading all lines into memory at once.
class LazyFileReader:
def __init__(self, filepath):
self.filepath = filepath
self._file = None
def __iter__(self):
# Open the file when iteration starts
self._file = open(self.filepath, 'r')
return self
def __next__(self):
if not self._file:
raise StopIteration # Or a more descriptive error if not initialized
line = self._file.readline()
if not line:
self._file.close() # Close the file when iteration ends
raise StopIteration
return line.strip()
def __del__(self):
# Ensure file is closed if object is garbage collected, as a fallback
if self._file and not self._file.closed:
self._file.close()
print(f"(Cleanup: File '{self.filepath}' closed by __del__)")
# Create a large dummy file
with open("big_data.txt", "w") as f:
for i in range(1000):
f.write(f"Data entry {i+1}\n")
print("\nUsing custom LazyFileReader:")
try:
for i, data_line in enumerate(LazyFileReader("big_data.txt")):
if i < 5 or i > 995: # Print first 5 and last 5 lines
print(f"Lazy Read: {data_line}")
elif i == 5:
print("...")
if i == 999:
break # Stop early to show file still closes
except FileNotFoundError:
print("Error: 'big_data.txt' not found.")
print("Lazy file reading with custom iterator complete.")
readlines()
Reads all lines from the file and returns them as a list of strings.
Each string in the list includes the newline character (\n).
This method is convenient but can consume a lot of memory for very large files, as it loads the entire file into RAM.
Code Examples:
Beginner-Friendly: Reading all lines into a list and printing them.
Python
# beginner_readlines_all.py
# Create a dummy file
with open("all_lines.txt", "w") as f:
f.write("Line one\n")
f.write("Line two\n")
f.write("Line three") # No newline on purpose
with open("all_lines.txt", "r") as file:
lines = file.readlines() # Reads all lines into a list
print("Lines read by readlines():")
for i, line in enumerate(lines):
print(f"Line {i+1}: '{line.strip()}'") # Use .strip() to remove the newline
print("Reading with readlines() complete.")
Intermediate: Processing a list of names from a file, filtering and sorting.
Python
# intermediate_process_names_readlines.py
# Create a dummy names file
with open("names.txt", "w") as f:
f.write("Alice\n")
f.write("Bob\n")
f.write("Charlie\n")
f.write("david\n") # Lowercase
f.write("Eve\n")
filtered_names = []
print("\nProcessing names with readlines():")
try:
with open("names.txt", "r") as file:
all_lines = file.readlines()
for line in all_lines:
name = line.strip()
if name and name[0].isupper(): # Filter names starting with uppercase
filtered_names.append(name)
filtered_names.sort() # Sort the names
print("Filtered and sorted names:")
for name in filtered_names:
print(name)
except FileNotFoundError:
print("Error: 'names.txt' not found.")
print("Name processing with readlines() complete.")
Advanced: Parsing structured data where each block is separated by blank lines.
Python
# advanced_parse_blocks_readlines.py
# Create a dummy file with data blocks
with open("data_blocks.txt", "w") as f:
f.write("Item: Apple\n")
f.write("Price: 1.50\n")
f.write("Category: Fruit\n")
f.write("\n") # Blank line separates blocks
f.write("Item: Laptop\n")
f.write("Price: 1200.00\n")
f.write("Category: Electronics\n")
items = []
current_item = {}
print("\nParsing data blocks with readlines():")
try:
with open("data_blocks.txt", "r") as file:
all_lines = file.readlines()
for line in all_lines:
clean_line = line.strip()
if clean_line:
if ":" in clean_line:
key, value = clean_line.split(':', 1)
current_item[key.strip()] = value.strip()
else: # Blank line encountered, means end of current item block
if current_item:
items.append(current_item)
current_item = {} # Reset for next item
if current_item: # Add the last item if file doesn't end with a blank line
items.append(current_item)
for item in items:
print(item)
except FileNotFoundError:
print("Error: 'data_blocks.txt' not found.")
print("Data block parsing with readlines() complete.")
Advanced: Counting word frequency in a file using readlines() (for smaller files).
Python
# advanced_word_frequency_readlines.py
from collections import Counter
import re
# Create a dummy text file
with open("sample_article.txt", "w") as f:
f.write("Python is a powerful language. It is easy to learn.\n")
f.write("Many people use Python for data science and web development.\n")
f.write("Python's community is large and supportive.")
word_counts = Counter()
print("\nCounting word frequency with readlines():")
try:
with open("sample_article.txt", "r") as file:
all_lines = file.readlines()
for line in all_lines:
# Convert to lowercase and remove punctuation
cleaned_line = re.sub(r'[^\w\s]', '', line.lower())
words = cleaned_line.split()
word_counts.update(words)
for word, count in word_counts.most_common(5): # Top 5 most common words
print(f"'{word}': {count}")
except FileNotFoundError:
print("Error: 'sample_article.txt' not found.")
print("Word frequency counting with readlines() complete.")
Expert: Implementing a simple in-memory database by loading data with readlines().
Python
# expert_in_memory_db_readlines.py
# This simulates loading a small dataset from a file into memory for quick queries.
# Each line represents a record in a simple format: ID,Name,Status
# Create a dummy data file
with open("user_records.csv", "w") as f:
f.write("101,Alice,Active\n")
f.write("102,Bob,Inactive\n")
f.write("103,Charlie,Active\n")
class InMemoryUserDB:
def __init__(self, filepath):
self.users = {}
self.filepath = filepath
self._load_data()
def _load_data(self):
try:
with open(self.filepath, "r") as f:
all_lines = f.readlines()
for line in all_lines:
parts = line.strip().split(',')
if len(parts) == 3:
user_id, name, status = parts
self.users[user_id] = {"name": name, "status": status}
print(f"Loaded {len(self.users)} users into in-memory DB.")
except FileNotFoundError:
print(f"Warning: Database file '{self.filepath}' not found. Starting with empty DB.")
except Exception as e:
print(f"Error loading data: {e}")
def get_user(self, user_id):
return self.users.get(user_id)
def get_active_users(self):
return [user for user_id, user in self.users.items() if user["status"] == "Active"]
user_db = InMemoryUserDB("user_records.csv")
print("\nQuerying in-memory DB:")
user_101 = user_db.get_user("101")
if user_101:
print(f"User 101: {user_101}")
else:
print("User 101 not found.")
active_users = user_db.get_active_users()
print("Active Users:")
for user in active_users:
print(user)
print("In-memory database simulation with readlines() complete.")
Writing to Files
Just as with reading, Python provides different methods for writing content to files, allowing you to choose between writing a single string or a list of strings.
Note: Write data to file Python, save text to file, Python file writing methods.
write()
Writes a string (in text mode) or bytes (in binary mode) to the file.
It does not automatically add a newline character (\n) at the end of the string. You must add it explicitly if you want each write to appear on a new line.
Returns the number of characters/bytes written.
Code Examples:
Beginner-Friendly: Writing multiple lines of text with explicit newlines.
Python
# beginner_write_multiple_lines.py
with open("my_story.txt", "w") as file:
file.write("Chapter 1: The Beginning\n") # Add newline explicitly
file.write("It was a dark and stormy night.\n")
file.write("Suddenly, a light appeared in the distance.")
# No newline here, so next write would be on same line if file wasn't closed
print("Content written to 'my_story.txt' using write().")
Intermediate: Writing formatted data to a report file.
Python
# intermediate_write_report.py
product_name = "Wireless Headphones"
price = 99.99
stock = 150
report_date = datetime.now().strftime("%Y-%m-%d")
from datetime import datetime
with open("product_report.txt", "w") as report_file:
report_file.write(f"--- Product Inventory Report ({report_date}) ---\n")
report_file.write(f"Product: {product_name}\n")
report_file.write(f"Price: ${price:.2f}\n") # Format to 2 decimal places
report_file.write(f"Current Stock: {stock} units\n")
if stock < 20:
report_file.write("Status: Low Stock - Order More!\n")
else:
report_file.write("Status: Good Stock Level.\n")
print("Product report generated in 'product_report.txt' using write().")
Advanced: Implementing a custom data serializer (simple key-value pairs).
Python
# advanced_custom_serializer_write.py
def serialize_data_to_file(data, filepath):
"""Writes a dictionary of data to a file in a custom key-value format."""
try:
with open(filepath, "w") as f:
for key, value in data.items():
f.write(f"{key}={value}\n")
print(f"Data serialized to '{filepath}'.")
except IOError as e:
print(f"Error writing serialized data: {e}")
user_profile = {
"user_id": "u001",
"username": "coder_gal",
"last_login": "2025-06-13 14:00:00",
"email_verified": "True"
}
serialize_data_to_file(user_profile, "user_profile.dat")
Advanced: Writing CSV data manually with write() (less robust than csv module).
Python
# advanced_write_csv_manual.py
# While the 'csv' module is preferred, this demonstrates manual CSV writing with 'write()'.
data_rows = [
["Name", "Age", "City"],
["Alice", 30, "New York"],
["Bob", 24, "London"]
]
with open("manual_data.csv", "w") as file:
for row in data_rows:
# Join elements with comma, convert non-strings to string
file.write(",".join(str(item) for item in row) + "\n")
print("CSV data manually written to 'manual_data.csv' using write().")
Expert: Building a simple bytecode generator that writes byte sequences.
Python
# expert_bytecode_generator_write.py
# This example simulates generating simple bytecode instructions and writing them to a binary file.
# This is highly simplified and for demonstration only.
# Instruction format: Opcode (1 byte), Arg1 (1 byte), Arg2 (1 byte)
# Opcodes
OP_LOAD = b'\x01'
OP_ADD = b'\x02'
OP_STORE = b'\x03'
OP_HALT = b'\xFF'
def generate_bytecode():
bytecode = bytearray()
# LOAD 10 into register A
bytecode.extend(OP_LOAD)
bytecode.extend(b'\x0A') # Arg1: Value 10
bytecode.extend(b'\x00') # Arg2: Unused
# LOAD 20 into register B
bytecode.extend(OP_LOAD)
bytecode.extend(b'\x14') # Arg1: Value 20
bytecode.extend(b'\x01') # Arg2: Unused (simulating register B)
# ADD A and B, store in C
bytecode.extend(OP_ADD)
bytecode.extend(b'\x00') # Arg1: Register A
bytecode.extend(b'\x01') # Arg2: Register B
# STORE result (C) to memory address 0x10
bytecode.extend(OP_STORE)
bytecode.extend(b'\x02') # Arg1: Register C (result)
bytecode.extend(b'\x10') # Arg2: Memory Address
bytecode.extend(OP_HALT)
return bytes(bytecode)
bytecode_data = generate_bytecode()
output_file = "program.bin"
with open(output_file, "wb") as f:
f.write(bytecode_data)
print(f"Generated {len(bytecode_data)} bytes of bytecode to '{output_file}'.")
print(f"Bytecode (hex): {bytecode_data.hex()}")
writelines()
Writes a list of strings (or any iterable of strings) to the file.
Similar to write(), it does not automatically add newline characters. You must ensure that each string in the list ends with \n if you want them on separate lines.
Note: Write list to file Python, writelines() method, writing multiple lines.
Code Examples:
Beginner-Friendly: Writing a list of quotes to a file.
Python
# beginner_writelines_quotes.py
quotes = [
"The only way to do great work is to love what you do.\n",
"Stay hungry, stay foolish.\n",
"Your time is limited, don't waste it living someone else's life."
]
with open("quotes.txt", "w") as file:
file.writelines(quotes) # Writes all strings from the list
print("Quotes written to 'quotes.txt' using writelines().")
Intermediate: Saving processed data as a list of lines.
Python
# intermediate_writelines_processed_data.py
data_points = [10.5, 20.3, 5.1, 12.8]
processed_lines = []
for dp in data_points:
processed_lines.append(f"Data point value: {dp:.2f}\n")
processed_lines.append(f"Squared value: {dp**2:.2f}\n")
with open("processed_data.log", "w") as file:
file.writelines(processed_lines)
print("Processed data logged to 'processed_data.log' using writelines().")
Advanced: Exporting a list of dictionary objects to a custom text format.
Python
# advanced_writelines_export_objects.py
users = [
{"id": 1, "name": "Alice", "role": "Admin"},
{"id": 2, "name": "Bob", "role": "User"},
{"id": 3, "name": "Charlie", "role": "Guest"}
]
export_lines = []
export_lines.append("--- USER EXPORT ---\n")
for user in users:
export_lines.append(f"ID: {user['id']}\n")
export_lines.append(f"Name: {user['name']}\n")
export_lines.append(f"Role: {user['role']}\n")
export_lines.append("---\n") # Separator for clarity
with open("user_export.txt", "w") as file:
file.writelines(export_lines)
print("User data exported to 'user_export.txt' using writelines().")
Advanced: Merging and writing contents of multiple files into one.
Python
# advanced_writelines_merge_files.py
# Create some dummy files to merge
with open("part1.txt", "w") as f: f.write("Section Alpha.\nLine A2.\n")
with open("part2.txt", "w") as f: f.write("Section Beta.\nLine B2.\n")
with open("part3.txt", "w") as f: f.write("Section Gamma.\nLine G2.\n")
files_to_merge = ["part1.txt", "part2.txt", "part3.txt"]
merged_content = []
for filename in files_to_merge:
try:
with open(filename, "r") as f:
merged_content.append(f"--- Content from {filename} ---\n")
merged_content.extend(f.readlines()) # Appends all lines (with newlines)
merged_content.append("\n") # Add extra newline for separation
except FileNotFoundError:
print(f"Warning: '{filename}' not found, skipping.")
with open("merged_document.txt", "w") as output_file:
output_file.writelines(merged_content)
print("Files merged into 'merged_document.txt' using writelines().")
Expert: Implementing a simple transaction log where each transaction is a list of events.
Python
# expert_transaction_log_writelines.py
from datetime import datetime
def record_transaction(transaction_id, events):
"""Records a transaction with multiple associated events."""
log_entries = []
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
log_entries.append(f"--- TRANSACTION START: {transaction_id} ({timestamp}) ---\n")
for event in events:
log_entries.append(f" EVENT: {event}\n")
log_entries.append(f"--- TRANSACTION END: {transaction_id} ---\n\n")
with open("transaction_log.txt", "a") as log_file:
log_file.writelines(log_entries)
print(f"Transaction {transaction_id} logged.")
# Simulate some transactions
record_transaction("TXN001", ["Item added to cart", "Payment initiated", "Order placed"])
record_transaction("TXN002", ["User login", "Profile updated"])
print("Transaction logging with writelines() complete.")
Working with Different File Types
While read(), readline(), readlines(), write(), and writelines() are great for plain text or simple binary data, many structured data formats have dedicated Python modules that make reading and writing much easier and more robust.
Note: Python file formats, CSV JSON handling, structured data files.
CSV files (csv module)
CSV (Comma Separated Values) files are a common way to store tabular data. Python's built-in csv module provides functions to easily read from and write to CSV files, handling quoting, delimiters, and other complexities for you.
Note: CSV file read Python, write CSV Python, csv module tutorial, data table handling.
Code Examples:
Beginner-Friendly: Reading a simple CSV file using csv.reader.
Python
# beginner_csv_read.py
import csv
# Create a dummy CSV file
with open("employees.csv", "w", newline='') as f:
writer = csv.writer(f)
writer.writerow(["Name", "Department", "HireDate"])
writer.writerow(["Alice", "HR", "2020-01-15"])
writer.writerow(["Bob", "IT", "2019-07-01"])
print("Reading employees.csv:")
with open("employees.csv", "r", newline='') as file:
reader = csv.reader(file)
header = next(reader) # Read the header row
print(f"Header: {header}")
for row in reader:
print(f"Row: {row}")
print("CSV reading with csv.reader complete.")
Intermediate: Writing data to a CSV file using csv.writer.
Python
# intermediate_csv_write.py
import csv
products = [
["Laptop", 1200.00, 10],
["Mouse", 25.50, 150],
["Keyboard", 75.00, 75]
]
print("\nWriting new_products.csv:")
# 'newline=''' is crucial to prevent blank rows on Windows
with open("new_products.csv", "w", newline='') as file:
writer = csv.writer(file)
writer.writerow(["Product Name", "Price", "Quantity"]) # Write header
writer.writerows(products) # Write all data rows
print("new_products.csv created with csv.writer.")
Advanced: Reading CSV data into a list of dictionaries using csv.DictReader.
Python
# advanced_csv_dictreader.py
import csv
# Create a dummy CSV with mixed data types (for demonstration)
with open("sales_data.csv", "w", newline='') as f:
writer = csv.writer(f)
writer.writerow(["OrderID", "Item", "Quantity", "Price"])
writer.writerow(["1001", "Book", "2", "15.99"])
writer.writerow(["1002", "Pen", "5", "2.50"])
writer.writerow(["1003", "Notebook", "1", "10.00"])
sales_records = []
print("\nReading sales_data.csv into dictionaries:")
with open("sales_data.csv", "r", newline='') as file:
reader = csv.DictReader(file) # Each row becomes a dictionary
for row in reader:
# Convert string values to appropriate types if needed
row['OrderID'] = int(row['OrderID'])
row['Quantity'] = int(row['Quantity'])
row['Price'] = float(row['Price'])
sales_records.append(row)
for record in sales_records:
print(record)
total_revenue = sum(r['Quantity'] * r['Price'] for r in sales_records)
print(f"Total revenue: ${total_revenue:.2f}")
print("CSV reading with csv.DictReader complete.")
Advanced: Writing a list of dictionaries to a CSV file using csv.DictWriter.
Python
# advanced_csv_dictwriter.py
import csv
users = [
{"id": "u001", "name": "Alice Smith", "email": "alice@example.com"},
{"id": "u002", "name": "Bob Johnson", "email": "bob@example.com"},
{"id": "u003", "name": "Charlie Brown", "email": "charlie@example.com"}
]
fieldnames = ["id", "name", "email"] # Must match dictionary keys
print("\nWriting users.csv from dictionaries:")
with open("users.csv", "w", newline='') as file:
writer = csv.DictWriter(file, fieldnames=fieldnames)
writer.writeheader() # Write the header row from fieldnames
writer.writerows(users) # Write all dictionaries
print("users.csv created with csv.DictWriter.")
Expert: Processing large CSV files with error handling and selective column extraction.
Python
# expert_csv_processing.py
import csv
import sys
# Create a larger dummy CSV with some potential errors
with open("large_transactions.csv", "w", newline='') as f:
writer = csv.writer(f)
writer.writerow(["TransactionID", "Date", "Amount", "Currency", "Description"])
writer.writerow(["TXN001", "2025-06-01", "150.75", "USD", "Online Purchase"])
writer.writerow(["TXN002", "2025-06-02", "abc", "USD", "Invalid Amount"]) # Error
writer.writerow(["TXN003", "2025-06-03", "200.00", "EUR", "International Transfer"])
writer.writerow(["TXN004", "2025-06-04", "50.00", "USD", ""]) # Missing description
def analyze_transactions(filepath):
total_usd_amount = 0.0
processed_count = 0
error_count = 0
print(f"\nAnalyzing '{filepath}' for USD transactions:")
try:
with open(filepath, "r", newline='') as file:
reader = csv.DictReader(file)
for i, row in enumerate(reader, 2): # Start line count from 2 (after header)
try:
if row.get("Currency") == "USD":
amount_str = row.get("Amount")
if amount_str:
amount = float(amount_str)
total_usd_amount += amount
processed_count += 1
# print(f" Processed USD transaction: {row['TransactionID']} - ${amount:.2f}")
else:
print(f"Warning: Line {i}: Missing Amount for TransactionID {row.get('TransactionID')}. Skipping.")
error_count += 1
else:
# print(f" Skipping non-USD transaction: {row.get('TransactionID')}")
pass
except ValueError:
print(f"Error: Line {i}: Invalid Amount '{row.get('Amount')}' for TransactionID {row.get('TransactionID')}. Skipping.")
error_count += 1
except KeyError as e:
print(f"Error: Line {i}: Missing column '{e}' in row. Skipping.")
error_count += 1
except FileNotFoundError:
print(f"Error: File '{filepath}' not found.")
return
print(f"\n--- Analysis Complete ---")
print(f"Total USD transactions processed: {processed_count}")
print(f"Total USD amount: ${total_usd_amount:.2f}")
print(f"Number of errors encountered: {error_count}")
analyze_transactions("large_transactions.csv")
JSON files (json module)
JSON (JavaScript Object Notation) is a lightweight data-interchange format. It's widely used for web APIs and configuration files. Python's json module provides methods to convert Python dictionaries and lists to JSON strings (serialization) and JSON strings to Python objects (deserialization).
Note: JSON file read Python, write JSON Python, json module basics, data serialization.
Code Examples:
Beginner-Friendly: Writing a simple Python dictionary to a JSON file.
Python
# beginner_json_write.py
import json
user_data = {
"name": "Alice Wonderland",
"age": 30,
"is_active": True,
"hobbies": ["reading", "hiking", "cooking"]
}
print("Writing user_data.json:")
# 'indent=4' makes the JSON output human-readable with indentation
with open("user_data.json", "w") as file:
json.dump(user_data, file, indent=4)
print("user_data.json created with json.dump().")
Intermediate: Reading JSON data from a file into a Python dictionary.
Python
# intermediate_json_read.py
import json
print("\nReading user_data.json:")
try:
with open("user_data.json", "r") as file:
loaded_data = json.load(file)
print("Loaded JSON data:")
print(loaded_data)
print(f"Name: {loaded_data['name']}")
print(f"Hobbies: {', '.join(loaded_data['hobbies'])}")
except FileNotFoundError:
print("Error: 'user_data.json' not found. Run the write example first.")
except json.JSONDecodeError as e:
print(f"Error decoding JSON: {e}")
print("JSON reading with json.load() complete.")
Advanced: Storing a list of objects (dictionaries) as JSON.
Python
# advanced_json_list_write.py
import json
books = [
{"title": "The Hitchhiker's Guide to the Galaxy", "author": "Douglas Adams", "year": 1979},
{"title": "1984", "author": "George Orwell", "year": 1949},
{"title": "Brave New World", "author": "Aldous Huxley", "year": 1932}
]
print("\nWriting books.json:")
with open("books.json", "w") as file:
json.dump(books, file, indent=2) # Use indent=2 for cleaner output
print("books.json created with a list of objects.")
Advanced: Reading and filtering/processing data from a JSON file.
Python
# advanced_json_read_process.py
import json
print("\nProcessing books.json:")
try:
with open("books.json", "r") as file:
all_books = json.load(file)
recent_books = [book for book in all_books if book["year"] > 1950]
print("Books published after 1950:")
for book in recent_books:
print(f"- {book['title']} by {book['author']} ({book['year']})")
except FileNotFoundError:
print("Error: 'books.json' not found. Run the write example first.")
except json.JSONDecodeError as e:
print(f"Error decoding JSON: {e}")
print("JSON processing with json.load() complete.")
Expert: Implementing a simple JSON-based configuration manager with default settings.
Python
# expert_json_config_manager.py
import json
import os
class ConfigManager:
def __init__(self, config_filepath="config.json", default_config=None):
self.filepath = config_filepath
self.config = {}
self.default_config = default_config or {
"theme": "dark",
"font_size": 14,
"notifications_enabled": True,
"api_key": "YOUR_DEFAULT_API_KEY"
}
self._load_config()
def _load_config(self):
if os.path.exists(self.filepath):
try:
with open(self.filepath, "r") as f:
self.config = json.load(f)
print(f"Configuration loaded from '{self.filepath}'.")
except json.JSONDecodeError:
print(f"Warning: Corrupt JSON in '{self.filepath}'. Using default config.")
self.config = self.default_config
except Exception as e:
print(f"An unexpected error occurred loading config: {e}. Using default config.")
self.config = self.default_config
else:
print(f"Config file '{self.filepath}' not found. Creating with default settings.")
self.config = self.default_config
self._save_config() # Save default config if file doesn't exist
# Merge defaults for any missing keys in loaded config
for key, value in self.default_config.items():
if key not in self.config:
self.config[key] = value
def _save_config(self):
try:
with open(self.filepath, "w") as f:
json.dump(self.config, f, indent=4)
print(f"Configuration saved to '{self.filepath}'.")
except IOError as e:
print(f"Error saving configuration: {e}")
def get_setting(self, key, default=None):
return self.config.get(key, default)
def set_setting(self, key, value):
self.config[key] = value
self._save_config() # Save immediately after setting
# Demonstrate usage
config_mgr = ConfigManager() # Loads or creates config.json
print("\nCurrent Settings:")
print(f"Theme: {config_mgr.get_setting('theme')}")
print(f"Font Size: {config_mgr.get_setting('font_size')}")
print(f"API Key: {config_mgr.get_setting('api_key')}") # Should be default initially
# Change a setting
config_mgr.set_setting("theme", "light")
config_mgr.set_setting("notifications_enabled", False)
config_mgr.set_setting("api_key", "ACTUAL_API_KEY_123") # Update an existing setting
config_mgr.set_setting("new_feature_flag", True) # Add a new setting
print("\nUpdated Settings:")
print(f"Theme: {config_mgr.get_setting('theme')}")
print(f"Notifications Enabled: {config_mgr.get_setting('notifications_enabled')}")
print(f"New Feature Flag: {config_mgr.get_setting('new_feature_flag')}")
# Create a new config manager instance to verify settings persistence
print("\nLoading config again to verify persistence:")
new_config_mgr = ConfigManager()
print(f"Theme (from new instance): {new_config_mgr.get_setting('theme')}")
print(f"API Key (from new instance): {new_config_mgr.get_setting('api_key')}")
print("JSON configuration manager demonstration complete.")