Files and Exceptions

What You Will Learn

• Understand files and persistent data.
• Read, write and update files.
• Use the with statement to release file resources safely.
• Serialize and deserialize objects with JSON.
• Understand the security risks of pickle serialization.
• Handle exceptions with try and except.
• Use else when no exception occurs and finally for code that should execute afterward.
• Raise exceptions explicitly.
• Understand stack unwinding and tracebacks.
• Read and write CSV files with the Python csv module.
• Load CSV data into pandas DataFrames.
• Explore the Titanic dataset with simple analysis and a histogram.

Chapter Structure

9.1 Introduction

Data stored only in variables or collections is temporary. When the program terminates, that in-memory data is normally gone. Files solve this problem by storing data on secondary storage such as SSDs and hard disks.

The chapter works with several important formats:

9.2 Files

Python views a text file as a sequence of characters and a binary file as a sequence of bytes. When you open a file, Python gives you a file object through which your program communicates with that file.

Text Files vs. Binary Files

Standard File Objects

Python creates three standard file objects when a program starts:

• sys.stdin — standard input.
• sys.stdout — standard output.
• sys.stderr — standard error output.

9.3 Text-File Processing

The chapter uses an accounts.txt file to demonstrate file processing. Each line represents a client record containing an account number, last name and balance.

100 Jones 24.98
200 Doe 345.67
300 White 0.00
400 Stone -42.16
500 Rich 224.62

Python does not automatically understand that these three values form a "record." The programmer decides how the data should be structured.

9.3.1 Writing to a Text File: Introducing the with Statement

Use open() to open a file. The mode 'w' opens a text file for writing. If the file does not exist, it is created. Be careful: 'w' deletes existing contents.

with open('accounts.txt', mode='w') as accounts:
    accounts.write('100 Jones 24.98\n')
    accounts.write('200 Doe 345.67\n')
    accounts.write('300 White 0.00\n')
    accounts.write('400 Stone -42.16\n')
    accounts.write('500 Rich 224.62\n')

You can also use print() to write to a file:

with open('accounts.txt', mode='w') as accounts:
    print('100 Jones 24.98', file=accounts)

Why use with?

The with statement manages the resource and automatically calls the file object's close() method when control leaves the block.

File Open Modes

9.3.2 Reading Data from a Text File

Open a file with mode 'r' when you only want to read it. You can iterate through the file one line at a time.

with open('accounts.txt', mode='r') as accounts:
    print(f'{"Account":<10}{"Name":<10}{"Balance":>10}')
    for record in accounts:
        account, name, balance = record.split()
        print(f'{account:<10}{name:<10}{balance:>10}')

Account   Name         Balance
100       Jones           24.98
200       Doe            345.67
300       White            0.00
400       Stone          -42.16
500       Rich            224.62

Iterating over a file reads one line at a time. For large files, this is useful because the whole file does not need to be loaded into a list first.

readlines()

readlines() returns the lines of a text file as a list of strings.

with open('accounts.txt', 'r') as accounts:
    lines = accounts.readlines()

print(lines)

seek()

A file object maintains a file-position pointer. seek(0) moves that pointer back to the beginning.

with open('accounts.txt', 'r') as accounts:
    print(accounts.readline(), end='')
    accounts.seek(0)
    print(accounts.readline(), end='')

Other Useful File Methods

9.4 Updating Text Files

Updating formatted text files can be tricky because records can have different lengths. For example, replacing White with longer text such as Williams can overwrite characters belonging to other fields.

A common technique is therefore to create a temporary file containing the updated records, then replace the original file.

import os

accounts = open('accounts.txt', 'r')
temp_file = open('temp_file.txt', 'w')

with accounts, temp_file:
    for record in accounts:
        account, name, balance = record.split()

        if account != '300':
            temp_file.write(record)
        else:
            new_record = ' '.join([account, 'Williams', balance])
            temp_file.write(new_record + '\n')

os.remove('accounts.txt')
os.rename('temp_file.txt', 'accounts.txt')

9.5 Serialization with JSON

Serialization means converting an object into a format that can be stored or transmitted. Deserialization converts the stored representation back into a Python object.

JSON stands for JavaScript Object Notation. It is widely used for data interchange, including communication with web and cloud services.

json.dump() — Write JSON to a File

import json

data = {
    'accounts': [
        {'account': 100, 'name': 'Jones', 'balance': 24.98},
        {'account': 200, 'name': 'Doe', 'balance': 345.67}
    ]
}

with open('accounts.json', 'w') as file:
    json.dump(data, file, indent=4)

json.load() — Read JSON from a File

with open('accounts.json', 'r') as file:
    loaded_data = json.load(file)

print(loaded_data)

dump/dumps and load/loads

9.6 Focus on Security: pickle Serialization and Deserialization

Python's pickle module can serialize and deserialize Python objects, but the chapter emphasizes an important security concern: do not unpickle untrusted data.

JSON is intended as a data representation format. Pickle is Python-specific and can execute dangerous behavior when malicious serialized data is loaded.

9.7 Additional Notes Regarding Files

File processing includes more than plain text. Binary files are opened with modes containing b, such as rb and wb+.

Files may also produce exceptions. For example, trying to read a file that does not exist can produce FileNotFoundError, while permission problems can produce PermissionError.

9.8 Handling Exceptions

An exception represents a problem detected while a program is executing. Python raises an exception at the point where the problem occurs.

9.8.1 Division by Zero and Invalid Input

Consider a program that asks for two integers and divides them. Two different problems can occur:

• The user enters non-numeric input → ValueError.
• The user enters zero as denominator → ZeroDivisionError.

while True:
    try:
        number1 = int(input('Enter numerator: '))
        number2 = int(input('Enter denominator: '))
        result = number1 / number2

    except ValueError:
        print('You must enter two integers')

    except ZeroDivisionError:
        print('Attempted to divide by zero')

    else:
        print(f'{number1:.3f} / {number2:.3f} = {result:.3f}')
        break

9.8.2 try Statements

The basic structure is:

try:
    # code that might raise an exception
except SomeException:
    # handle that exception
else:
    # executes only when no exception occurred
finally:
    # executes after the try process

9.8.3 Catching Multiple Exceptions in One except Clause

Several exception types can be handled by one clause using a tuple.

try:
    value = int(input('Enter an integer: '))
    result = 100 / value

except (ValueError, ZeroDivisionError):
    print('Please enter a non-zero integer')

9.8.4 What Exceptions Does a Function or Method Raise?

Library functions and methods can raise exceptions too. When using an API, know which exceptions can occur so that your program can handle expected problems appropriately.

9.8.5 What Code Should Be Placed in a try Suite?

Place the statements that can reasonably raise the exceptions you intend to handle inside the try suite. Avoid putting unrelated code there because it can make debugging and exception handling less precise.

try:
    with open('grades.txt', 'r') as file:
        for record in file:
            print(record.strip())

except FileNotFoundError:
    print('The file name you specified does not exist')

9.9 finally Clause

The finally suite executes after the try processing, whether an exception occurred or not, as long as program control entered the corresponding try statement and the program does not terminate first.

try:
    print('try suite with no exceptions raised')
except:
    print('this will not execute')
else:
    print('else executes because no exceptions occurred')
finally:
    print('finally always executes')

try suite with no exceptions raised
else executes because no exceptions occurred
finally always executes

If an exception occurs:

try:
    print('try suite that raises an exception')
    int('hello')
    print('this will not execute')

except ValueError:
    print('a ValueError occurred')

else:
    print('else will not execute')

finally:
    print('finally always executes')

try suite that raises an exception
a ValueError occurred
finally always executes

9.10 Explicitly Raising an Exception

The raise statement lets your program explicitly indicate a runtime problem.

def check_age(age):
    if age < 0:
        raise ValueError('Age cannot be negative')
    return age

print(check_age(25))

25

If an invalid value is supplied:

check_age(-5)

ValueError: Age cannot be negative

9.11 (Optional) Stack Unwinding and Tracebacks

A traceback records the sequence of function calls that led to an exception. It is one of the most useful tools for debugging.

def function1():
    function2()

def function2():
    raise Exception('An exception occurred')

function1()

The traceback shows that function1() called function2(), and function2() was the location where the exception was raised.

Stack Unwinding

If an exception is not caught in a function, that function terminates and control returns to its caller. If the caller also does not handle the exception, the process continues upward through the call stack. This is called stack unwinding.

9.12 Intro to Data Science: Working with CSV Files

CSV means comma-separated values. CSV is a particularly popular format for datasets because rows and columns can be represented in a simple text file.

9.12.1 Python Standard Library Module csv

Python's built-in csv module provides functions for reading and writing CSV files.

Writing a CSV File

import csv

with open('accounts.csv', 'w', newline='') as file:
    writer = csv.writer(file)

    writer.writerow(['account', 'name', 'balance'])
    writer.writerow([100, 'Jones', 24.98])
    writer.writerow([200, 'Doe', 345.67])
    writer.writerow([300, 'White', 0.00])

The resulting file can look like:

account,name,balance
100,Jones,24.98
200,Doe,345.67
300,White,0.0

9.12.2 Reading CSV Files into pandas DataFrames

pandas provides read_csv() for loading CSV data into a DataFrame.

import pandas as pd

df = pd.read_csv(
    'accounts.csv',
    names=['account', 'name', 'balance']
)

print(df)

   account   name  balance
0      100  Jones    24.98
1      200    Doe   345.67
2      300  White     0.00

You can save a DataFrame back to CSV with to_csv().

df.to_csv('accounts_from_dataframe.csv', index=False)

index=False prevents pandas from writing the DataFrame's row indexes into the CSV file.

9.12.3 Reading the Titanic Disaster Dataset

The chapter uses the Titanic disaster dataset as a practical example of loading and analyzing real-world data. Each row represents a passenger and columns contain information such as name, survival status, sex, age and passenger class.

import pandas as pd

titanic = pd.read_csv(
    'TitanicSurvival.csv'
)

print(titanic.head())
print(titanic.tail())

For large datasets, head() and tail() are convenient ways to inspect the beginning and end of the data.

9.12.4 Simple Data Analysis with the Titanic Disaster Dataset

The dataset contains missing age values. pandas represents missing numerical data with NaN ("not a number"). The chapter changes awkward column names to clearer names:

titanic.columns = ['name', 'survived', 'sex', 'age', 'class']

print(titanic.head())

You can use describe() to get descriptive statistics for numerical columns.

print(titanic.describe())

9.12.5 Passenger Age Histogram

A histogram groups numerical values into ranges so you can see the distribution of those values.

%matplotlib
histogram = titanic.hist()

The chapter uses pandas' hist() to visualize numerical data. For a dataset with multiple numerical columns, pandas can create a histogram for each numerical column.

Quick Concept Map

Common Mistakes

Remember These Points

Revision Questions

1. What is persistent data?
2. What is the difference between a text file and a binary file?
3. What does open() return?
4. What is the purpose of the with statement when working with files?
5. What is the difference between file modes r, w and a?
6. Why can updating a formatted text file be difficult?
7. What is serialization?
8. What is the difference between json.dump() and json.dumps()?
9. Why should untrusted pickle data not be deserialized?
10. What is an exception?
11. How does a try statement work?
12. When does the else clause execute?
13. When does the finally clause execute?
14. What does the raise statement do?
15. What is stack unwinding?
16. What information does a traceback provide?
17. What does CSV stand for?
18. How do you load a CSV file into a pandas DataFrame?
19. Why is index=False useful with to_csv()?
20. What does describe() show?

Practice Programs

Chapter Wrap-Up

This chapter introduces files as a way to store data persistently. You learned how to create, read and update text files and how the with statement safely manages file resources.

You also learned about JSON serialization and deserialization, the security concerns surrounding pickle, and additional file-processing details. The chapter then introduced exception handling with try, except, else and finally, along with explicit raise, tracebacks and stack unwinding.

Finally, the data-science section showed how CSV files can be processed with Python's csv module and pandas. The Titanic dataset demonstrates loading real-world data, inspecting rows, handling missing values, calculating descriptive statistics and visualizing numerical data.

Teaching edition note: This HTML is a beginner-friendly explanation based on the chapter's organization and concepts. It uses simplified explanations and original examples rather than reproducing the book verbatim.