Robust code: errors, files, and edge cases

The code in the previous eleven modules assumed clean, well-formed data. Real data is not clean. A CBK auction PDF has a stray footnote in the yield column; a KNBS export uses a Latin-1 dash that pandas can't decode; a survey CSV has three rows where the enumerator typed 'N/A' into a numeric field. Code that works on the demo dataset and crashes on the real one is the single most common way an analysis dies the night before it's due. This module is about writing Python that fails loudly where it should, recovers gracefully where it can, and never silently produces a wrong number.

Exceptions — the try/except block

When Python hits something it cannot do — dividing by zero, converting 'N/A' to a float, opening a file that isn't there — it raises an exception. An unhandled exception halts the program and prints a traceback. The try/except block lets you intercept that exception and decide what to do instead of crashing.

raw = 'N/A'
try:
    rate = float(raw)
except ValueError:
    rate = None          # couldn't parse — fall back to a missing value

print(rate)              # None  (no crash, no traceback)

Catch specific exceptions, not everything

Every failure mode has a named exception type. Catch the specific one you expect and can handle — that way a bug you did NOT anticipate still surfaces as a loud traceback instead of being swallowed. The four you'll meet most often in data work:

• ValueError — a value has the right type but the wrong content: float('N/A'), int('12.5').
• KeyError — a dict (or DataFrame column, or JSON field) is missing the key you asked for: row['deposit_rate'] when the column is named 'deposit'.
• FileNotFoundError — open() can't find the path: a typo, a wrong working directory, a file that never downloaded.
• ZeroDivisionError — dividing by zero: computing a spread ratio in a month where the deposit rate was recorded as 0.

row = {'bank': 'KCB', 'lending': 18.5}

try:
    deposit = row['deposit_rate']     # not in this dict
except KeyError:
    deposit = None                    # KeyError caught -> deposit is None

try:
    ratio = row['lending'] / 0        # 18.5 / 0
except ZeroDivisionError:
    ratio = float('inf')             # sentinel for 'undefined'

print(deposit, ratio)                 # None inf

else and finally — the full block

A try block has four clauses. try holds the risky code. except handles a failure. else runs only if NO exception was raised — put the code that depends on the try succeeding here, so it isn't accidentally shielded by the except. finally runs no matter what — success, failure, or even a return — which makes it the place for cleanup like closing a connection.

def parse_rate(raw):
    try:
        rate = float(raw)
    except ValueError:
        print(f'  bad value: {raw!r}')
        return None
    else:
        # only runs if float() succeeded
        return rate / 100 if rate > 1 else rate
    finally:
        # always runs, on every path
        print(f'  processed: {raw!r}')

parse_rate('18.5')   # prints 'processed: ...' then returns 0.185
parse_rate('N/A')    # prints 'bad value...' and 'processed...', returns None

Raising your own exceptions

You don't only catch exceptions — you raise them when your code detects something wrong. A function that receives a negative interest rate should refuse to continue rather than quietly return nonsense. `raise` makes the failure explicit and gives the caller a chance to handle it.

def annualise(monthly_rate):
    if monthly_rate < 0:
        raise ValueError(f'rate cannot be negative: {monthly_rate}')
    return (1 + monthly_rate) ** 12 - 1

annualise(0.01)     # 0.1268...
annualise(-0.01)    # ValueError: rate cannot be negative: -0.01

EAFP vs LBYL — the Pythonic idiom

There are two philosophies for guarding against failure. LBYL — 'Look Before You Leap' — checks preconditions first: does the key exist, is the value a number, is the file present? EAFP — 'Easier to Ask Forgiveness than Permission' — just tries the operation and catches the exception if it fails. Python culture strongly favours EAFP: it's usually shorter, and it avoids a race condition where the world changes between your check and your action (the file exists when you check, then is deleted before you open it).

row = {'bank': 'Equity', 'lending': 17.2}

# LBYL — look before you leap
if 'deposit_rate' in row:
    d = row['deposit_rate']
else:
    d = None

# EAFP — easier to ask forgiveness (the Pythonic version)
try:
    d = row['deposit_rate']
except KeyError:
    d = None

# Or, for this exact case, dict.get does it in one line:
d = row.get('deposit_rate')     # returns None if the key is absent

Files — open() and the with statement

open() returns a file object. The problem: an open file holds an operating-system handle that must be closed, and if an exception fires between open and close, a plain close() call is skipped and the handle leaks. The `with` statement solves this: it's a context manager that guarantees the file is closed when the block ends, exception or not. Always use `with` — never a bare open() you have to remember to close.

# Writing
with open('rates.txt', 'w', encoding='utf-8') as f:
    f.write('month,lending,deposit\n')
    f.write('2024-01,18.5,7.2\n')
# file is closed here, automatically

# Reading, line by line
with open('rates.txt', 'r', encoding='utf-8') as f:
    for line in f:
        print(line.rstrip())   # rstrip drops the trailing newline

# month,lending,deposit
# 2024-01,18.5,7.2

CSV — the csv module

The standard-library csv module handles the quoting and escaping rules of CSV correctly — something you must never do by hand with line.split(','), because a value like "Nairobi, Kenya" contains a comma inside quotes that split would wrongly break in two. csv.DictReader gives you each row as a dict keyed by the header.

import csv

# Write
rows = [
    {'bank': 'KCB', 'lending': 18.5},
    {'bank': 'Equity', 'lending': 17.2},
]
with open('banks.csv', 'w', newline='', encoding='utf-8') as f:
    writer = csv.DictWriter(f, fieldnames=['bank', 'lending'])
    writer.writeheader()
    writer.writerows(rows)

# Read
with open('banks.csv', newline='', encoding='utf-8') as f:
    for row in csv.DictReader(f):
        print(row['bank'], row['lending'])   # values are STRINGS

# KCB 18.5
# Equity 17.2

CSV — the pandas way

For analysis you'll almost always reach for pandas instead: pd.read_csv infers types, handles the header, and recognises common missing-value markers automatically. It's one line, and its arguments are the escape hatches for every messy-file problem.

import pandas as pd

df = pd.read_csv(
    'banks.csv',
    encoding='utf-8',
    na_values=['N/A', 'n/a', '-', ''],   # treat these as NaN
    thousands=',',                       # parse '1,234' as 1234
)

df.to_csv('clean.csv', index=False)      # write back, no row-number column

Defensive data work — missing values, validation, assertions

Once the data is in, three habits keep you honest. First, handle NaN deliberately: know whether you want to drop it, fill it, or let it propagate — a NaN silently poisons a mean if you don't. Second, validate inputs at the boundary, before the number flows into a model. Third, use assertions to encode the invariants you believe are true — an assert that fires is a bug caught early; the alternative is a wrong chart caught late.

import pandas as pd, numpy as np

df = pd.DataFrame({'lending': [18.5, np.nan, 17.2, 200.0]})

df['lending'].isna().sum()        # 1  (one missing value)
df['lending'].mean()              # 78.56... — NaN skipped, but the 200 outlier remains
df = df.dropna(subset=['lending'])          # drop rows missing lending
# or: df['lending'] = df['lending'].fillna(df['lending'].median())

# Validate: an interest rate should be in a sane range
bad = df[(df['lending'] < 0) | (df['lending'] > 100)]
assert bad.empty, f'implausible rates found:\n{bad}'
# AssertionError fires on the 200.0 row — a data-entry error caught before modelling

Worked example — robustly loading a messy Kenyan CSV

Here is the pattern that ties it together: read a real-world CSV where some rows are broken — an 'N/A' rate, a missing field, a non-numeric entry — parse each row inside a try/except, collect the good rows, and report the bad ones instead of crashing on the first one. This is roughly what production ingestion code looks like.

import csv

# Imagine this file on disk: monthly bank rates, three rows are broken.
messy = '''month,bank,lending,deposit
2024-01,KCB,18.5,7.2
2024-02,Equity,N/A,7.4
2024-03,Coop,17.9
2024-04,NCBA,seventeen,7.1
2024-05,DTB,18.1,7.0
'''
with open('messy.csv', 'w', newline='', encoding='utf-8') as f:
    f.write(messy)

good, bad = [], []
with open('messy.csv', newline='', encoding='utf-8') as f:
    for lineno, row in enumerate(csv.DictReader(f), start=2):
        try:
            record = {
                'month': row['month'],
                'bank': row['bank'],
                'lending': float(row['lending']),
                'deposit': float(row['deposit']),
            }
        except (ValueError, KeyError, TypeError) as e:
            # ValueError: 'N/A'/'seventeen' won't parse
            # KeyError:   a column is absent
            # TypeError:  float(None) when DictReader saw a short row
            bad.append((lineno, type(e).__name__, dict(row)))
            continue          # skip the bad row, keep going
        good.append(record)

print(f'loaded {len(good)} good rows, skipped {len(bad)}')
for lineno, kind, row in bad:
    print(f'  line {lineno}: {kind} on {row}')

# loaded 2 good rows, skipped 3
#   line 3: ValueError on {'month': '2024-02', 'bank': 'Equity', ...}
#   line 4: TypeError on {'month': '2024-03', 'bank': 'Coop', 'deposit': None}
#   line 5: ValueError on {'month': '2024-04', 'bank': 'NCBA', ...}