9. Chapter 9: Python Advanced Topics¶
9.1. List Comprehension¶
Python List Comprehension is a high-performance, declarative construct for generating lists through compact expressions. It replaces verbose iterative patterns with expressive, memory-efficient, and semantically rich syntax.
9.1.1. Basic List Comprehension¶
squares = [x**2 for x in range(5)]
# Equivalent to:
squares = []
for x in range(5):
squares.append(x**2)
9.1.2. Conditional Filtering¶
even_numbers = [x for x in range(10) if x % 2 == 0]
9.1.3. Nested List Comprehensions¶
matrix = [[i * j for j in range(3)] for i in range(3)]
9.1.4. List Comprehension with Function Calls¶
def transform(x):
return x * 10
result = [transform(x) for x in range(5)]
9.1.5. Multiple Conditions¶
result = [x for x in range(20) if x > 5 if x % 2 == 0]
9.1.6. If-Else Expressions¶
status = ["even" if x % 2 == 0 else "odd" for x in range(5)]
9.1.7. Advanced Pattern: Flattening Lists¶
nested = [[1, 2], [3, 4], [5]]
flat = [item for sublist in nested for item in sublist]
9.1.8. Data Cleaning Pipeline Example¶
raw = [" apple ", " Banana", "cherry "]
cleaned = [item.strip().lower() for item in raw]
9.1.9. Combining with zip()¶
pairs = [(x, y) for x, y in zip([1,2], [3,4])]
9.1.10. Dictionary Creation via List Comprehension¶
keys = ["a", "b"]
values = [1, 2]
data = {k: v for k, v in zip(keys, values)}
9.2. Python Lambda / Anonymous Function¶
9.2.1. 1. What is a Lambda Function¶
A lambda function is a small anonymous function defined using the lambda keyword:
square = lambda x: x ** 2
print(square(5)) # Output: 25
Lambda functions are used for concise, one-line operations.
9.2.2. 2. Lambda with Multiple Arguments¶
add = lambda a, b: a + b
print(add(10, 5)) # Output: 15
9.2.3. 3. Lambda Without Assignment (Inline Usage)¶
print((lambda x, y: x * y)(4, 5)) # Output: 20
9.2.4. 4. Lambda with map()¶
numbers = [1, 2, 3, 4]
squares = list(map(lambda x: x ** 2, numbers))
print(squares) # Output: [1, 4, 9, 16]
9.2.5. 5. Lambda with filter()¶
numbers = [1, 2, 3, 4, 5, 6]
even_numbers = list(filter(lambda x: x % 2 == 0, numbers))
print(even_numbers) # Output: [2, 4, 6]
9.2.6. 6. Lambda with reduce()¶
from functools import reduce
numbers = [1, 2, 3, 4]
total = reduce(lambda x, y: x + y, numbers)
print(total) # Output: 10
9.2.7. 7. Lambda for Sorting Key¶
students = [("Alice", 85), ("Bob", 72), ("Charlie", 90)]
students.sort(key=lambda x: x[1])
print(students)
9.2.8. 8. Lambda in List Comprehension Context¶
multipliers = [(lambda x: x * n) for n in range(1, 4)]
print(multipliers)
9.2.9. 9. Lambda vs Normal Function¶
def square_func(x):
return x ** 2
square_lambda = lambda x: x ** 2
print(square_func(4)) # 16
print(square_lambda(4)) # 16
9.2.10. 10. Real-World Lambda Example¶
employees = [
{"name": "Alice", "salary": 5000},
{"name": "Bob", "salary": 3000},
{"name": "Charlie", "salary": 7000}
]
highest_paid = max(employees, key=lambda emp: emp["salary"])
print(highest_paid)
9.3. Python Iterators¶
9.3.1. 1. What is an Iterator¶
An iterator is an object that allows traversal through all elements of a collection, one element at a time:
numbers = [1, 2, 3, 4]
iterator = iter(numbers)
print(next(iterator)) # 1
print(next(iterator)) # 2
An iterator follows two core methods: __iter__() and __next__().
9.3.2. 2. Iterable vs Iterator¶
data = [10, 20, 30]
print(iter(data)) # Iterable converted to iterator
print(isinstance(data, list)) # Iterable
Iterable → Can be looped over
Iterator → Produces values one-by-one
9.3.3. 3. Using next() with Iterators¶
items = ["A", "B", "C"]
it = iter(items)
print(next(it))
print(next(it))
print(next(it))
9.3.4. 4. StopIteration Exception¶
items = [1, 2]
it = iter(items)
print(next(it))
print(next(it))
# print(next(it)) # Raises StopIteration
9.3.5. 5. Iterators in for Loops¶
colors = ["red", "green", "blue"]
for color in colors:
print(color)
9.3.6. 6. Creating Custom Iterator Class¶
class CountUp:
def __init__(self, limit):
self.limit = limit
self.current = 0
def __iter__(self):
return self
def __next__(self):
if self.current < self.limit:
self.current += 1
return self.current
raise StopIteration
counter = CountUp(3)
for number in counter:
print(number)
9.3.7. 7. Infinite Iterator Example¶
class InfiniteNumbers:
def __iter__(self):
self.num = 0
return self
def __next__(self):
self.num += 1
return self.num
9.3.8. 8. Iterator from Built-in Functions¶
data = "Python"
it = iter(data)
print(list(it)) # ['P', 'y', 't', 'h', 'o', 'n']
9.3.9. 9. Checking if an Object is an Iterator¶
from collections.abc import Iterator
numbers = iter([1, 2, 3])
print(isinstance(numbers, Iterator)) # True
9.3.10. 10. Real-World Iterator Example¶
class FileLineReader:
def __init__(self, filename):
self.file = open(filename, "r")
def __iter__(self):
return self
def __next__(self):
line = self.file.readline()
if line:
return line.strip()
self.file.close()
raise StopIteration
for line in FileLineReader("data.txt"):
print(line)
9.4. Python Generators¶
9.4.1. 1. What is a Generator¶
A generator is a special type of function that returns an iterator and yields values one at a time using the yield keyword:
def simple_generator():
yield 1
yield 2
yield 3
gen = simple_generator()
print(next(gen)) # 1
print(next(gen)) # 2
Generators produce values lazily, improving memory efficiency.
9.4.2. 2. Generator vs Normal Function¶
def normal_function():
return 10
return 20
def generator_function():
yield 10
yield 20
print(normal_function()) # 10
print(list(generator_function())) # [10, 20]
9.4.3. 3. Iterating Over a Generator¶
def count_up(n):
for i in range(1, n + 1):
yield i
for num in count_up(5):
print(num)
9.4.4. 4. Generator State Preservation¶
def demo():
print("Start")
yield 1
print("Resume")
yield 2
g = demo()
print(next(g))
print(next(g))
9.4.5. 5. Memory Efficiency of Generators¶
def large_numbers():
for i in range(1000000):
yield i
gen = large_numbers()
print(next(gen))
print(next(gen))
9.4.6. 6. Generator Expression¶
squares = (x ** 2 for x in range(5))
print(list(squares)) # [0, 1, 4, 9, 16]
9.4.7. 7. Using Generators with next()¶
def alpha_generator():
yield "A"
yield "B"
yield "C"
gen = alpha_generator()
print(next(gen))
print(next(gen))
9.4.8. 8. Infinite Generator¶
def infinite_counter():
num = 1
while True:
yield num
num += 1
counter = infinite_counter()
print(next(counter))
print(next(counter))
9.4.9. 9. Generator with try…finally¶
def resource_handler():
try:
yield "Using resource"
finally:
print("Resource released")
gen = resource_handler()
print(next(gen))
9.4.10. 10. Real-World Generator Example¶
def read_large_file(filename):
with open(filename, "r") as file:
for line in file:
yield line.strip()
for line in read_large_file("data.txt"):
print(line)
9.5. Python Namespace and Scope¶
9.5.1. 1. What is a Namespace¶
A namespace is a container that holds identifiers (variables, functions, objects) and maps them to their corresponding values:
a = 10
Here, a resides in a namespace and points to the value 10.
Types of namespaces:
Built-in namespace
Global namespace
Local namespace
9.5.2. 2. Built-in Namespace¶
print(len([1, 2, 3]))
print(type(100))
These functions belong to the built-in namespace and are always available.
9.5.3. 3. Global Namespace¶
x = 50 # Global variable
def show():
print(x)
show()
Global variables are accessible throughout the module.
9.5.4. 4. Local Namespace¶
def calculate():
y = 20 # Local variable
print(y)
calculate()
# print(y) # NameError
9.5.5. 5. Accessing Namespace Using globals()¶
x = 100
print(globals()["x"]) # Output: 100
9.5.6. 6. Accessing Namespace Using locals()¶
def demo():
a = 10
b = 20
print(locals())
demo()
9.5.7. 7. LEGB Rule (Scope Resolution Order)¶
Python resolves variable names in the following order:
Local
Enclosing
Global
Built-in
x = "Global"
def outer():
x = "Enclosing"
def inner():
x = "Local"
print(x)
inner()
outer()
9.5.8. 8. Enclosing Namespace¶
def outer():
message = "Hello"
def inner():
print(message)
inner()
outer()
9.5.9. 9. Modifying Enclosing Variables with nonlocal¶
def counter():
value = 0
def increment():
nonlocal value
value += 1
return value
return increment
count = counter()
print(count())
print(count())
9.5.10. 10. Namespace Lifecycle Demonstration¶
x = "Global"
def test():
x = "Local"
print("Inside:", x)
test()
print("Outside:", x)
9.6. Python Closures¶
9.6.1. 1. What is a Closure¶
A closure is a function that remembers and has access to variables from its enclosing scope even after the outer function has finished execution:
def outer():
message = "Hello"
def inner():
return message
return inner
func = outer()
print(func()) # Output: Hello
9.6.2. 2. Basic Closure Structure¶
def multiplier(x):
def multiply(y):
return x * y
return multiply
times2 = multiplier(2)
print(times2(5)) # Output: 10
9.6.3. 3. Closure Retaining State¶
def counter():
count = 0
def increment():
nonlocal count
count += 1
return count
return increment
c = counter()
print(c()) # 1
print(c()) # 2
9.6.4. 4. nonlocal in Closures¶
def power(base):
exponent = 2
def calculate():
nonlocal exponent
exponent += 1
return base ** exponent
return calculate
p = power(2)
print(p())
print(p())
9.6.5. 5. Closure vs Global Variable¶
def using_closure():
factor = 10
return lambda x: x * factor
print(using_closure()(5))
9.6.6. 6. Multiple Closures from Same Function¶
def make_adder(x):
def adder(y):
return x + y
return adder
add5 = make_adder(5)
add10 = make_adder(10)
print(add5(3)) # 8
print(add10(3)) # 13
9.6.7. 7. Inspecting Closure Variables¶
def outer():
value = 42
def inner():
return value
return inner
func = outer()
print(func.__closure__[0].cell_contents) # 42
9.6.8. 8. Closures as Function Factories¶
def logger(prefix):
def log(message):
print(f"{prefix}: {message}")
return log
info_logger = logger("INFO")
error_logger = logger("ERROR")
info_logger("System started")
error_logger("System failed")
9.6.9. 9. Closure in Decorators (Foundation Pattern)¶
def my_decorator(func):
def wrapper(*args, **kwargs):
print("Before execution")
return func(*args, **kwargs)
return wrapper
9.6.10. 10. Real-World Closure Example¶
def threshold_checker(threshold):
def check(value):
return value > threshold
return check
high_temp = threshold_checker(40)
print(high_temp(35)) # False
print(high_temp(45)) # True
9.7. Python Decorators¶
9.7.1. 1. What is a Decorator¶
A decorator is a function that modifies the behavior of another function without changing its source code:
def my_decorator(func):
def wrapper():
print("Before function execution")
func()
print("After function execution")
return wrapper
@my_decorator
def say_hello():
print("Hello")
say_hello()
9.7.2. 2. Decorator Without @ Syntax¶
def greet():
print("Welcome")
greet = my_decorator(greet)
greet()
9.7.3. 3. Decorator with Arguments¶
def log_decorator(func):
def wrapper(*args, **kwargs):
print("Arguments:", args, kwargs)
return func(*args, **kwargs)
return wrapper
@log_decorator
def add(a, b):
return a + b
print(add(5, 3))
9.7.4. 4. Chaining Multiple Decorators¶
def bold(func):
def wrapper():
print("<b>")
func()
print("</b>")
return wrapper
def italic(func):
def wrapper():
print("<i>")
func()
print("</i>")
return wrapper
@bold
@italic
def display():
print("Text")
display()
9.7.5. 5. Decorator with Return Value¶
def uppercase(func):
def wrapper():
result = func()
return result.upper()
return wrapper
@uppercase
def message():
return "hello world"
print(message())
9.7.6. 6. Decorator Preserving Function Metadata¶
from functools import wraps
def my_decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
9.7.7. 7. Decorator for Timing Execution¶
import time
from functools import wraps
def timer(func):
@wraps(func)
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
print("Execution Time:", time.time() - start)
return result
return wrapper
@timer
def process():
for _ in range(1000000):
pass
process()
9.7.8. 8. Decorator with Parameters¶
def repeat(n):
def decorator(func):
def wrapper(*args, **kwargs):
for _ in range(n):
func(*args, **kwargs)
return wrapper
return decorator
@repeat(3)
def greet():
print("Hello")
greet()
9.7.9. 9. Class-Based Decorator¶
class Logger:
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print("Calling function...")
return self.func(*args, **kwargs)
@Logger
def show():
print("Processing data")
show()
9.7.10. 10. Real-World Decorator Example¶
def require_login(func):
def wrapper(user):
if not user.get("authenticated"):
return "Access Denied"
return func(user)
return wrapper
@require_login
def view_profile(user):
return f"Welcome {user['name']}"
user = {"name": "Alice", "authenticated": True}
print(view_profile(user))
9.8. Python @property Decorator¶
9.8.1. 1. What is @property¶
The @property decorator allows a method to be accessed like an attribute while still providing controlled logic behind the scenes:
class Person:
def __init__(self, name):
self._name = name
@property
def name(self):
return self._name
p = Person("Alice")
print(p.name) # Alice
9.8.2. 2. Encapsulation Using @property¶
class Product:
def __init__(self, price):
self._price = price
@property
def price(self):
return self._price
item = Product(100)
print(item.price)
9.8.3. 3. Read-Only Property¶
class Circle:
def __init__(self, radius):
self._radius = radius
@property
def area(self):
return 3.14 * self._radius ** 2
c = Circle(5)
print(c.area)
9.8.4. 4. Using @property with Setter¶
class Employee:
def __init__(self, salary):
self._salary = salary
@property
def salary(self):
return self._salary
@salary.setter
def salary(self, value):
if value < 0:
raise ValueError("Salary cannot be negative")
self._salary = value
e = Employee(5000)
e.salary = 6000
print(e.salary)
9.8.5. 5. @property with Deleter¶
class Account:
def __init__(self, balance):
self._balance = balance
@property
def balance(self):
return self._balance
@balance.deleter
def balance(self):
del self._balance
acc = Account(1000)
del acc.balance
9.8.6. 6. Computed Property Example¶
class Rectangle:
def __init__(self, length, width):
self.length = length
self.width = width
@property
def area(self):
return self.length * self.width
rect = Rectangle(10, 5)
print(rect.area) # 50
9.8.7. 7. Difference: @property vs Normal Method¶
class User:
def __init__(self, name):
self._name = name
def get_name(self):
return self._name
@property
def name(self):
return self._name
u = User("Bob")
print(u.get_name()) # Method call
print(u.name) # Property access
9.8.8. 8. Enforcing Data Integrity¶
class Temperature:
def __init__(self, celsius):
self._celsius = celsius
@property
def celsius(self):
return self._celsius
@celsius.setter
def celsius(self, value):
if value < -273.15:
raise ValueError("Temperature below absolute zero")
self._celsius = value
9.8.9. 9. Lazy Evaluation with @property¶
class DataLoader:
def __init__(self):
self._data = None
@property
def data(self):
if self._data is None:
print("Loading data...")
self._data = [1, 2, 3]
return self._data
loader = DataLoader()
print(loader.data)
print(loader.data)
9.8.10. 10. Enterprise-Style Implementation¶
class BankAccount:
def __init__(self, owner, balance):
self.owner = owner
self._balance = balance
@property
def balance(self):
return self._balance
@balance.setter
def balance(self, amount):
if amount < 0:
raise ValueError("Balance cannot be negative")
self._balance = amount
account = BankAccount("Alice", 1000)
account.balance = 1500
print(account.balance)
9.9. Python RegEx¶
9.9.1. 1. What is RegEx¶
Regular Expressions (RegEx) provide a powerful way to search, match, and manipulate text patterns:
import re
text = "Python is powerful"
match = re.search("Python", text)
print(match.group()) # Python
9.9.2. 2. Using re.match()¶
import re
text = "Python Programming"
result = re.match("Python", text)
print(result.group()) # Python
match() checks for a pattern at the start of the string only.
9.9.3. 3. Using re.search()¶
import re
text = "Learn Python Programming"
result = re.search("Python", text)
print(result.group()) # Python
search() scans the entire string for the first match.
9.9.4. 4. Using re.findall()¶
import re
text = "The rain in Spain"
matches = re.findall("ai", text)
print(matches) # ['ai', 'ai']
9.9.5. 5. Using re.sub() (Replace Text)¶
import re
text = "I love Java"
new_text = re.sub("Java", "Python", text)
print(new_text) # I love Python
9.9.6. 6. Using Character Classes¶
import re
text = "abc123"
result = re.findall("[a-z]", text)
print(result) # ['a', 'b', 'c']
Common patterns:
[a-z] → lowercase letters
[A-Z] → uppercase letters
[0-9] → digits
9.9.7. 7. Metacharacters and Special Sequences¶
import re
text = "Email: test123@gmail.com"
pattern = r"\w+@\w+\.\w+"
match = re.search(pattern, text)
print(match.group())
Common sequences:
d → digit
w → alphanumeric
s → whitespace
9.9.8. 8. Using Quantifiers¶
import re
text = "Helloooo"
pattern = r"o+"
print(re.findall(pattern, text)) # ['oooo']
Quantifiers:
* → 0 or more
+ → 1 or more
? → 0 or 1
{n} → exact count
9.9.9. 9. Grouping with Parentheses¶
import re
text = "John Doe"
pattern = r"(John) (Doe)"
match = re.search(pattern, text)
print(match.groups()) # ('John', 'Doe')
9.9.10. 10. Real-World RegEx Example¶
import re
def validate_phone(phone):
pattern = r"^\+?\d{10,13}$"
return bool(re.match(pattern, phone))
print(validate_phone("+911234567890")) # True
print(validate_phone("12345")) # False