• https://agileabstractions.com/advanced/

Typically taught as a three day hands on course that will take you deeper into the Python programming language and ecosystem. This course will take delegates from beginner/intermediate level in Python to Advanced Python experts. The course provides a solid overview of the Python language including some low level details essential to working confidently and fluidly with Python. The focus is on practical programming and the skills learned here can be applied in any field where Python is used.

This course is taught by Michael Foord. Michael has been teaching Python for over a decade and has over twenty years industry experience as an application developer. Michael is a Python core developer and the creator of unittest.mock in the Python standard library, and is the author of “The Absolute Minimum Every Python Web Application Developer Must Know About Security”.

In this course delegates will learn a great deal of Python, from an essential foundation like how assignment works to taking advantage of multicore systems with multiprocessing. Included is networking, from API clients to understanding sockets and how servers work - the request response cycle of the REST API model, language features like closures, generators, context managers and the whole Python object model along with testing with pytest.

For smart programmers this course provides a solid foundation for working with Python along with many advanced language features and concepts and powerful libraries for tackling many common programming scenarios. As well as learning and discussion every section is backed by lab exercises.

Full list of modules available:

• Exceptions and the Call Stack
• Closures and Decorators
• Concurrency
• Context Managers and __init_subclass__
• Iterators, Generators, References and Assignment
• Imports, Modules and Namespaces
• Networking
• The Core Python Object Model
• Testing with pytest
• Floats, Unicode, and Regular Expressions
• List Comprehensions, Generator Expressions & Function Signatures
• Advanced Python OOP

An additional optional module “Data Science Overview” is available on request.

For several of the modules there are previews of the material in a series of “Python Knowledge Share Videos”:

• https://agileabstractions.com/python-knowledge-share-videos/

The materials/slides for these sessions can be found here:

• https://github.com/voidspace/talks

The Open Source Initiative (OST) is the authority that defines Open Source. They’ve published my guide to security for Python web applications as a three part series. *“The Absolute Minimum Every Python Web Application Developer Must Know About Security”.

Never store passwords in plain text. At a minimum you should be storing salted hashed passwords (which is probably what went wrong in the image here), even better use a key derivation function, but best of all delegate password management to a modern framework and keep it up to date. And never implement your own cryptography algorithms.

Data should be encrypted in transit and at rest. Just stealing your database shouldn’t reveal sensitive information because data is encrypted at rest. You can do this with the StringEncryptedType for SQLAlchemy for example. mTLS (Mutual TLS) is how we can keep data encrypted in transit, even on our internal networks. Just getting access to your network shouldn’t reveal any sensitive information because data is encrypted in transit.

These are just a few of the principles from a Defence in Depth, multi-layered, approach to security explored in this article series:

• Essential Python Web Security

  The Defence in Depth approach, important security principles, The OWASP Top Ten, the CVE warning system, authentication controls and security tooling.

• Security and cryptography algorithms: A guide

  Cryptography algorithms, using the cryptography library and the Python standard library. Hashing, encryption, key exchange protocols and public/private key signature algorithms with their use cases.

• TLS and Networking

  Much of our security is network security and much of that comes from TLS. We now understand all the crypto algorithms that make up TLS as a security system and can understand when, how and why to use TLS. We’ll also look at the request->response cycle of HTTP, the abstraction layer developers work at, and the security issues around networking. From firewalls to cookies via LANs and sockets.

(Being able to talk about the Blowfish Cipher and the Diffie-Hellmann Elliptic Curve Key Exchange Protocol is cyberpunk, so these articles are teaching real life skills.)

Thanks to Dr David Cassandra Mertz and PyDanny (Daniel Greenfeld) for improving these articles through technical review and to Nick Vidal (and volunteers) from the OSI for their work editing and formatting the articles.

A new one hour Knowledge Share video is online on YouTube.

• List Comprehensions, Generator Expressions & Function Signatures

This session is on working with data using enumerate, zip plus list comprehensions and generator expressions. Along the way we’ll talk about tuple unpacking, set and dictionary comprehensions and even nested list comprehensions.

We finish off with a look at function signatures: required and optional arguments plus positional and keyword arguments and positional only and keyword only arguments.

The slides/presentation can be downloaded here:

• https://github.com/voidspace/talks/tree/main/listcomprehensions

You can see all the sessions, eight so far, on a range of Python topics from generators to concurrency to testing with pytest on YouTube:

• Python Knowledge Share Videos on YouTube

The Open Source Initiative have published part one of an article of mine. The article is called “Essential Python Web Security” and it’s part one of a series called “The Absolute Minimum Every Python Web Application Developer Must Know About Security”. The subject is Full Stack Security for Python web applications, based on the Defence in Depth approach.

This series explores the critical security principles every Python web developer should know. Whilst hard and fast rules, like avoiding plaintext passwords and custom security algorithms, are essential - a deeper understanding of broader security principles is equally important. This first pots delves into fundamental security best practises, ranging from general principles to specific Python-related techniques.

• Essential Python Web Security: Part 1

Part 2, on Cryptographic Algorithms, will be published soon. When the series is complete it will probably also be available as an ebook. The full document, about fifty pages, can be read here:

• The Absolute Minimum Every Python Web Application Developer Must Know About Security

Special thanks to Gigaclear Ltd who sponsored the creation of this article. Also thanks to Dr David Mertz and Daniel Roy Greenfeld for technical reviews of this article prior to publication.

• Object Oriented Programming Theory with Python

Course Contents

Fundamentals: Classes and methods

• Computer architecture and programming languages
• Python as a high-level, object-oriented language
• Objects as abstractions, for thinking
• The class statement
• The explicit self
• The initialiser method __init__
• Bound methods
• Attributes and the built-in attribute access functions
• References and assignment (how Python works)
• Mutable objects (and call by object)
• Object copying

Object Oriented Features

• Class attributes
• Class methods
• Properties
• Private attributes
• Single inheritance
• Inheritance to extend and modify the parent
• The use of super
• Cooperative multiple inheritance
• Mixin Classes
• Attribute lookup and the method resolution order
• The type system: isinstance and issubclass

Inside Python Objects

• The instance dictionary
• Slots
• Class dictionaries
• The descriptor protocol

Python Protocols

• Magic methods and Python protocols
• Operator overloading
• The string representation protocol
• The container protocols
• Implementing custom containers
• Inheriting from collections.abc.MutableSequence

Alternative Approaches

• namedtuples
• dataclasses
• type as a class factory

Object oriented theory:

• History of Object Oriented Programming
• The pillars of OOP
• Abstraction
• Inheritance
• Encapsulation
• Polymorphism
• Object oriented design principles
• Design patterns
• The Liskov Substitution Principle
• Composition versus inheritance

Optional advanced section (third day):

1. Advanced OOP Concepts

   • Interfaces and API design
   • Abstract base classes and protocols
   • Type hints and static typing with mypy
   • Class decorators
   • Decoration via inheritance with __init_sublass__
   • Metaclasses
   • Context managers and the with statement
   • Weak references and destructors
   • The descriptor protocol

2. OOP Design Principles

   • SOLID principles
   • Law of Demeter
   • Liskov Substitution Principle
   • Composition vs inheritance
   • Domain Driven Design
   • Test Driven Development
   • Stop writing classes
   • The hexagon pattern (microservices)
   • The C4 Model for system architecture

• Secure Python Web Application Development

Course Contents

This is a practical and hands on, two day, course. Learn how to use the security tools that come in the Python standard library. Modules covered include:

• hashlib
• hmac
• secrets
• random
• socket
• ssl

Web application frameworks:

• Security features in web application frameworks for API servers and web applications
• How to secure data in Django, Flask and other popular web application frameworks
• Secure deployment practises with containers and application servers (WSGI or ASGI)

Third party libraries for cryptography and secure network access:

• authlib
• cryptography
• httpx and requests
• websocket
• jwt
• OpenZiti for application level zero trust architecture
• certifi for TLS certification verification

Tooling for secure Python development and as part of your CI pipelines:

• uv/pipenv
• pip-audit
• bandit
• ruff
• mypy
• dependabot/renovate
• Security testing

Network security with TLS:

• How, why and when to use TLS
• How TLS works
• mTLS for enhanced security
• Generating self-signed certificates for local development, with the cryptography library

Michael Foord has been a Python application developer for over twenty years, is a Python core developer and the creator of unittest.mock in the Python standard library, and is the author of The Absolute Minimum Every Python Web Application Developer Must Know About Security.

Topics Covered:

Core Python Security Fundamentals

• Security principles and defence in depth strategies with Python frameworks
• Implementing OWASP Top 10 protections in Django, Flask and FastAPI applications
• Security principles and defence in depth strategies
• Principles from The OWASP Web Security Testing Guide
• Threat modelling and the security requirements document
• The principles of least privilege and deny by default
• Zero trust architecture fundamentals
• Building zero trust architecture with OpenZiti’s Python SDK

Cryptography and Data Security

• Hashing, encryption, and digital signatures
• Symmetric encryption and public key encryption
• Secure password storage and management
• Using Python’s hashlib and hmac modules for secure hashing
• Using Python’s cryptography libraries correctly
• Data encryption at rest and in transit
• Data encryption at rest using Django’s encrypted model fields and SQLAlchemy StringEncryptedType

Authentication and Authorization

• Secure session management
• OAuth 2.0 and JWT for authentication
• Oauth2 with the Python library authlib
• JWT handling with PyJWT and managing token lifecycles
• Role-based access control (RBAC), plus alternatives
• Multi-factor authentication
• Managing access tokens and permissions

Secure Coding Practices

• Proven security with modern cryptography algorithms
• Protection against SQL injection
• Input validation and sanitisation
• Using secrets vs random for cryptographic operations
• Sanitizing logs in Python applications
• Django template escaping and Flask/Jinja2 for XSS prevention
• Preventing timing attacks and token prediction attacks
• Cryptographically secure randomness
• Cross-site scripting (XSS) prevention
• Cross-site request forgery (CSRF) protection
• Secure file handling and upload validation
• Preventing TLS downgrade attacks

Network Security

• Networking fundamentals
• TLS/SSL implementation and certificate management
• Secure API design and implementation
• WebSocket security
• Network architecture and segmentation
• Firewalls, routers, network interfaces
• Protocols, HTTP & TLS, with the Python standard library
• Application deployment
• Software Defined Networking

Infrastructure Security

• Container security best practices
• Secure deployment patterns
• Network interfaces and routing
• Building DMZ architectures for Python web applications
• Virtual private networks (VPN)

Security Tools and Testing

• Static analysis with bandit and ruff
• Dependency scanning using pip-audit
• Automated security testing integration
• Container scanning and runtime protection
• Code review practices for security

Operational Security

• Live security alerts
• Statutory duties around security
• Monitoring Python applications for security issues
• Security patch management for Python applications
• Updating and patching strategies

The course includes practical exercises throughout, with participants implementing secure coding patterns, identifying vulnerabilities in sample code, and building secure components.

I’ve seen this meme a bunch of times recently. I always reply; what is asserted without evidence may be dismissed without consideration.

Current Gen AI is flawed by hallucination issues, mired in copyright controversy, expensive to run and lacking clear use cases. (Although it’s pretty good at code generation). It’s a massive hype train.

Gen AI, as it is now, was made possible by the invention of “Transformer Architecture” by Google in 2017. We’re seeing fast paced change and development, but all built on that technology.

At some point another quantum breakthrough will change things all over again - and make another step towards AGI. Although it will take several such steps, and order of magnitudes larger models (and multi models), to create anything resembling true AI.

So a huge number of disparate individuals, institutions, governments and companies are pursuing the development of AI. There’s no single cohesive agenda behind it. As new technologies arise we adapt to them, find uses for them, and everyone pursues their agendas with them.

Not particularly special to AI I don’t think.

Python metaclasses, considered advanced programming and Python “black magick” (*) explained in eight words:

The type of a class is a class.

Here’s what knowledge of Object Oriented theory and type systems permit you to deduce from this:

Using the word “class”, instead of “the type of a class is type” or even “the type of a class is a type, classes are types”, implies that a user defined class can be a metaclass. This is indeed the case, and the point of metaclasses in Python.

The type is responsible for creating new instances. So if the type of a class is a class then we can write classes that create classes. Indeed this is the primary usecase for metaclasses.

(Deeper knowledge of Python, and the two phase object creation protocol, may lead you to deduce that this is done by overriding the __new__ method. If you’re familiar with “type” as a class factory you can probably even guess the signature and that you must inherit from type.)

If the type of a class is a class then the type system will permit a type check for the class against its class. And indeed isinstance(klass, metaclass) returns true.

(And deeper knowledge of Python will tell you that the magic methods, the protocol methods, are always looked up on the type. So we can implement behaviour for class objects by providing magic methods on the metaclass.)

All of this implies that classes are themselves objects. Which is true in Python for everything is an object in Python (and everything is a reference).

And so on…

• Type and class are synonyms in Python.
• type(type) is type

And to further round out the type system, these are also Python invariants:

• isinstance(object, object) is True # object is an object
• isinstance(object, type) is True # but also a type
• isinstance(type, object) is True # type is an object
• isinstance(type, type) is True # but also a type

(*) Like all black magick it is useful for understanding the world but never for actual use. Well, except perhaps in very rare circumstances if you know what you’re doing.

📘 Written in 2021.

IronPython in Action was published on the 7th April 2009 and we sold a little over 7000 copies.

Royalties for last quarter amounted to $25.

It took me two years to write thirteen chapters and a couple of appendices, and took Christian Muirhead about the same to write two chapters and an appendix. Jonathan Hartley did the diagrams and illustrations and the worst part was compiling the index.

It took so long because IronPython was still in alpha (!) when we started and it changed several times (including a Silverlight version being released) whilst writing!

After leaving Resolver Systems in 2010 I spent a year contracting on Line of Business apps that ran in Silverlight (Django on the server): Python code running in the browser on the client side. It was glorious.

We even had functional tests on unittest built in to the app.

Work on mock accelerated massively once IronPython in Action was complete. MagickMock was born not long afterwards.

I was also helping maintain the python.org website and adding test discovery to unittest at the time, and speaking at every conference I could find.

It felt like the glory days of the Python community. It’s almost time for PyCon (online) and I’m nostalgic once again.

My first PyCon, the second Dallas PyCon and my first time in the US, there were about 600 attendees. You could almost know everyone.

I shaved my beard to enter Dallas and wore my hair in a pony tail. All I knew was they didn’t like hippies there. It was the nicest greeting at a US airport I’ve ever had.

I went on a road trip with Andrzej Krzywda afterwards trying to find mountains. We found the Ouchita mountains in Oaklahoma and drove back through Arkansas to visit friends of mine in Houston. Along the peaks of the mountains, which are hills really, we found a view called Dead Man’s Vista and we I laughed together at Microsoft.

Not long after this the web explosion happened and Django happened, google adopted Python as an official language and the community started to explode and grow.

That was even before Python became huge as a teaching language and before Python exploded in data science too.

I once paired with Jacob Kaplan Moss at a PyCon sprint and fixed some issue by adding a metaclass to the Django codebase. Which he never committed and found a better way.

That’s the closest I’ve come to deploying a metaclass I think, although I’ve removed a few in my time.

I knew Python had “made it” as a language when one bag stuffing pre-PyCon I met someone who didn’t want to be there. He’d been sent by work. Before that Python was obscure, and only people who really loved it went to PyCon. Which I’m convinced is the secret of Python’s success.

It was built by passion not by money. For the sheer love and the joy of building something beautiful with other people.

I was a Mac user then and had a running joke with Jonathan Hartley about Linux and projectors.

One time he plugged his laptop into the projector prior to his PyCon talk (Testing is a Silver Bullet), tried to fix the x-config from the terminal and rendered his laptop unusable. He did the presentation on mine. The next year Mark Shuttleworth did a keynote talk at PyCon and running some bleeding edge version of Ubuntu also couldn’t plug it into the projector system. Hilarity on my part.

The biggest conference I ever spoke at was a Microsoft one in Brighton where they demoed Silverlight and I demoed IronPython on Silverlight. They didn’t tell me I would be on main stage in front of a few thousand Microsoft devs. I was used to talking to a few hundred at a time!

I had a slide deck built from S5 with reStructured Text markup and a Far Side slide mocking static typing. Which went down a bomb to an audience of C# devs. I still managed, by coincidence, to demo almost the same features of Silverlight as Microsoft bigwig Scott Hanselman who did the keynote.

It was an “interesting experience”, evangelising Python and dynamic languages in “the heart of the beast” as it were. Microsoft went on to step up their involvement with Python and sincere Open Source commitments which they’ve maintained since.

Since I first wrote this Python has finally made it, ranked as the most widely used programming language in the world by TIOBE and PyPL. World number one.

I joined Twitter fourteen years ago and have tweeted over fifty-two thousand times. I follow 1,636 accounts, which is too many, and have 8,670 followers. I use Tweetdeck which is run by Twitter and doesn’t show ads or promoted tweets or mess with tweet order and it lets me use two different accounts.

I use twitter a lot less than I did during my social media and community frenzy whilst I delighted to learn Python, but I still enjoy it.

During that time (2006-2011) I “drank from the firehose”. I read all of slashdot (scanned every headline and read relevant articles), read all of comp.lang.python (every message title - read and replied to many), read all of python-dev (similarly) and all of testing-in-python, blogged almost daily and worked full time as a software engineer commuting to London four times a week and developed mock in my spare time and worked on unittest in the Python standard library. And wrote a book and worked part time doing community liaison and service development for a local charity working with the homeless and disadvantaged. I was Microsoft MVP for three years for my work with IronPython, I spoke at countless conferences and received the Python Software Foundation Community Award for my work running Planet Python and helping out with the Python.org website and mailing infrastructure.

Then in 2011 my first child was born and I started working for Canonical. Three years of large Django web applications then three years of Go and MongoDB and then a year with Red Hat testing Ansible Tower and now four years self employed.

During that time I remembered that the primary drive in my life was spiritual and I started meditating again. One hour a day of mindfulness of breathing. That transformed my life all over again.

I once rode in the back of a beaten up station wagon owned and operated by the creator of the Python programming language whilst sat alongside the creator of Bitorrent, which was written in Python.

I also once had a pub lunch in Oxford with the creator of the Erlang programming language and the creator of the Haskell programming language. We were all three speaking at the ACCU conference. I was speaking on IronPython.

It’s been a fun journey.

I’ve been teaching Python in one hour knowledge sharing sessions, some of which I’ve put online on youtube.

This is the link to the playlist of the sessions:

• Python Knowledge Share Videos on YouTube

The slides for each of the sessions, along with some example code, can be found in this github repository:

• https://github.com/voidspace/talks

So far there are eight one-hour sessions (with more planned) on:

• Python Core Object Model
  • Python objects
  • Slots
  • Attribute lookup and the MRO
  • Inheritance, multiple inheritance and super
  • Inside Python objects and classes
• Closures and decorators (functional programming)
  • Functional programming: higher order functions and functions as objects
  • Lambdas
  • Closures: functions that build functions
  • Variable scoping: global, local and nonlocal
  • Decorators: functions wrapping functions
  • Decorator factories (decorators that take arguments)
  • Class decorators
  • Decorator order and using functools.wraps
• Generators and Iterators
  • The iteration protocol
  • Stateful iteration with generators
  • Adding iteration support to objects
  • References, assignment and mutability
  • Identity versus equality
  • Call by object
  • Object copying
• Unicode, Floats and regex
  • Floating point numbers
  • Unicode, encodings and strings
  • Regular expressions
• Concurrency (async, threads, processes, the GIL)
  • The history of concurrency from AmigaOS to a multi-core world
  • Python and the Global Interpreter Lock
  • I/O bound and CPU bound tasks
  • Threads and processes
  • Async programming (green threading, coroutines)
  • Concurrency with threads
  • Concurrency with multiprocessing
  • Looking to the future (Python 3.13): optional GIL (PEP 703) and subinterpreters (PEP 554)
• Testing with pytest
  • virtual environments and pipenv (installing pytest)
  • pytest command line for collecting and running tests
  • Simple test functions and asserts
  • Test fixtures and conftest.py
  • Testing exceptions
  • Test parameterisation for test combinations
  • Test marking for running test subsets
  • Principles of testing (unit tests versus end to end testing, building test helpers etc)
  • Mocking and patching
• Modules and Namespaces
  • Import syntax variations
  • namespaces and variable lookups
  • sys.modules and the import cache
  • Module objects
  • Module level functionality: __dir__ and __getattr__
  • Packages and the filesystem
  • Relative import syntax
  • Module reloading (how to do it and why not to do it)
  • Circular imports, avoiding and fixing
  • Executable modules and packages
• List Comprehensions, Generator Expressions & Function Signatures
  • zip and enumerate, builtin functions
  • Tuple unpacking, iterating over pairs
  • List comprehensions
  • Generator Expressions
  • Set and dictionary comprehensions
  • Nested comprehensions
  • Function signatures: required and optional arguments
  • Positional and keyword arguments
  • Positional only and keyword only arguments

Other Talks

A selection of some of the talks and interviews I’ve given on Python and software engineering across my career.

• TLS and Networking (article, Jan 2025)
• Security and cryptography algorithms: A guide (article, Jan 2025)
• Essential Python Web Security (article, Dec 2024)
• UK Health Security Agency Software Development Practise Conf 2024
• PyCon UK 2023, Metaclasses in 5 Minutes Lightning Talk
• PyCon MEA 2022 How Python Took Over the World
• Test and Code Podcast Episode 145: For Those About to Mock
• PyCon Belarus 2020 How Python Took Over the World
• PyLondinium 2019 The Python Object Model
• Interview on Podcast.__init__ on testing, Mock and the Python community (2018)
• The Role of Abstractions: Lightning Talk PyCon US 2018
• Best Practises for Software Development and Testing (2017)
• PyCon UK Panel 2015
• To the Clouds: EuroPython 2015
• Automated Deployments with Juju: PyCon UK 2014
• Python and Pythons: PyCon NZ 2013
• Testing with Mock: PyCon US 2011
• A Little Bit of Python Podcast (2010-11)
• New and Improved unittest 2: PyCon US 2010
• Michael Foord on IronPython: Hanselminutes 2009
• IronPython in Action (2009, book)
• Michael Foord on IronPython: TechEd 2007