1000 Python Examples [Leanpub PDF/iPad/Kindle]

Fixtures and Mocking in Python

How do you test Moon-landing?
How do you test a system …
Plan
About me
Goal
Fixtures
Fixtuers in Pytest
Traditional xUnit fixtures
Dependency Injection
Temporary directory - tmpdir
Capture STDOUT and STDERR - capsys
Home-made fixture
Home-made fixture - conftest
Home-made fixture with tempdir
Home-made fixture with yield
Fixture Autouse
Fixture Autouse with yield
Fixture for MongoDB
Test Doubles
Test Doubles explained
Verify behavior or state?
What is Mocking and Monkey Patching?
Situations
Unit testing vs. Integration testing
Experiment with mocking in various situations
Examples are simple
Hard coded path
Manually Patching attribute
Monkey Patching attribute
Monkey Patching functions
Monkey Patching dictionary items
Mocking a whole class
Mocking input/output
Mocking input/output
Mocking random numbers
Exercises
Work in pairs
Exercise: test login expiration
Solution: test login expiration
Exercise: Record e-mail sending
Solution: Record e-mail sending
Exercise: Fixture database
Exercise: One Dimentsional space-fight
Exercise: web client
Exercise: Open WeatherMap client
Exercise: Mocking A Bank
Testing the whole application
Resources
Retrospective
Job searching help
Solutions - game
Solutions - Mocking the database access

First steps

What is Python?
What is needed to write a program?
The source (code) of Python
Python 2 vs. Python 3
Installation
Installation on Linux
Installation on Apple Mac OSX
Installation on MS Windows
Editors, IDEs
Documentation
Program types
Python on the command line
First script - hello world
Examples
Comments

marks single line comments.

Variables
Exercise: Hello world
What is programming?
What are the programming languages
A written human language
A programming language
Words and punctuation matter!
Literals, Value Types in Python
Floating point limitation
Value Types in Numpy
Rectangle (numerical operations)
Multiply string
Add numbers
Add strings
Exercise: Calculations
Solution: Calculations

Second steps

Modules
A main function
The main function - called
Indentation
Conditional main
Input - Output I/O
print in Python 2
print in Python 3
print in Python 2 as if it was Python 3
Exception: SyntaxError: Missing parentheses in call
Prompting for user input in Python 2
Prompting for user input in Python 3
Python2 input or raw_input?
Prompting both Python 2 and Python 3
Add numbers entered by the user (oups)
Add numbers entered by the user (fixed)
How can I check if a string can be converted to a number?
Converting string to int
Converting float to int
Conditionals: if
Conditionals: if - else
Divide by 0
Conditionals: if - else (other example)
Conditionals: else if
Conditionals: elif
Ternary operator (Conditional Operator)
Case or Switch in Python
Exercise: Rectangle
Exercise: Calculator
Exercise: Age limit
Exercise: What is this language?
Exercise: Standard Input
Solution: Area of rectangle
Solution: Calculator
Solution: Calculator eval
Solution: Age limit
Solution: What is this language?
Command line arguments
Command line arguments - len
Command line arguments - exit
Exercise: Rectangle (argv)
Exercise: Calculator (argv)
Solution: Area of rectangle (argv)
Solution: Calculator (argv)
Solution: Calculator eval
Compilation vs. Interpretation
Is Python compiled or interpreted?
Flake8 checking
Pylint checking

Numbers

Numbers
Operators for Numbers
Integer division and the future
Pseudo Random Number (unform distribution)
Fixed random numbers
Rolling dice - randrange
Random choice
built-in method
Exception: TypeError: ‘module’ object is not callable
Fixing the previous code
Exception: AttributeError: module ‘random’ has no attribute
Exercise: Number guessing game - level 0
Exercise: Fruit salad
Solution: Number guessing game - level 0
Solution: Fruit salad

Comparison and Boolean

if statement again
Comparison operators
Compare numbers, compare strings
Do NOT Compare different types!
Complex if statement with boolean operators
Boolean operators
Boolean truth tables
Boolean values: True and False
Using True and False in variables
Comparison returns True or False
Assign comparisons to variables
Flag
Toggle
Short circuit
Short circuit fixed
Does this value count as True or False?
True and False values in Python
Incorrect use of conditions
Exercise: compare numbers
Exercise: compare strings
Solution: compare numbers
Solution: compare strings

Strings

Single quoted and double quoted strings
Long lines
Triple quoted strings (multiline)
String length (len)
String repetition and concatenation
A character in a string
String slice (instead of substr)
Change a string
How to change a string
String copy
String functions and methods (len, upper, lower)
index in string
index in string with range
rindex in string with range
find in string
Find all in the string
in string
index if in string
Encodings: ASCII, Windows-1255, Unicode
raw strings
ord
ord in a file
chr - number to character
Exercise: one string in another string
Exercise: to ASCII CLI
Exercise: from ASCII CLI
Solution: one string in another string
Solution: compare strings
Solution: to ASCII CLI
Solution: from ASCII CLI

Loops

Loops: for-in and while
for-in loop on strings
for-in loop on list
for-in loop on range
Iterable, iterator
for in loop with early end using break
for in loop skipping parts using continue
for in loop with break and continue
while loop
Infinite while loop
While with complex expression
While with break
While True
Duplicate input call
Eliminate duplicate input call
do while loop
while with many continue calls
Break out from multi-level loops
Exit vs return vs break and continue
Exercise: Print all the locations in a string
Exercise: Number guessing game
Exercise: Count unique characters
Exercise: Convert for-loop to while-loop
Solution: Print all the locations in a string
Solution 1 for Number Guessing
Solution 2 for Number Guessing (x)
Solution 3 for Number Guessing (s)
Solution for Number Guessing (debug)
Solution for Number Guessing (move)
Solution for Number Guessing (multi-game)
Solution: Count unique characters
Solution: Convert for-loop to while-loop

Formatted printing

format - sprintf
Examples using format - indexing
Examples using format with names
Format columns
Examples using format - alignment
Format - string
Format characters and types
Format floating point number
f-strings (formatted string literals)
printf using old %-syntax
Format braces, bracket, and parentheses
Examples using format with attributes of objects
raw f-strings

Lists

Anything can be a list
Any layout
Lists
List slice with steps
Change a List
Change with steps
List assignment and list copy
Shallow vs. Deep copy of lists
join
join list of numbers
split
for loop on lists
in list
Where is the element in the list
Index improved
[].insert
[].append
[].remove
Remove element by index [].pop
Remove first element of list
Remove several elements of list by index
Use list as a queue
Queue using deque from collections
Fixed size queue
List as a stack
stack with deque
Exercies: Queue
Exercise: Stack
Exercise: MasterMind
Solution: Queue with list
Solution: Queue with deque
Solution: Reverse Polish calculator (stack) with lists
Solution: Reverse Polish calculator (stack) with deque
Solution: MasterMind
MasterMind to debug
Debugging Queue
sort
sort numbers
sort mixed
key sort
sort with sorted
sort vs. sorted
key sort with sorted
Sorting characters of a string
range
Looping over index
Enumerate lists
List operators
List of lists
List assignment
List documentation
tuple
Sort tuples
Exercise: color selector menu
Exercise: count digits
Exercise: Create list
Exercise: Count words
Exercise: Check if number is prime
Exercise: DNA sequencing
Solution: menu
Solution: count digits
Solution: Create list
Solution: Count words
Solution: Check if number is prime
Solution: DNA sequencing
Solution: DNA sequencing with filter
Solution: DNA sequencing with filter and lambda
[].extend
append vs. extend
split and extend

Files

File types: Text vs Binary
Open vs. Read vs. Load
Binary files: Images
Reading an Excel file
Open and read file (easy but not recommended)
Open and read file using with (recommended)
Read file remove newlines
Filename on the command line
Filehandle with return
Read all the lines into a list
Read all the characters into a string (slurp)
Not existing file
Open file exception handling
Open many files - exception handling
Writing to file
Append to file
Binary mode
Does file exist? Is it a file?
Direct access of a line in a file
Exercise: count numbers
Exercise: strip newlines
Exercise: print lines with Report:
Exercise: color selector
Exercise: ROT13
Exercise: Combine lists
Solution: count numbers
Solution: strip newlines
Solution: print lines with Report:
Solution: color selector
Solution: Combine lists
Filehandle using with and not using it

Dictionary (hash)

What is a dictionary
When to use dictionaries
Dictionary
keys
Loop over keys
Loop over dictionary keys
Loop using items
values
Not existing key
Get key
Does the key exist?
Does the value exist?
Delete key
List of dictionaries
Shared dictionary
immutable collection: tuple as dictionary key
immutable numbers: numbers as dictionary key
Sort dictionary by value
Sort dictionary keys by value
Insertion Order is kept
Change order of keys in dictionary - OrderedDict
Set order of keys in dictionary - OrderedDict
Exercise: count characters
Exercise: count words
Exercise: count words from a file
Exercise: Apache log
Exercise: Combine lists again
Exercise: counting DNA bases
Exercise: Count Amino Acids
Exercise: List of dictionaries
Exercise: Dictinoary of dictionaries
Exercise: Age limit with dictionaries
Solution: count characters
Default Dict
Solution: count characters with default dict
Solution: count words
Solution: count words in file
Solution: Apache log
Solution: Combine lists again
Solution: counting DNA bases
Solution: Count Amino Acids
Do not change dictionary in loop

Sets

sets
set operations
Creating a set
Creating an empty set
Adding an element to a set (add)
Merging one set into another set (update)
set intersection
set subset
set symmetric difference
set union
set relative complement

Functions (subroutines)

Why use functions?
Defining simple function
Passing positional parameters to a function
Function parameters can be named
Mixing positional and named parameters
Default values, optional parameters, optional parameters
Default value in first param
Several defaults, using names
Arbitrary number of arguments *
Fixed parmeters before the others
Arbitrary key-value pairs in parameters **
Extra key-value pairs in parameters
Every parameter option
Duplicate declaration of functions (multiple signatures)
Pylint duplicate declaration
Return more than one value
Recursive factorial
Recursive Fibonacci
Non-recursive Fibonacci
Unbound recursion
Variable assignment and change - Immutable
Variable assignment and change - Mutable
Parameter passing of functions
Passing references
Function documentation
Sum ARGV
Copy-paste code
Copy-paste code fixed
Copy-paste code further improvement
Palindrome
Exercise: statistics
Exercise: recursive
Exercise: Tower of Hanoi
Exercise: Merge and Bubble sort
Exercise: Refactor previous solutions to use functions
Solution: statistics
Solution: recursive
Solution: Tower of Hanoi
Solution: Merge and Bubble sort

Modules

Before modules
Create modules
path to load modules from - The module search path
sys.path - the module search path
Flat project directory structure
Absolute path
Relative path
Python modules are compiled
How “import” and “from” work?
Runtime loading of modules
Conditional loading of modules
Duplicate importing of functions
Script or library
Script or library - import
Script or library - from import
assert to verify values
mycalc as a self testing module
doctest
Scope of import
Export import
Export import with all
import module
Execute at import time
Import multiple times
Exercise: Number guessing
Exercies: Scripts and modules
Exercise: Module my_sum
Exercise: Convert your script to module
Exercise: Add doctests to your own code
Solution: Module my_sum

Regular Expressions

What are Regular Expressions (aka. Regexes)?
What are Regular Expressions good for?
Examples
Where can I use it ?
grep
Regexes first match
Match numbers
Capture
Capture more
Capture even more
findall
findall with capture
findall with capture more than one
Any Character
Match dot
Character classes
Common characer classes
Negated character class
Optional character
Regex 0 or more quantifier
Quantifiers
Quantifiers limit
Quantifiers on character classes
Greedy quantifiers
Minimal quantifiers
Anchors
Anchors on both end
Match ISBN numbers
Matching a section
Matching a section - minimal
Matching a section negated character class
DOTALL S (single line)
MULTILINE M
Two regex with logical or
Alternatives
Grouping and Alternatives
Internal variables
More internal variables
Regex DNA
Regex IGNORECASE
Regex VERBOSE X
Substitution
findall capture
Fixing dates
Duplicate numbers
Remove spaces
Replace string in assembly code
Full example of previous
Split with regex
Exercises: Regexes part 1
Exercise: Regexes part 2
Exercise: Sort SNMP numbers
Exercise: parse hours log file and give report
Exercise: Parse ini file
Exercise: Replace Python
Exercise: Extract phone numbers
Solution: Sort SNMP numbers
Solution: parse hours log file and give report
Solution: Processing INI file manually
Solution: Processing config file
Solution: Extract phone numbers
Regular Expressions Cheat sheet
Fix bad JSON
Fix very bad JSON
Raw string or escape
Remove spaces regex
Regex Unicode
Anchors Other example

PyCharm

PyCharm Intro
PyCharm configurations
PyCharm Project
PyCharm Files
PyCharm - run code
PyCharm Python console at the bottom left
Refactoring example with PyCharm

Python standard modules

Some Standard modules
sys
Writing to standard error (stderr)
Current directory (getcwd, pwd, chdir)
OS dir (mkdir, makedirs, remove, rmdir)
python which OS are we running on (os, platform)
Get process ID
OS path
Traverse directory tree - list directories recursively
os.path.join
Directory listing
expanduser - handle tilde ~
Listing specific files using glob
External command with system
subprocess
subprocess in the background
Accessing the system environment variables from Python
Set env and run command
shutil
time
sleep in Python
timer
Current date and time datetime now
Converting string to datetime
datetime arithmeticis
Rounding datetime object to nearest second
Signals and Python
Sending Signal
Catching Signal
Catching Ctrl-C on Unix
Catching Ctrl-C on Unix confirm
Alarm signal and timeouts
deep copy list
deep copy dictionary
Exercise: Catching Ctrl-C on Unix 2nd time
Exercise: Signals
Ctrl-z

JSON

JSON - JavaScript Object Notation
dumps
loads
dump
load
Round trip
Pretty print JSON
Sort keys in JSON
Set order of keys in JSON - OrderedDict
Exercise: Counter in JSON
Exercise: Phone book
Exercise: Processes
Solution: Counter in JSON
Solution: Phone book

Command line arguments with argparse

Modules to handle the command line
argparse
Basic usage of argparse
Positional argument
Many positional argument
Convert to integers
Convert to integer
Named arguments
Boolean Flags
Short names
Exercise: Command line parameters
Exercise: argparse positional and named
argparse print help explicitely
Argparse xor - mutual exlucise - only one - exactly one

Exception handling

Hierarchy of calls
Handling errors as return values
Handling errors as exceptions
A simple exception
Working on a list
Catch ZeroDivisionError exception
Module to open files and calculate something
File for exception handling example
Open files - exception
Handle divide by zero exception
Handle files - exception
Catch all the exceptions and show their type
List exception types
Exceptions
How to raise an exception
Stack trace
Exercies: Exception int conversion
Exercies: Raise Exception
Solution: Exception int conversion (specific)
Solution: Exception int conversion (all other)
Solution: Raise Exception

Classes - OOP - Object Oriented Programming

Why Object Oriented Programming?
Generic Object Oriented Programming terms
OOP in Python
OOP in Python (numbers, strings, lists)
OOP in Python (argparse)
Create a class
Import module containing class
Import class from module
Initialize a class - constructor, attributes
Attributes are not special
Create Point class
Initialize a class - constructor, attributes
Methods
Stringify class
Inheritance
Inheritance - another level
Modes of method inheritance
Modes of method inheritance - implicit
Modes of method inheritance - override
Modes of method inheritance - extend
Modes of method inheritance - delegate - provide
Composition - Line
Some comments
Class in function
Serialization of instances with pickle
Quick Class definition and usage
Exercise: Add move_rad to based on radians
Exercise: Improve previous examples
Exercise: Polygon
Exercise: Number
Exercise: Library
Exercise: Bookexchange
Exercise: Represent turtle graphics
Solution - Polygon

PyPi - Python Package Index

What is PyPi?
Easy Install
pip
Upgrade pip
PYTHONPATH
Virtualenv
Virtualenv for Python 3

SQLite Database Access

SQLite
Connecting to SQLite database
Create TABLE in SQLite
INSERT data into SQLite database
SELECT data from SQLite database
A counter

MySQL

Install MySQL support
Create database user (manually)
Create database (manually)
Create table (manually)
Connect to MySQL
Connect to MySQL and Handle exception
Select data
Select more data
Select all data fetchall
Select some data fetchmany
Select some data WHERE clause
Select into dictionaries
Insert data
Update data
Delete data
Exercise MySQL
Exercise: MySQL Connection
Solution: MySQL Connection

PostgreSQL

PostgreSQL install
Python and Postgresql
PostgreSQL connect
INSERT
INSERT (from command line)
SELECT
DELETE

SQLAlchemy

SQLAlchemy hierarchy
SQLAlchemy engine
SQLAlchemy autocommit
SQLAlchemy engine CREATE TABLE
SQLAlchemy engine INSERT
SQLAlchemy engine SELECT
SQLAlchemy engine SELECT all
SQLAlchemy engine SELECT fetchall
SQLAlchemy engine SELECT aggregate
SQLAlchemy engine SELECT IN
SQLAlchemy engine SELECT IN with placeholders
SQLAlchemy engine connection
SQLAlchemy engine transaction
SQLAlchemy engine using context managers
Exercise: Create table
SQLAlchemy Metada
SQLAlchemy types
SQLAlchemy ORM - Object Relational Mapping
SQLAlchemy ORM create
SQLAlchemy ORM schema
SQLAlchemy ORM reflection
SQLAlchemy ORM INSERT after automap
SQLAlchemy ORM INSERT
SQLAlchemy ORM SELECT
SQLAlchemy ORM SELECT cross tables
SQLAlchemy ORM SELECT and INSERT
SQLAlchemy ORM UPDATE
SQLAlchemy ORM logging
Solution: Create table
Exercise: Inspector
SQLAlchemy CREATE and DROP
SQLAlchemy Notes
SQLAlchemy Meta SQLite CREATE
SQLAlchemy Meta Reflection
SQLAlchemy Meta INSERT
SQLAlchemy Meta SELECT

NoSQL

Types of NoSQL databases

MongoDB

MongoDB CRUD
Install MongoDB support
Python MongoDB insert
MongoDB CLI
Python MongoDB find
Python MongoDB find refine
Python MongoDB update
Python MongoDB remove (delete)
Python MongoDB replace
Python MongoDB upsert
Python Mongodb: TypeError: upsert must be True or False

Redis

Redis CLI
Redis list keys
Redis set get
Redis incr
Redis incrby
Redis setex

Web client

urllib the web client
urllib2 the web client
httpbin.org
requests get
Download image using requests
Download image as a stream using requests
Download zip file
Extract zip file
Interactive Requests
requests get JSON
requests get JSON UserAgent
requests get JSON UserAgent
requests get header
requests change header
requests post
Tweet
API config file
bit.ly
Exercise: Combine web server and client

Python Web server

Hello world web
Dump web environment info
Web echo
Web form
Resources

Python Flask

Python Flask intro
Python Flask installation
Flask: Hello World
Flask: Run Hello World
Flask: testing hello world
Flask generated page - time
Flask generated page - time tested
Flask: Echo GET
Flask: Echo POST
Flask: templates
Flask: templates
Flask: templates with parameters
Flask: runner
Exercise: Flask calculator
Static files
Flask Logging
Flask: Counter
Color selector without session
Session management
Flask custom 404 page
Flask Error page
Flask URL routing
Flask Path params
Flask Path params (int)
Flask Path params add (int)
Flask Path params add (path)
Jinja loop, conditional, include
Exercise: Flask persistent
Exercise: Flask persistent
Flask Exercises
Flask login
Flask JSON API
Flask and AJAX
Flask and AJAX
passlib
Flask Testing
Flask Deploy app
Flask Simple Authentication + test
Flask REST API
Flask REST API - Echo
Flask REST API - parameters in path
Flask REST API - parameter parsing
Flask REST API - parameter parsing - required

Networking

Secure shell
ssh
ssh from Windows
Parallel ssh
telnet
prompt for password
Python nmap
ftp

Interactive shell

The Python interactive shell
REPL - Read Evaluate Print Loop
Using Modules
Getting help
Exercise: Interactive shell

Testing Demo

How do you test your code?
What is testing?
What is testing really?
Testing demo tools
Testing demo methodology
Testing demo - AUT - Application Under Test
Testing demo - use the module
Testing demo: doctest
Testing demo: doctest with failure
Testing demo: Unittest success
Testing demo: Unittest failure
Testing demo: pytest using classes
Testing demo: pytest using classes - failure
Testing demo: pytest without classes
Testing demo: pytest run doctests
Testing demo: pytest run unittest
Exercise: Testing demo
Solution: Testing demo

Types in Python

mypy
Types of variables
Types of function parameters
Types used properly
TODO: mypy

Testing Intro

The software testing equasion
The software testing equasion (fixed)
The pieces of your software?
Manual testing
What to tests?
Continuous Integration

Functional programming

Functional programming
Iterators (Iterables)
range
range with list
range vs. list size
for loop with transformation
map
map delaying function call
map on many values
map with list
double with lambda
What is lambda in Python?
lambda returning tuple
map returning tuples
lambda with two parameters
map for more than one iterable
map on uneven lists
replace None (for Python 2)
map on uneven lists - fixed (for Python 2)
map mixed iterators
map fetch value from dict
Exercise: string to length
Exercise: row to length
Exercise: compare rows
Solution: string to length
Solution: row to length
Solution: compare rows
filter
filter with lambda
filter - map example
filter - map in one expression
Get indexes of values
reduce
reduce with default
zip
Creating dictionary from two lists using zip
all, any
Compare elements of list with scalar
List comprehension - double
List comprehension - simple expression
List generator
List comprehension
Dict comprehension
Lookup table with lambda
Read lines without newlines
Read key-value pairs
Create index-to-value mapping in a dictionary based on a list of values
Exercise: min, max, factorial
Exercise: Prime numbers
Exercise: Many validator functions
Exercise: Calculator using lookup table
Exercise: parse file
Solution: min, max, factorial
Solution: Prime numbers
Solution: Many validator functions
Solution: Calculator using lookup table
map with condtion
map with lambda
map with lambda with condition
List comprehension - complex

Iterators - with and without Itertools

Advantages of iterators and generators
The Fibonacci research institute
Fibonacci plain
Fibonacci copy-paste
Iterators Glossary
What are iterators and iterables?
A file-handle is an iterator
range is iterable but it is not an iterator
Iterator: a counter
Using iterator
Iterator without temporary variable
The type of the iterator
Using iterator with next
Mixing for and next
Iterable which is not an iterator
Iterator returning multiple values
Range-like iterator
Unbound or infinite iterator
Unbound iterator Fibonacci
Operations on Unbound iterator
itertools
itertools - count
itertools - cycle
Exercise: iterators - reimplement the range function
Exercise: iterators - cycle
Exercise: iterators - alter
Exercise: iterators - limit Fibonacci
Exercise: iterators - Fibonacci less memory
Exercise: read char
Exercise: read section
Exercise: collect packets
Exercise: compare files
Solution: iterators - limit Fibonacci
Solution: iterators - Fibonacci less memory
Solution: read section
Solution: compare files
Solution: collect packets

Generators and Generator Expressions

Generators Glossary
Iterators vs Generators
List comprehension and Generator Expression
List comprehension vs Generator Expression - less memory
List comprehension vs Generator Expression - lazy evaluation
Generator: function with yield - call next
Generators - call next
Generator with yield
Generators - fixed counter
Generators - counter
Generators - counter with parameter
Generators - my_range
Fibonacci - generator
Infinite series
Integers
Integers + 3
Integers + Integers
Filtered Fibonacci
The series.py
generator - unbound count (with yield)
iterator - cycle
Exercise: Alternator
Exercise: Prime number generator
Exercise: generator
Exercise: Tower of Hanoi
Exercise: Binary file reader
Exercise: File reader with records

Logging

Simple logging
Simple logging - set level
Simple logging to a file
Simple logging format
Simple logging change date format
getLogger
Time-based logrotation
Size-based logrotation

Closures

Counter local - not working
Counter with global
Create incrementors
Create internal function
Create function by a function
Create function with parameters
Counter closure
Make incrementor with def (closure)
Make incrementor with lambda
Exercise: closure bank
Solution: closure bank
Solution: counter with parameter

Decorators

Function assignment
Function inside other function
Decorator
Use cases for decorators in Python
A recursive Fibonacci
trace fibo
tron decorator
Decorate with direct call
Decorate with parameter
Decorator accepting parameter
Decorate function with any signature
Decorate function with any signature - implementation
Exercise: Logger decorator
Exercise: memoize decorator
Solution: Logger decorator
Solution: Logger decorator (testing)
Solution memoize decorator

Context managers (with statement)

Why use context managers?
Context Manager examples
cd in a function
open in function
open in for loop
open in function using with
Plain context manager
Param context manager
Context manager that returns a value
Use my tempdir - return
Use my tempdir - exception
cwd context manager
tempdir context manager
Context manager with class
Context managers with class
Context manager: with for file
With - context managers
Exercise: Context manager
Exercise: Tempdir on Windows
Solution: Context manager

Advanced lists

Change list while looping: endless list
Change list while looping
Copy list before iteration
for with flag
for else
enumerate
do while
list slice is copy

Advanced Exception handling

Exceptions else
Exceptions finally
Exit and finally
Catching exceptions
Home made exception
Home made exception with attributes
Home made exception hierarcy
Home made exception hierarcy - 1
Home made exception hierarcy - 2
Home made exception hierarcy - 3
Exercise: spacefight with exceptions
Exercies: Raise My Exception
Solution: spacefight with exceptions
Solution: Raise My Exception
Exception finally return

Warnings

Warnings

CSV

Reading CSV the naive way
CSV with quotes and newlines
Reading a CSV file
CSV dialects
CSV to dictionary
Exercise: CSV
Solution: CSV

Excel

Spreadsheets
Python Excel
Create an Excel file from scratch
Worksheets in Excel
Add expressions to Excel
Format field
Number series and chart
Read Excel file
Update Excel file
Exercise: Excel

XML

XML Data
Expat - Callbacks
XML DOM - Document Object Model
XML SAX - Simple API for XML
SAX collect
XML elementtree

SciPy - for Scientific Computing in Python

Data Science tools in Python
Data Analysis resources

Python and Biology

Biopython
Biopython background
Bio python sequences
Download data
Read FASTA, GenBank files
Search nucleotids
Download nucleotids
Exercise: Nucleotid
Biology background

Chemistry

Chemistry links
Bond length
Covalent radius
Python energy landscape explorer
Other chemistry links

numpy

What is NumPy
Numpy - vector
NumPy 2D arrays
Numpy - set type
NumPy arrays: ones and zeros
Numpy: eye
NumPy array random
NumPy Random integers
NumPy array type change by division (int to float)
Numpy: Array methods: transpose
Numpy: reference, not copy
Numpy: copy array
Numpy: Elementwise Operations on Arrays
Numpy: multiply, matmul, dot for vectors
Numpy: multiply, matmul, dot for vector and matrix
Numpy: multiply, matmul, dot for matrices
Numpy: casting - converting from strings to integer.
Numpy: indexing 1d array
Numpy: slice is a reference
Numpy: slice - copy
Numpy: abs value on a Numpy array
Numpy: Logical not on a Numpy array
Numpy: Vectorize a function
Numpy: Vectorize len
Numpy: Vectorize lambda
Numpy: Filtering array
Numpy: Filter matrix values
Numpy: Filter matrix rows
Numpy: Stat
Numpy: Serialization
Numpy: Load from Matlab file
Numpy: Save as Matlab file
Numpy: Horizontal stack vectors (hstack)
Numpy: Append or vertically stack vectors and matrices (vstack)
Numpy uint8
Numpy int8

Pandas

Pandas
Planets
Pandas Planets - Dataframes
Pandas Stocks
Pandas Stocks
Merge Dataframes
Analyze Alerts
Analyze IFMetrics
Create Excel file for experiment with random data
Calculate Genome metrics
Calculate Genome metrics - add columns
Calculate Genome metrics - vectorized
Calculate Genome metrics - vectorized numpy
Genes using Jupyter
Combine columns
Pandas more
Pandas Series
Pandas Series with names

Matplotlib

About Matplotlib
Matplotlib Line
Matplotlib Line with dates
Matplotlib Simple Pie
Matplotlib Simple Pie with params
Matplotlib Pie
Matplotlib Pie 2
Plot, scatter, histogram

Seaborn

Searborn use examples
Seaborn tip
Seaborn Anscombes Quartet

Jupyter notebooks

Jupyter on Windows
Jupyter on Linux and OSX
Jupyter add
Planets
Jupyter notebook Planets
Jupyter StackOverflow
Jupyter StackOverflow - selected columns
Jupyter processing chunks
Jupyter StackOverflow - selected rows
Jupyter StackOverflow - biggest countries (in terms of number of responses)
Jupyter StackOverflow - historgram
Jupyter StackOverflow - filter by country
Jupyter StackOverflow - OpenSourcer
Jupyter StackOverflow - cross tabulation
Jupyter StackOverflow - salaries
Jupyter StackOverflow - replace values
Jupyter StackOverflow - selected rows
Jupyter notebook Intellisense (TAB completition)
Jupyter examples
IPy Widgets

Testing

Traditional Organizations
Quality Assurance
Web age Organizations
TDD vs Testing as an Afterthought
Why test?
Testing Modes
Testing Applications
Testing What to test?
Testing in Python
Testing Environment
Testing Setup - Fixture
Testing Resources

Testing with unittest

Use a module
Test a module
The tested module
Testing - skeleton
Testing
Test examples

Testing with PyTest

Pytest features
Pytest setup
Testing with Pytest
Testing functions
Testing class and methods
Pytest - execute
Pytest - execute
Pytest simple module to be tested
Pytest simple tests - success
Pytest simple tests - success output
Pytest simple tests - failure
Pytest simple tests - failure output
Exercise: test math functions
Exercise: test this app
Exercise: test the csv module
Solution: Pytest test math functions
Solution: Pytest test this app
Solution: test the csv module
PyTest bank deposit
PyTest expected exceptions (bank deposit)
PyTest expected exceptions (bank deposit) - no exception happens
PyTest expected exceptions (bank deposit) - different exception is raised
PyTest expected exceptions
PyTest expected exceptions output
PyTest expected exceptions (text changed)
PyTest expected exceptions (text changed) output
PyTest expected exceptions (other exception)
PyTest expected exceptions (other exception) output
PyTest expected exceptions (no exception)
PyTest expected exceptions (no exception) output
PyTest: Multiple Failures
PyTest: Multiple Failures output
PyTest Selective running of test functions
PyTest: stop on first failure
Pytest: expect a test to fail (xfail or TODO tests)
Pytest: expect a test to fail (xfail or TODO tests)
PyTest: show xfailed tests with -rx
Pytest: skipping tests
Pytest: show skipped tests woth -rs
Pytest: show extra test summmary info with -r
Pytest: skipping tests output in verbose mode
Pytest verbose mode
Pytest quiet mode
PyTest print STDOUT and STDERR using -s
PyTest failure reports
PyTest compare numbers
PyTest compare numbers relatively
PyTest compare strings
PyTest compare long strings
PyTest is one string in another strings
PyTest test any expression
PyTest element in list
PyTest compare lists
PyTest compare short lists
PyTest compare short lists - verbose output
PyTest compare dictionaries
PyTest compare dictionaries output
PyTest Fixtures
PyTest Fixture setup and teardown
PyTest Fixture setup and teardown output
PyTest: Class setup and teardown
PyTest: Class setup and teardown output
Pytest Dependency injection
Pytest fixture - tmpdir
Pytest capture STDOUT and STDERR with capsys
Pytest Fixture - home made fixtures
More fixtures
Pytest: Mocking - why?
Pytest: Mocking - what?
Pytest: One dimensional spacefight
Pytest: Mocking input and output
Pytest: Mocking random
Pytest: Flask echo
Pytest: testing Flask echo
PyTest: Run tests in parallel with xdist
PyTest: Order of tests
PyTest: Randomize Order of tests
PyTest: Force default order
PyTest: no random order
Anagram on the command line
PyTest testing CLI
PyTest test discovery
PyTest test discovery - ignore some tests
PyTest select tests by name
PyTest select tests by marker
PyTest: Test Coverage
Exercise: module
Exercise: Open Source
Pytest resources
Pytest and tempdir
PyTest compare short lists - output
PyTest with parameter
PyTest with parameters
Pytest reporting in JUnit XML format
No test selected

Advancted functions

Variable scopes
Name resolution order (LEGB)
Scoping: global seen from fuction
Assignment creates local scope
Local scope gone wrong
Changing global variable from a function
Global variables mutable in functions
Scoping issues
sub in sub
Scoping sub in sub (enclosing scope)
Function objects
Functions are created at run time
Mutable default
Use None as default parameter
Inner function created every time the outer function runs
Static variable
Static variable in generated function
Inspect

Variable number of function arguments

Python function arguments - a reminder
Functions with unknown number of argumerns
Variable length argument list with * and **
Passing arguments as they were received (but incorrectly)
Unpacking args before passing them on
Exercise: implement the my_sum function
Solution: implement the my_sum function
Exercise: implement the reduce function
Soluton: implement the reduce function
Exercise: sort pairs
Solution: sort pairs

Python Packages

Why Create package
Create package
Internal usage
use module in package - relative path
use package (does not work)
package importing (and exporting) module
use package (module) with import
use package with import
Creating an installable Python package
Create tar.gz file
Install Package
Dependencies
Add README file
Add README file (setup.py)
Include executables
Add tests
Add tests calc
Add tests all
setup.py
Run tests and create package
Packaging applications (creating executable binaries)
Using PyInstaller
Other PyInstaller examples
Other
Py2app for Mac
Exercise: package
Exercise: create executable

Ctypes

ctypes - hello
concat
links

Advanced OOP

Class count instances
Class Attributes
Class Attributes in Instances
Attributes with method access
Instance Attribute
Methods are class attributes
Monkey patching
Classes: instance method
Class methods and class attributes
Classes: constructor
Class methods - alternative constructor
Abstract Base Class
Abstract Base Class with abc
ABC working example
ABC - cannot instantiate the base-class
ABC - must implement methods
Use Python @propery to fix bad interface (the bad interface)
Use Python @propery to fix bad interface (first attempt)
Use Python @propery to fix bad API
Use Python @propery decorator to fix bad API
Use Python @propery for value validation
class and static methods
Destructor: del
Destructor delayed
Destructor delayed for both
Opearator overloading
Operator overloading methods
Exercise: rectangular
Exercise: SNMP numbers
Exercise: Implement a Gene inheritance model combining DNA
Exercise: imaginary numbers - complex numbers
Solution: Rectangular
Solution: Implement a Gene inheritance model combining DNA
Instance counter

2to3

Convertig from Python 2 to Python 3
division
print in Python 2
print in Python 3
input and raw_input
Code that works on both 2 and 3
Compare different types
Octal numbers
2to3 Resources

Design Patterns

What are Design Patterns?
Don’t replace built-in objects
Facade - simple interface to complex system
Monkey Patching
Creation DPs “Just One”
Singleton
Monostate (Borg)
Dispatch table

Parallel

Types of Problems
Types of solutions
How many parallels to use?
Dividing jobs
Performance Monitoring

Threads

Python Threading docs
Threaded counters
Simple threaded counters
Simple threaded counters (parameterized)
Pass parameters to threads - Counter with attributes
Create a central counter
Lock - acquire - release
Counter - plain
GIL - Global Interpreter Lock
Thread load
Exercise: thread files
Exercise: thread URL requests.
Exercise: thread queue
Solution: thread queue
Solution: thread URL requests.

Forking

Fork
Forking
Fork skeleton
Fork with load
Fork load results
Marshalling / Serialization
Fork with random
Exercise: fork return data
Solution: fork return data

Asyncronus programming with AsyncIO

Sync chores
Async chores
Explanation
Coroutines
More about asyncio
Async files

Asynchronus programming with Twisted

About Twisted
Echo
Echo with log
Simple web client
Web client

Multiprocess

Multiprocess CPU count
Multiprocess Process
Multiprocess N files: Pool
Multiprocess load
Multiprocess: Pool
Multiprocess load async
Multiprocess and logging
Exercise: Process N files in parallel
Exercise: Process N Excel files in parallel
Exercise: Fetch URLs in parallel
Exercise: Fetch URLs from one site.
Solution: Fetch URLs in parallel

Multitasking

What is Multitasking?
Multitasking example
Multitasking example with wait
Multitaksing - second loop waits for first one
Multitasking counter
Multitasking counter with thread locking

Improving Performance - Optimizing code

Problems
Optimization strategy
Locate the source of the problem
Optimizing tactics
DSU: Decorate Sort Undecorate
Profile code
Slow example
profile slow code
cProfile slow code
Benchmarking
Benchmarking subs
Levenshtein distance
Generate words
Levenshtein - pylev
Levenshtein - editdistance
Editdistance benchmark
A Tool to Generate text files
Count characters
Memory leak
Garbage collection
Weak reference
Exercise: benchmark list-comprehension, map, for
Exercise: Benchmark Levenshtein
Exercise: sort files
Exercise: compare split words:
Exercise: count words

GUI with Python/Tk

Sample Tk app
GUI Toolkits
Installation
Python Tk Documentation
Python Tk Button
Python Tk Button with action
Python Tk Label
Python Tk Label - font size and color
Python Tk Keybinding
Python Tk Entry (one-line text entry)
Python Tk Entry for passwords and other secrets (hidden text)
Python Tk Checkbox
Python Tk Radiobutton
Python Tk Listbox
Python Tk Listbox Multiple
Python Tk Menubar
Python Tk Text
Python Tk Dialogs
Python Tk Filedialog
Python Tk messagebox
Python Tk Combobox
Python Tk OptionMenu
Python Tk Scale
Python Tk Progressbar
Python Tk Frame
Not so Simple Tk app with class
Tk: Hello World
Tk: Quit button
Tk: File selector
Tk: Checkbox
Tk: Runner
Tk: Runner with threads
Getting started with Tk
Exercise: Tk - Calculator one line
Exercise: Tk Shopping list
Exercise: Tk TODO list
Exercise: Tk Notepad
Exercise: Tk Copy files
Exercise: Tk Implement Master Mind board
Exercise: Tk
Solution: Tk - Calculator one line
Solution: Tk Implement Master Mind board
Solution: Tk
Solution: Tk Notepad
Simple file dialog

Python Pitfalls

Reuse of existing module name
Use the same name more than once
Compare string and number
Compare different types
Sort mixed data

Linters

Static Code Analyzis - Linters
PEP8
F811 - redefinition of unused
Warn when Redefining functions

Python .NET

IronPython
Use .NET libraries from Python
Python and .NET console
Python and .NET examples
Exercise Python and .NET

Python and Java

Jython
Calling Java from Python

Jython - Python running on the JVM

Jython Installation
Jython Installation
Jython load Java class
Jython load Java class in code
Jython test Java class

PIL - Pillow

Install Pillow
Create First Image
Write Text on Image
Select font for Text on Image
Font directories
Get size of an Image
Get size of text
Resize an existing Image
Crop an existing Image
Combine two images
Rotated text
Rotated text in top-right corner
Embed image (put one image on another one)
Draw a triangle
Draw a triangle and write text in it
Draw a triangle and write rotated text in it
Draw a rectangular
Draw a rectangle
Draw circle
Draw heart
Rectangle with rounded corners
TODO

FAQ

How not to name example scirpts?
Platform independent code
How to profile a python code to find causes of slowness?
pdb = Python Debugger
Avoid Redefining functions

Appendix

print_function
Dividers (no break or continue)
Lambdas
Abstract Class
Remove file
Modules: more
import hooks
Python resources
Progress bar
from future
Variable scope
scope
type
Look deeper in a list
Exercise: iterators - count
Simple function (before generators)

marks single line comments.

pydoc
How can I check if a string can be converted to a number?
Spyder Intro
Interactive Debugging
Parameter passing
Command line arguments and main
Infinite loop
break
continue
While with many conditions
while loop with many conditions
Format with conversion (stringifiation with str or repr)
Name of the current function in Python
Name of the caller function in Python
Stack trace in Python using inspect
Module Fibonacci
PyTest - assertion
PyTest - failure
PyTest - list
SAX with coroutine
Getting the class name of an object
Inheritance - super
Inheritance - super - other class
iterator - pairwise
iterator - grouped
itertools - groupby
Circular references
Context managers: with (file) experiments
itertools - izip
mixing iterators
mixing iterators
itertools - pairwise
itertools - grouped
range vs xrange in Python
profile (with hotshot) slow code
Abstract Base Class without abc
Abstract Base Class with abc Python 2 ?
Abstract Base Class with metaclass
Create class with metaclass
Python Descriptors
alter iterator
Create a counter queue
A Queue of tasks
Filtered Fibonacci with ifilter
Python from .NET

About

Share this book

Feedback

Price

You pay

Author earns

...Or Buy With Credits!

Number of credits (Minimum 1)

Author

Contents

Fixtures and Mocking in Python

First steps

marks single line comments.

Second steps

Numbers

Comparison and Boolean

Strings

Loops

Formatted printing

Lists

Files

Dictionary (hash)

Sets

Functions (subroutines)

Modules

Regular Expressions

PyCharm

Python standard modules

JSON

Command line arguments with argparse

Exception handling

Classes - OOP - Object Oriented Programming

PyPi - Python Package Index

SQLite Database Access

MySQL

PostgreSQL

SQLAlchemy

NoSQL

MongoDB

Redis

Web client

Python Web server

Python Flask

Networking

Interactive shell

Testing Demo

Types in Python

Testing Intro

Functional programming

Iterators - with and without Itertools

Generators and Generator Expressions

Logging

Closures

Decorators

Context managers (with statement)

Advanced lists

Advanced Exception handling

Warnings

CSV

Excel

XML

SciPy - for Scientific Computing in Python

Python and Biology

Chemistry

numpy

Pandas

Matplotlib

Seaborn

Jupyter notebooks

Testing

Testing with unittest

Testing with PyTest

Advancted functions

Variable number of function arguments

Python Packages

Ctypes

Advanced OOP

2to3

Design Patterns

Parallel