{ "cells": [ { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "# APS4000 Introduction to Honours Computing\n", "### Introduction to Python\n", "\n", "## Basics\n", "\n", "### Introduction\n", "\n", "Python is a object-oriented interpreted script language. There is also a recent nature\n", "article at http://www.nature.com/news/programming-pick-up-python-1.16833. For\n", "documentation and tutorial, I generally recommend the original online python documentation at https://docs.python.org/3/. Of course, you can always “google” for specific\n", "recipes. Python allows you to work interactively, write routines you can call, but also to\n", "write standalone applications. There is modules for almost anything. Python is used for\n", "applications like running web servers to super computing. It also allows to interface to\n", "other languages, e.g., FORTRAN (f2py), C/C++, R, ... You can use python for scripted\n", "text processing, data analysis (numpy), plotting/visualization (matplotlib). The later two\n", "packages we will use in the second part of this course. Here we will first focus on a basic\n", "introduction to python3. This introduction is not comprising, it just is meant to give you\n", "an idea of the power of python, what you all might be able to do, but may have to look\n", "up later. It will not replace you actually reading the manual and tutorial, which I highly\n", "recommend.\n", "\n", "Python is object-oriented – in Python everything is an object. Everything. [Object oriented programming](http://en.wikipedia.org/wiki/Object-oriented_programming)\n", "combines data and code, allowing you data encapsulation and includes inheritance and\n", "polymorphism.\n", "\n", "In this crash course - and this will crash any person’s ability to absorb it all in just 3h - I\n", "will give an overview so that you have seen what you may be able to all use in python, to\n", "inspire you, to model you research projects and ideas in python, and find the right data\n", "structures and organization for it. So that later you may remember having seen things\n", "that you could use. I do not expect that you remember all by heart. I need to look up\n", "and try out things continuously myself.\n" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### Starting Python\n", "In this course we want to use Python 3. At the time of this writing, the current version\n", "is Python 3.5.1. For simpler editing, we use IPython. The current version is 4.1.1.\n", "In the following the shell prompt (Bash) is displayed as\n", "```shell\n", "~>\n", "```\n", "To start IPython we use\n", "```shell\n", "~> ipython\n", "```\n", "you should then see a message like\n", "```python\n", "Python 3.5.1 (default, Dec 29 2015, 23:29:32)\n", "Type \"copyright\", \"credits\" or \"license\" for more information.\n", "IPython 4.1.1 -- An enhanced Interactive Python.\n", "? -> Introduction and overview of IPython’s features.\n", "%quickref -> Quick reference.\n", "help -> Python’s own help system.\n", "object? -> Details about ’object’, use ’object??’ for extra details.\n", "\n", "In [1]:\n", "```\n", "On some systems you may have to type\n", "```shell\n", "~> ipython3\n", "```\n", "Now you are ready to use python. There is also more fancy shells, e.g., the IPython\n", "notebook, which you would start using\n", "```shell\n", "~> ipython notebook\n", "```\n", "This should pop up a new tab in your web browser button on the top right.[1](#install_notes) The you press the New Notebook\n", "\n", "Very useful later in this course - and indispensable for you later when developing python\n", "codes - will be to automatically reload modules when you modify them.\n", "\n", "You can do the manually, every time you start IPython, by first loading the autoreload\n", "extension, then activating it to reload all loaded modules automatically when changed.\n", "```python\n", "In [2]: %load_ext autoreload\n", "In [3]: %autoreload 2\n", "```\n", "Instructions on how to set this up by default so it is done automatically every time\n", "you start IPython can be found at, e.g., https://www.reddit.com/r/Python/comments/rsfsi/tutorial_spend_30_seconds_setting_up/.\n", "\n", "##### Executing commands\n", "On the IPython / Python shell you type `` to execute commands. In the IPython notebook you have to type `+` to execute a cell; just `` start a new line, allowing you to execute a block of command at once. For the purpuse of repruducibility in other Python shells, I do not use that here usually.\n", "\n", "---\n", "1: For me this required installation of `pyzmq` and `jinja2`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%load_ext autoreload" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%autoreload 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Basic Data Types\n", "#### Scalar data types\n", "##### Integer\n", "Just the number. Can be arbitrary large. Must not contain \"`.`\". Example" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "2" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "2000000000000000000000000000000000000000000000000000000000000000" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Integer constants can also be specified in other bases, e.g.," ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "0x32" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "0o32" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "0b101" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Floating Point\n", "Floating point number with finite precision. Use \"`.`\" to separate fraction part. Use Just\n", "the number. Use \"`e`\" to separate exponent. Example:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "2." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "-2.e23" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "2.0001e23" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "2.0000000000000000000000000000000001e23" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Precision is system-dependent, typical is [IEEE-754](http://en.wikipedia.org/wiki/IEEE_floating_point) 8-byte binary floating point (15 digits precision, exponent $\\pm300$. Classes for (internal) decimal representation exists as well." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Complex Numbers\n", "Use \"`j`\" for imaginary part." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "3 -4j" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "##### Operators" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The usual, `+`, `-`, `*`, `/`, `%` (modulo), `**` (power). Special: `//` (integer division). Combining integer with float will result in float. Examples:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "7 // 3" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "7 / 3" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "7. // 3" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "10**400 + 1." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There is also bit-wise binary operations on integers using `&` (and), `|` (or), `^` (xor), `~` (not), `<<` (left shift), `>>` (right shift):" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "7 & 3" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "7 | 8" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "7 ^ 3" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "~3" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "3 << 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Truth and Logical Operations\n", "Define logical values `True` and `False`. Logical operations include `or`, `not`, and `and`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "True or False" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "True and False" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "not True" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Comparison Operators\n", "These include `<`, `>`, `<=`, `==` (equality, in contrast to assignment), `!=` (not equal), `is`\n", "(object identity), and `is not` (negated object identity)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "(1 > 3) or (3 == 4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Nothing\n", "... and there is the None object - we will use later." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "None" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`...` and the Ellipsis object" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "Ellipsis" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Strings\n", "Sequence of characters enclosed by matching single or double quotation marks. Multi-line\n", "strings can be defined using triple quotation marks." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'abc 123'" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "\"\"\"abc\n", "123\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "On a regular (I)Python(3) prompt you would have seen a line starting with `....:` for continuation:\n", "```python\n", "In [30]: \"\"\"abc\n", "....: 123\"\"\"\n", "```\n", "\n", "Here, the special characters like `new line` (`\\n`) start with a `\\`. You can add them manually:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'abc\\n123'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To input special characters in to a string w/o interpretation, use a *raw string* with has\n", "an `r` in front of the string:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "r'abc\\n123'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here the backslash itself is represented by a double backslash. To see the difference, we\n", "can use the `print` function" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "print('abc\\n123')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "print(r'abc\\n123')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also add strings or replicate strings" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'abc' + '123'" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'12' * 12" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There is a whole variety of other string methods to be discussed later or in the man-\n", "ual. Note that in Python 3 there is also a string-like data types `bytes`, `bytearray`, and\n", "`memoryview` but these are for more advanced use cases - and to keep competent python\n", "programmers employed." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "bytearray(12)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "b'1234'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Indexing and Slicing\n", "Slicing and indexing generally can be applied to all ordered data that can be indexed. It\n", "is done with the \"`[_]`\" operator, where \"`_`\" stands for an argument.\n", "\n", "For strings, you can get individual characters (index is base-0)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'abc123'[2]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "or sub-strings (called \"*slicing*\" - **last index excluded!**). The basic syntax is \"`start:stop[:step]`\".\n", "Default step size is 1 and omitted start/stop values run to the end of the structure (string).\n", "Negative values count from the back, with -1 referring to the last element." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'abc123'[2:4]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'abc123'[2::2]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'abc123'[::-2]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'abc123'[:-1]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'abc123'[-2:0:-1]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is the default behaviour of slicing, but in principle, each object can define how\n", "it wants to react to this, so can objects (classes) you define by defining the necessary\n", "attributes. More later." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Organizing Data - Variables\n", "You can also assign values to variables." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = 12" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "print(a)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Variable can have letters, undershorts and numbers but must not start with a number.\n", "Variable names starting with one or two underscores usually have special meaning. Vari-\n", "able names are case sensitive.\n", "\n", "**Note:** *A variable is a name (pointer to) that object. Assignment to an existing\n", "variable does change where it points to not the object that it points to.*" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = 12" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "b = a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = 13" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "b" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Variables do not need to be defined and given a type in advance; the type comes with the\n", "object it points to." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = 12" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a = 'abc'" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In-place assignment operators are short hand for written-out expression 2 This way you\n", "can apparently add even to non-mutable objects like strings. Operators comprise the\n", "usual suspects, `+=`, `-=`, `*=`, `/=`, `%=`, `**=`, `&=`, `|=`, `^=`, ..." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "i = 3" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "i += 4." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "i" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "s = 'abc'" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "s += 'd'" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "s" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the string case, \"s\" is instead now pointing to a new string object. Later we will see\n", "for numpy that mutable objects can modify this behavior." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Organizing Data - Containers\n", "#### Lists\n", "List of elements in square brackets" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "[1, 2, 3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "[1, 2, 3] + [4, 5, 6]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "[1,2,3] * 4" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = [1, 2, 3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a += [4]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a += [4, 5]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "List elements as assigmnet targets" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = [1, 2, 3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a[1] = 4" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a[1] = 'c'" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a[0] = a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a[0][0][0][1][0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That is, list entries can be any kind of object, even itself, and lists are *mutable*. In\n", "contrast, strings are not mutable." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "s = '123'" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "s[2] = 'a'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can even replace ranges by ranges" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = [1, 2, 3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a[0:2] = [4, 5, 6]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "But note that assignment to elements is different" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = [1, 2, 3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a[1] = [1, 2, 3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and that ranges cannot be replaced by scalars" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a[1:2] = 1" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a[1:2] = [1]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There is also specific list functions and methods, e.g., `append`, `len`, `index`, `count`, `min`, `max`, `copy`, `insert`, `clear`, `remove`, `pop`, `reverse`, `sort`, ... and the `sorted` function" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = [1, 3, 2]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a.sort()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the example above, the \"()\" stand for a function call, here w/o any parameter." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "print(a)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "sorted([3, 5, 2, 4])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Empty lists:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "[]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "list()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can test whether an element is in the list" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "2 in [1, 2, 3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "2 not in [1, 2, 3]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Deleting elements" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = [1, 2, 3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "del a[1]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Tuples\n", "Like lists, but not mutable. Generated by the comma operator, enclosed by bracket if\n", "ambiguous otherwise." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a = (1, 'a', [1])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a[2]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Empty and 1-element tuples:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = ()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = (1,)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Except operations that *change* the tuple, most list operations work for tuples as well." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "(1,) * 4" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "1 in (1, 2, 3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Dictionaries\n", "Dictionaries are very efficient to organize data that cannot be indexes easily. It consists\n", "of a pair of key and index; the key has to be a “hashable” (i.e., usually non-mutable)\n", "object like a number, string, or Tuple. Truth values, `None`, and `Ellipsis` are fine as well.)\n", "Dictionaries are nor ordered or sorted. Create with square brackets or 'dict' function." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a = {'a': 3, True: 4, 5: 7, (1,2): 4}" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "b = dict(a=7, c=11)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "b" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that in the last example the keywords are converted to strings.\n", "\n", "You can test whether a key is present" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'c' in b" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can combine dictionaries using `update`" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a.update(b)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and access elements using the indexing syntax or the get *method*" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a[True]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a.get(True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Get a default value if key is not defined" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a.get(False, 0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "remove elements (element returned)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a.pop(True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "set default values for undefined entries" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a.setdefault(False, 0)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "New values can be easily added, by assignment; if the key does not yet exists it is added,\n", "if it dies exist, it is overwritten." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = {}" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a['a'] = 1" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a['b'] = 2" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a['a'] = 3" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Determine number of elements" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "len(a)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Deleting elements" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a = dict(a=2,b=3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "del a['b']" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are variation classes like `OrderedDict`, etc., as well. Have a look at the [Python Standard Library](https://docs.python.org/3/library/)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Sets\n", "Sort of like dictionaries, but only keys, no values. Provides many useful set operations." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "{1, 2, 3}" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "{1, 2, 3} | {3, 4, 5}" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "{1, 'a', (1, 2)}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Define empty set" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "set()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Set operations include `<=` (subset), `<` (proper subset), `>=` (superset), `>` (proper superset), `|` (union), `&` (intersection), `−` (difference),`ˆ`(symmetric difference), ...\n", "There is also a *hashable* (non-mutable) set variation, `frozenset`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Converting Between Types\n", "Dictionary to list" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "b = dict(a = 7, c = 11)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "list(b)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "list(b.items())" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "list(b.values())" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "list(b.keys())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Conversion between `list`, `set`, `tuple` are trivial" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "list(tuple(set([1, 2, 3, 2, 1])))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Object Identity\n", "You can see whether objects are identical using the `is` operator. Or `==` on the object ID,\n", "which you can get with the `id` function." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x = [1,2,3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "y = x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x is y" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "id(x) == id(y)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "id(x)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "y = x + [4]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x is y" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "y" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "y = x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "y += [4]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x is y" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x = (1,2,3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "id(x)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x += (4,)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "id(x)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "i = 1." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "id(i)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "i += 1" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "id(i)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So, we see that the inplace operator behave differently for mutable objects than for im-\n", "mutable objects. This is special behaviour of the *mutable* objects! Recall that strings and\n", "numbers are also immutable objects." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Modules and Code Organization\n", "Besides interactive use, code can be organized in modules - python source files with exten-\n", "sion \"`.py`\". Many of the functions of the standard library also \"live\" in modules. Modules\n", "are imported using the \"import\" statement. As an example, let’s look at mathematical\n", "functions:\n", "\n", "#### Mathematical Functions\n", "These sit in the module `math`. Functions generally are called with round brackets. To use\n", "them, we first need to import this module" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import math" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "math.sin(12)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "math.factorial(12)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Modules are objects.* In the above example, the \"`.`\" is used to access the model's `cos`\n", "function.\n", "\n", "We can also import names for direct use" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from math import cos" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "cos(12)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "or even be lazy" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "from math import atan2 as a\n", "a(2,3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What is all in the math module?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "? math" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "Type: module\n", "String form: \n", "File: ~/Python/lib/python3.5/lib-dynload/math.cpython-35m-x86_64-linux-gnu.so\n", "Docstring:\n", "This module is always available. It provides access to the\n", "mathematical functions defined by the C standard.\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "dir(math)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [], "source": [ "math.__dict__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Making your Own Module\n", "Use your favourite editor to edit the file `test.py`.\n", "\n", "As a measure of style, we start the module - any python object - with a \"doc string\":\n", "On first line with a brief description, then a blank line, then the extended description on\n", "many lines and paragraphs as needed. Our code might be\n", "\n", "```python\n", "\"\"\"\n", "Module for python tests.\n", "This Model cotains a selection of my python and learing codes.\n", "will change a lot over time.\n", "\"\"\"\n", "import math\n", "\n", "a = math.cos(12)\n", "```\n", "\n", "and we can use this in IPython:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test1" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test1.a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test1.__doc__" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "? test1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "Type: module\n", "String form: \n", "File: ~/Class/ASP4000-python-2016/test1.py\n", "Docstring:\n", "Module for python tests.\n", "This Model cotains a selection of my python and learing codes.\n", "will change a lot over time.\n", "```\n", "\n", "You can also add comments. Everything after a `#` symbol is treated as comment (unless\n", "in string). They can be at the end of a line, or taking up the entire line. Usually should\n", "go with the indentation of their scope\n", "\n", "```python\n", "# I am a comment\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Function is Form\n", "In python code blocks are started by a colon at the end of a statement, then indentation\n", "(four white spaces) is used. The code block ends when de-indented. Lines can be continuedwith a backslash at the end of the line, or when a bracket (round, square, curl) encloses\n", "an expression.\n", "\n", "Here, let’s define out first function, using the def statement that we save in `test2.py`\n", "\n", "```python\n", "def f(x):\n", " y = x + 3\n", " return 2*y\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test2" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test2.f(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also return more than one value. If there is no return statement, the default\n", "return value is `None`. I will save this in `test3.py`\n", "\n", "```python\n", "def f(x):\n", " \"\"\"My test function\"\"\"\n", " y = x + 3\n", " return 2 * y, 3 * y\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test3" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test3.f(3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "a, b = test3.f(4)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "a" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "b" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and we can also access the function doc string" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "? test3.f" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "Signature: test3.f(x)\n", "Docstring: My test function\n", "File: ~/Class/ASP4000-python-2016/test3.py\n", "Type: function\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test3.f.__doc__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Assignment to formal parameters will not overwrite actual parameter, just define inside\n", "the function to what object the name now points there. Variable defined inside the\n", "function are local and not visible outside.\n", "\n", "```python\n", "def f(x):\n", " x = x + 3\n", " y = x + 4\n", " return y\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import test4" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x = 5" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test4.f(x)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test4.f.y" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "... but we can tell python to make a variable global (to the module) and variables not\n", "local (allow assignment to enclosing scope)\n", "\n", "```python\n", "def f(x):\n", " global y\n", " z = 0\n", " def g(y):\n", " nonlocal z\n", " z = 4 * x + y\n", " return z + 1\n", " y = g(2 * x) + z\n", " return y + 1\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test5" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test5.f(2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test5.y" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In places where python expects code but you don’t want to do anything, you can use the pass statement. Can also be used for prototyping.\n", "```python\n", "def f(x):\n", " pass\n", "```\n", "Functions do not have to return anything, they can just perform a task, .e.g,\n", "```python\n", "def f(x):\n", " print('The value is:', x)\n", "```\n", "That is, they can act just like a subroutine in FORTRAN. As stated before, if there is not return value, the return None by default. If the return value is not used for anything, it is just ignored in codes (on the console it would be printed), even if the function does return a value." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Functions are Objects Too" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "f = test5.f" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "f(2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import math" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "c = math.cos" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "c(4 * math.pi)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x = [test5.f, test5.f(2)]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x[0](3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "... and we can pass them as arguments\n", "```\n", "def f(g, x):\n", " return 2 * g(x)\n", "```\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test6" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test6.f(math.cos, 0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "... or return them ...\n", "```python\n", "def f(n):\n", " def g(x):\n", " return x**n\n", " return g\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test7" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "h = test7.f(3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "h" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "h(2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note the (``) above indicating a \"closure\" included - the function encapsulate the\n", "environment in which it was defined.\n", "\n", "Instead of having to reload test by hand all the time, we can instruct IPython to do this\n", "automatically for us whenever the code has changed:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%aimport test" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "But for the purpose of this course I had to create new files for the purpose of documentation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### IPython Notebook\n", "The IPython Notebook and (I)Python also allow you to define functions, etc., interactively, however, some things like modules do not work. For most cases I do not recooment this work mode. Here and exampe, nevertheless:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def f(x):\n", " x = x**3\n", " return x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "f(5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Anonymous “Lambda” Functions\n", "You can also define *anonymous* functions" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "lambda x,y: x*y" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "f = _" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "f(2,3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The second input also shows you how to capture previous output. You can also use the\n", "`%hist` magic IPython function to list previous input. Or you can use `Out[#]` where `#` is the number of the output you refer to.\n", "\n", "These anonymous functions are particularly useful if a function needs to be passed as\n", "argument:\n", "```python\n", "def f(g):\n", " def h(x):\n", " return g(g(x))\n", " return h\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test8" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "h = test8.f(lambda x: x*2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "h(3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "h('abc')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Function Arguments\n", "... can have default values, and can be passed by keyword, or just by position. In either\n", "case, all positional arguments need to precede keyword arguments. Keyword arguments\n", "can be in arbitrary order, but the must not be a conflict with positional arguments when\n", "calling the function.\n", "```python\n", "def f(x, y = 2):\n", " return x**y\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test9" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test9.f(2, 3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test9.f(y=2, x=3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test9.f(2, y=3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can collect extra/remaining positional and keyword arguments in a tuple or dictio-\n", "nary using \"\\*args\" and/or \"\\*\\*kwargs\" after the last positional or keyword argument,\n", "respectively. The same syntax can be used to pass such parameters from lists/tuples or\n", "dictionaries to functions. First Python will match all positional arguments, then match\n", "the keywords.\n", "```python\n", "def f(x, *args, y = 2, **kwargs):\n", " return x, args, y, kwargs\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test10" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test10.f(1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test10.f(1,2,3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test10.f(1, z = 4)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "args = dict(a = 3, b = 4, y = 5)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "kwargs = dict(a = 3, b = 4, y = 5)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test10.f(1, 2, *args, z = 3, **kwargs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Note:** Keyword parameter defaults are evaluated at the time of function definition\n", "only." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Flow Control Structures\n", "So far we can only do rather boring things. We want to control program flow.\n", "#### If Statement\n", "The full statement includes a required leading if, followed by arbitrary many elif and\n", "an optional final else statement catching the remaining cases.\n", "```python\n", "def f(x):\n", " if x < 0:\n", " return -2 * x\n", " elif x == 0:\n", " return 0\n", " elif (x < 1) or (x > 5):\n", " x = x**0.5\n", " return x\n", " else:\n", " return 1\n", "```\n", "There is no \"case\" statement as in many other languages. But there is an “inline” version\n", "of the if statement:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "print('a') if 3 > 4 else print('b')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### While Statement\n", "This will continue a loop as long as a given condition is fulfilled. An `lse` clause is\n", "executed if the condition test `False`, even if this is just the first time. break allows to\n", "terminate the loop, not executing the else clause, and a continue statement skips the\n", "rest and immediately continues with the next iteration test.\n", "```python\n", "def f(x):\n", " y = 0\n", " while x > 5:\n", " x -= 1\n", " y += x\n", " if x > 10:\n", " break\n", " if x == 7:\n", " continue\n", " else:\n", " return y\n", " return -1\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### For Statement\n", "This allows you to have a set number of iterations or iterate over a member of a container,\n", "etc. Similar use of `else`, `continue`, and `break`. Iterated item is assigned to a variable.\n", "```python\n", "def f(x, n = 7):\n", " y = 0\n", " for i in range(1, n):\n", " y += x**i\n", " return y\n", "```\n", "and it can come from a list, dictionary, ...\n", "```python\n", "def f(x):\n", " y = 0\n", " for i in [2, 4, 7]:\n", " y += x**i\n", " return y\n", "```\n", "or using enumerate to return 2-tuples of the item and its index\n", "```python\n", "def f(x):\n", " d = dict()\n", " for i,z in enumerate(x):\n", " d[z] = i\n", " return d\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test15" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test15.f('abc')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Another key example is the `zip` function\n", "```python\n", "def f(x):\n", " d = dict()\n", " for i,z in enumerate(x):\n", " d[z] = i\n", " return d\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test16" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test16.f('abc', '1234')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A much larger selection can be found in the `itertools` module." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "#### Mutants - Modifying Function Arguments\n", "At this point I would like to note that that while you can’t change what the formal\n", "parameter (from the function call) points to, the object itself that it points to, if mutable,\n", "can be changed:\n", "```python\n", "def f(x):\n", " x = x + [1]\n", " return x\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test17" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x = [1,2,3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test17.f(x)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**but**\n", "```python\n", "def f(x):\n", " x += [1]\n", "```\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test18" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ " x = [1, 2, 3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test18.f(x)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Unless absolutely intended, this can be hard to debug and should be avoided if you can. A very popular programming style (and languages designed around it) is called **functional programming**: Functions only return values but do not modify their arguments. It is already somewhat hard to violate this in python because you can’t pass parameters by reference (as in C or FORTRAN), but in some cases you sort of can mimic this if you really want. Be careful about this when modifying mutable containers that were passed as arguments to functions!\n", "\n", "As said, except in cases where you really know what you do and it is the most efficient way\n", "to you task and well documented: Avoid it. Especially for the API and user interfaces to\n", "your code. It is much better to use function return values or only modify objects if the\n", "API explicitly calls for and indicates this.\n", "\n", "This can be particularly troublesome for keyword parameters with defaults, as mentioned\n", "above:\n", "```python\n", "def f(x = []):\n", " x += [1]\n", " return x\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test19" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test19.f()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test19.f()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test19.f()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ooops ..." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Iterators and Comprehensions\n", "A key design of Python 3 is *lazy* data evaluation. Items will be produced as needed, and the same mechanism can be used to iterate over items. The range object is such an example. You can define you own iterators or iterator functions using the yield statement.\n", "\n", "Very useful are comprehensions, sort of the and inline version of the `for` loop." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "[i*2 for i in range(10) if i % 2 == 0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and can be nested ..." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "[i*j for i in range(5) for j in range(5) if i % 2 == 0 if j % 3 == 0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and for a rainy day, there is generator objects from comprehensions as well ..." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "(2*i for i in range(5))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x = (2*i for i in range(5))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "list(x)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "list(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Iterators can be exhausted. \n", "\n", "In practice, the `next` function return the next item form an iterator and raises and an `StopIteration` exceptions when there is no more items." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Advanced Python\n", "### Classes - The Heart of Python\n", "In python every object also has a *type*. The type of an object is its class. You can use the\n", "`type` function to find out about the type of an object." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "type((1,))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can define you own class by deriving from an exiting class - which can be one of your own classes -, hence *inheriting* its methods and attributes, but add more features and specialization. If you start a new class hierarchy from scratch, you usually would start by inheriting from object. But you can also inherit from multiple classes, merging their features.\n", "```python\n", "class MyClass(object):\n", " \"\"\" My test class\"\"\"\n", " pass\n", "You can now create an object of this type, i.e., an *instance* of this new class by \"calling\"\n", "it." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test20" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "o = test20.MyClass()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "o" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is not yet very exciting. The most important first step is to initialize the object.\n", "This is done by the `__init__` method, to which the arguments of the object creation are\n", "passes. Additionally, a first argument `self` is passed, which is a reference to the current\n", "instance of this class and can be used to access the class’s attributes and methods. Note\n", "that you can read the classes attributes but by default do not overwrite them using self.\n", "The `__init__` method must not return a value.\n", "```python\n", "class MyClass(object):\n", " \"\"\" My test class\"\"\"\n", " def __init__(self, x):\n", " \"\"\"initialize my object\"\"\"\n", " self.x = x\n", " self.y = x + 1\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test21" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "o = test21.MyClass(2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "o.x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "o.y" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You could also set things by hand on your class ..." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "o.z = 4" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "o.__dict__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "... but this would be much more painful if you have to do it by hand everywhere this\n", "object is used, especially for more involved setup cases.\n", "\n", "You can also define constants and do computations in the class body, and of course define\n", "any number of your own functions. You can even define function names by assignment,\n", "e.g., \"`__iadd__ = __add__`\". You can define how you objects reacts to operands, how it is\n", "printed, what its length is, how it reacts to indexing (`[]`) or being called (`()` - `call`).\n", "\n", "As an example, let’s have a class that stores temperature:\n", "```python\n", "class Temperature(object):\n", " unit = 'K'\n", " def __init__(self, T = 0):\n", " self.set(T)\n", " def set(self, T):\n", " if isinstance(T, Temperature):\n", " T = T._TK\n", " assert T >= 0\n", " self._TK = T # temperature in Kelvin\n", " def get(self):\n", " return self._TK\n", " def e(self):\n", " return 7.5657e-15 * self._TK**4\n", " def absolute(self):\n", " return self._TK\n", " def __str__(self):\n", " return '{} {}'.format(self.get(), self.unit)\n", " __repr__ = __str__\n", "\n", "class Celsius(Temperature):\n", " unit = 'C'\n", " offset = 273.15\n", " def set(self, T):\n", " super().set(T + self.offset)\n", " def get(self):\n", " return super().get() - self.offset\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test22" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "t = test22.Temperature(3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t.e()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t = test22.Celsius(3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t.absolute()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Exercise:**\n", "\n", "1. Add a class that deals with temperature in Kelvin.\n", "2. Add function that computes the energy flux for black body radiation.\n", "3. Add a functionality so two temperature objects can be added. (Consider:\n", " what happens if you add two different object types? What should the\n", " resulting object type be?)\n", " \n", "An example of a class of which instances behave like a function:\n", "```python\n", "class F(object):\n", " def __init__(self, n):\n", " self.n = n\n", " def __call__(self, x):\n", " return x**self.n\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test23" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "f = test23.F(3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "f(5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also have \"computed\" properties of you object defining the `__getattr__` method - but this is very advanced python.\n", "```python\n", "class X(object):\n", " def __init__(self, x, v):\n", " assert isinstance(x, str)\n", " self._x = x\n", " self._v = v\n", " def __getattr__(self, x):\n", " if x == self._x:\n", " return self._v\n", " raise AttributeError()\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test24" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x = test24.X('s', 3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x.s" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Naming and Formatting Conventions\n", "* use doc strings\n", "* CamelCase for classes\n", "* runinnames for functions and methods\n", "* names starting with underscore for object private methods and attributes\n", "* use white space before and after operators and after comma.\n", "* use 4 spaces for indentation; not less (or more) and no tabulators." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Making Executable Scripts\n", "There is four things that you need.\n", "1. A proper operating system (not Windows) - well, can be done there as well I suppose.\n", "2. Make the python file (script) executable. On Linux we would do this on the shell\n", " ```shell\n", " chmod u+x text.py\n", " ```\n", " Usually this is the last step\n", "3. Add a “shebang” (#!) at the beginning of the script. Typically I use\n", " ```python\n", " #! /usr/bin/env python3\n", " ```\n", " This tells it what interpreter to use to execute your script. What you need to have\n", " there may vary from system to system. Alternatively, you can call you script later\n", " using\n", " ```shell\n", " python3 test.py\n", " ```\n", " The file does not require to retain the extension .py, or you can make a symbolic\n", " link (this is what I tend to do).\n", "4. Execute specific script code if the module is called as main program. In this case\n", " the variable `__name__` contains the string \"`__main__`\", so you start your scrip code block\n", " using the line\n", " ```python\n", " if __name__ == \"__main__\":\n", " ```\n", " \n", "Other useful things are to access parameters passed to the script. These can be found in\n", "`sys.argv` - you need to import sys of course to use this. Note that `sys.argv[0]` is the\n", "program name. Note that the parameter are strings.\n", "```python\n", "#! /usr/bin/env python3\n", "\n", "\"\"\"my test script\"\"\"\n", "\n", "import sys\n", "\n", "a = 4\n", "\n", "def f(x):\n", " print(x * a)\n", "\n", "if __name__ == \"__main__\":\n", " f(sys.argv[1])\n", "```\n", "and on the shell\n", "```shell\n", "~/>./test25.py 34\n", "34343434\n", "```\n", "A very useful package for dealing with input parameters for scripts is `argparse`. Having\n", "this available is what made me switch from Python 2 to Python 3." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### When Things Fail - Exceptions\n", "Exceptions are a regular means of \"out of band\" communication in python. Some things\n", "are easiest done this way and some not really in any other way. *Use them.*\n", "\n", "The code block to be monitored is started with a `try` statement; exceptions are “caught”\n", "by one or several `except` clauses - generally or specialized for specific exceptions, the else\n", "clause is executed if nothing fails and the `finally` clause is execute in any case - failure or not.\n", "```python\n", "def f(x):\n", " y = 0\n", " try:\n", " for x in range(x, 3):\n", " y = 1 / x\n", " except ZeroDivisionError as e:\n", " print('Error:', e)\n", " except Exception as e:\n", " print('Unexpected error:', e)\n", " else:\n", " print('all went fine')\n", " finally:\n", " print(y)\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test26" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test26.f(2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test26.f(-2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test26.f(-2.5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also use just \"`except:`\" to catch all exceptions if you don’t care about the\n", "exception object itself or leave off the \"`as ...`\" part if don’t need the exception info.\n", "```python\n", "def f(x):\n", " y = 0\n", " try:\n", " for x in range(x, 3):\n", " y = 1 / x\n", " except ZeroDivisionError:\n", " print('Error!')\n", " except:\n", " print('Unexpected error!')\n", " else:\n", " print('all went fine')\n", " finally:\n", " print(y)\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Input and Output\n", "#### Format Strings\n", "Strings have a format method. It allows you to replace \"placeholders\" - in curly braces - by arguments. They can be matched by index (zero-based) or by keyword. A detailed description is found at https://docs.python.org/3.4/library/string.html. The layout of these placeholder is to first give what is to be formatted - either by number or by name, followed by a colon, then the format string. If no number or name is supplied, numbering is automatic. \n", "\n", "Examples:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'The Winner is {:} on {:}'.format('Mr. X', 'best movie')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'The Winner is {act:>10s} for A${prize:05d}'.format(prize=1000, act='Jim')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Another very useful string method is join to combine a list of strings." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "'/'.join(('home','alex','xxx'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### File I/O\n", "Very useful to consequently use the routines in os and os.path to manage paths. More\n", "recently the module pathlib was added, but I do not have experience with it and find it\n", "sort of clunky. You may also need some routines from the sys module.\n", "\n", "The open routine opens a file and returns a file object. To close the file, call it’s close\n", "method. Always close you files when done. Resources are finite. https://docs.python.org/3/library/functions.html#open\n", "\n", "The routine takes a file name and an open “mode”. Useful modes are t for \"text\", `b`\n", "for binary, `U` for \"universal new line\" mode, `r` for read, `w` for write, `a` for append; `x` for exclusive creation; and added `+` opens it for \"updating\" (may truncate).\n", "\n", "Use the write routine to write to a file, `read` to read (the entire) file, or read a single line using `readline`.\n", "```python\n", "def f():\n", " f = open('xxx.txt', 'wt')\n", " f.write('123\\n345\\n5677')\n", " f.close()\n", "```\n", "Text files can be iterated over - each iteration yields one line:\n", "```python\n", "def f():\n", " f = open('xxx.txt', 'rt')\n", " for i, l in enumerate(f):\n", " print('{:05d} {}'.format(i, l.strip()))\n", " f.close()\n", "```\n", "Here the `strip` method of the string gets rid of (lead/trail) which spaces and the trailing\n", "newline (`\\n`).\n", "\n", "We may deal with binary files later." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Resource Management - With Statement\n", "The with statement allows you to manage resources. For example, automatically close\n", "them, and deal with exceptions (even close them then).\n", "```python\n", "def f():\n", " with open('xxx.txt', 'rt') as f:\n", " for i, l in enumerate(f):\n", " print('{:05d} {}'.format(i, l.strip()))\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Decorators\n", "Start with the `@` symbol and are essentially functions that return modified “decorated”\n", "functions. You may see this sometimes.\n", "```python\n", "def mul5(f):\n", " def g(*args, **kwargs):\n", " print('function was decorated.')\n", " return f(*args, **kwargs) * 5\n", " return g\n", "\n", "@mul5\n", "def h(x):\n", " return x**2\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test31" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test31.h(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The key point is that the decorator takes a function and returns another function.\n", "\n", "The decorator itself can also be a function - can have parameters - that returns the actual\n", "wrapping function.\n", "```python\n", "def multiply(x):\n", " def wrap(f):\n", " def g(*args, **kwargs):\n", " print('function was decorated:', x)\n", " return f(*args, **kwargs) * x\n", " return g\n", " return wrap\n", "\n", "@multiply(5)\n", "def h(x):\n", " return x**2\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test32" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "test32.h(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Or you can design it as a class with a `__call__` method that takes parameters for its\n", "`__init__` method.\n", "```python\n", "class Multiply(object):\n", " def __init__(self, factor):\n", " self._factor = factor\n", " def __call__(self, f):\n", " def g(*args, **kwargs):\n", " print('function was decorated:', self._factor)\n", " return f(*args, **kwargs) * self._factor\n", " return g\n", "\n", "@Multiply(5)\n", "def h(x):\n", " return x**2\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Properties\n", "Allows you to provide an interface to internal data of you object. You can generally\n", "control object access using `__get__` and `__set__`.\n", "```python\n", "class Temperature(object):\n", " def __init__(self, T = 0):\n", " self.T = T\n", " @property\n", " def T(self):\n", " \"\"\"T in K\"\"\"\n", " return self._TK\n", " @T.setter\n", " def T(self, T):\n", " self._TK = T\n", " def e(self):\n", " \"\"\"energy density in cgs\"\"\"\n", " return 7.5657e-15 * self._TK**4\n", "\n", "class Celsius(Temperature):\n", " offset = 273.15\n", " @property\n", " def T(self):\n", " \"\"\"T in C\"\"\"\n", " return self._TK - self.offset\n", " @T.setter\n", " def T(self, T):\n", " self._TK = T + self.offset\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test34" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t = test34.Celsius()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t.T" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t = test34.Celsius()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t.T" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "t.T = 100" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t._TK" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t.T" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "using the `property` function you can also define properties that can only be set but not\n", "read.\n", "```python\n", "class Temperature(object):\n", " def __init__(self, T = 0):\n", " self.T = T\n", " @property\n", " def T(self):\n", " \"\"\"T in K\"\"\"\n", " return self._TK\n", " @T.setter\n", " def T(self, T):\n", " self._TK = T\n", " def e(self):\n", " \"\"\"energy density in cgs\"\"\"\n", " return 7.5657e-15 * self._TK**4\n", "\n", "class Celsius(Temperature):\n", " offset = 273.15\n", " @property\n", " def T(self):\n", " \"\"\"T in C\"\"\"\n", " return self._TK - self.offset\n", " @T.setter\n", " def T(self, T):\n", " self._TK = T + self.offset\n", " def _set_TF(self, T):\n", " self.T = (T - 40) * 5 / 9\n", " TF = property(fset = _set_TF)\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test35" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "t = test35.Celsius()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "t.TF = -20" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t.T" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "t.TF" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Regular Expressions\n", "Knowing how to use regular expressions will make your life much easier. Yes, it requires\n", "some work to get started. Python offers some powerful tools to use them for you text\n", "processing, e.g., extracting data from text files or web pages. The manual is at https://docs.python.org/3.4/library/re.html The main use it to match items that have\n", "certain patterns.\n", "\n", "Key tokens are ( and ) to “capture” strings you want to extract, . to match anything, `?`\n", "to match the previous item zero to one times, `*` to match the previous item any number\n", "of times, `+` to match the previous item at least once, ^ for beginning of string/line, `$` for\n", "end of string/line, `[` and `]` to define a set of characters, etc. You also have special tokens\n", "like `\\d` matching a digit, etc." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import re" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "re.findall('([123]+)', '1234ghgs7sjj4399')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "An example script I have used to renumber all the input/output prompts from IPython\n", "to be consecutive in the script.\n", "```python\n", "#! /usr/bin/env python3\n", "\n", "import sys, re, os\n", "\n", "def format(infile):\n", " outfile = infile + '.tmp'\n", " In = re.compile('^(In \\[)[0-9]+(\\]:.*)')\n", " Out = re.compile('^(Out\\[)[0-9]+(\\]:.*)')\n", " Search = (In, Out)\n", " count = 0\n", " with open(infile, 'rt') as f, open(outfile, 'xt') as g:\n", " for line in f:\n", " for prompt in Search:\n", " m = prompt.findall(line)\n", " if len(m) == 0:\n", " continue\n", " if prompt is In:\n", " count += 1\n", " line = prompt.sub(r'\\g<1>{:d}\\g<2>'.format(count),\n", " line)\n", " g.write(line)\n", " os.remove(infile)\n", " os.rename(outfile, infile)\n", "\n", "if __name__ == \"__main__\":\n", " format(sys.argv[1])\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### NumPy\n", "A very widely used convention is to import `numpy` as `np`" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you start IPython with the `--pylab` flag, it will do this automatically, however, in you\n", "scripts you still have to do it by hand.\n", "\n", "In the IPython notebook we can do the same with the `%pylab` macro" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%pylab" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "NumPy provides multi-dimensional array class, ndarray, optimized for numerical data\n", "processing. It allows various data types - it defines its own data types, called \"dtypes\",\n", "and even has record arrays. I recommend you the very good online documentation at\n", "http://www.numpy.org/.\n", "\n", "There is too much to tell about NumPy to fit into the time allocated for this course.\n", "Some of the key features are to allow you to do indexing nut just in the slice notation,\n", "but also by lists of indices or multi-dimensional constructs, or by arrays of truth values.\n", "The latter is extremely useful to avoid loop with if statements.\n", "\n", "**Key advice:** Use “advanced slicing” to replace loops with if statements! These are\n", "very slow. Should this really not be possible to avoid, you can easily write FORTRAN\n", "extension modules using `f2py.`\n", "\n", "Examples:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x = np.arange(12).reshape(3, -1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x.shape" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x[0,2]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x[2]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "ii = x % 2 == 1" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "ii" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x[ii] = 0" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "y = np.array([1, 2, 3, 4])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "y" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x *= y" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "y = np.array([1,2,3])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x *= y[:, np.newaxis]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "x" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The example above shows NumPy by default tries to match the last dimension and\n", "automatically expands the others. To change that, you can add “extra axes” using\n", "`np.newaxis`.\n", "\n", "Telling you all about *NumPy* would likely be an entire course by itself." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Matplotlib\n", "Matplotlib is a frequently used Python package for plotting. To use it interactively on the\n", "IPython shell, use the `--pylab` flag when starting IPython or to use the `%pylab` macro in the IPython notebook. Much of the interactive plotting interface is quite similar to Mathlab, so it may not seem all that strange. Matplotlib is fully object-oriented, and so is the graphics it produces: Objects can be modified, even interacted with, e.g., react to users clicking at them (advanced programming, I have never used).\n", "\n", "Whereas you can do things interactively on the shell for development - I do that, in part -\n", "I highly recommend that you put the finalized scripts inside a script, possibly a function,\n", "but surely preferable an object. Even it you just use the init function for all the\n", "plotting at first - later you can delegate some of the tasks, e.g., setups you frequently use,\n", "into separate routines, maybe in a base class (\"'MyPlot'\") from which you derive specialized\n", "plots. You can store properties of the plot, the figure object, axes objects, etc. in the\n", "class and later access them.\n", "\n", "The key is that having plots in scripts, you can easily modify things and rerun the script,\n", "or if your data or model has changed, you can just re-run the script - especially for last\n", "minute changes when you thesis is due (advisor says: \"Change the color of that line!\").\n", "\n", "There is a vast variety of different plot and recipes to make them. A prominent gallery\n", "with code examples can be found at http://matplotlib.org/gallery.html. Personally,\n", "I don’t think this is always done very well, but it is a good start. I highly recommend you\n", "have a look at the Beginner’s and Advanced guides (http://matplotlib.org/devdocs/contents.html).\n", "\n", "As an example, let’s make a simple line plot:\n", "```python\n", "import matplotlib as mpl\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import os.path\n", "\n", "projectpath = os.path.expanduser('~/xxx')\n", "\n", "class MyPlot(object):\n", " def __init__(self, func = lambda x: np.sin(x)**2):\n", " f = plt.figure()\n", " ax = f.add_subplot(1, 1, 1)\n", " x = np.linspace(1, 10, 100)\n", " y = func(x)\n", " ax.plot(x, y,\n", " color = 'r',\n", " lw = 3,\n", " label = 'Model A')\n", " ndata = 10\n", " x = np.random.choice(x, ndata)\n", " y = func(x) + np.random.rand(ndata) * 0.1\n", " ax.plot(x, y,\n", " color = 'b',\n", " marker = '+',\n", " linestyle = 'None',\n", " markersize = 12,\n", " markeredgewidth = 2,\n", " label = 'Data')\n", " ax.set_xscale('log')\n", " ax.set_xlabel(r'$x\\,(\\mathrm{cm})$')\n", " ax.set_ylabel(r'$\\sin^2\\left(x\\right)$')\n", " ax.legend(loc='best')\n", " f.tight_layout()\n", " plt.show()\n", " self.f = f\n", "\n", " def save(self, filename):\n", " self.f.savefig(os.path.join(projectpath, filename))\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import test37" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "p = test37.MyPlot()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "p.save('xxx.pdf')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Matplotlib will automatically create an output file format based on\n", "the file extension.\n", "\n", "Insted of having plots opne in separate wondows, you can also have them embedded into the notebook, as you may be used form other notebook tools. Do this with the `%matplotlib` macro" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "plot([0,1])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "you can disable this agian using `%matplotlib` wityhout arguments." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%matplotlib" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "plot([0,1])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**The End.**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Please feel free to contact me if you have further questions or need some help with tough python problems. " ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.1" } }, "nbformat": 4, "nbformat_minor": 0 }