Python (programming language)

Enhancement Proposals

Python Enhancement Proposals^{[note 1]} are a design document for either providing information to the Python community, or proposal for new feature in Python.^[80] PEPs are intented to explain new processes in Python, provide naming conventions or document the processes in the language.^[81] PEPs are overseen by Python Steering Council.^[81]

There are 3 kinds of PEPs, with those are being standards track PEP^{[note 2]}, Informational PEP^{[note 3]} and Process PEPs^{[note 4]} which has their own unique meanings.^[80][82] They were firstly introduced in 2000, inspired by other RfCs (requests for comments) and Design Enhancement Proposals.^[82] Most known PEPs are PEP – 1, PEP – 8, PEP – 20, PEP – 257 and others.^[82]

Indentation

Python uses whitespace indentation, rather than curly brackets or keywords, to delimit blocks. An increase in indentation comes after certain statements; a decrease in indentation signifies the end of the current block.^[84] Thus, the program's visual structure accurately represents its semantic structure.^[85] This feature is sometimes termed the off-side rule. Some other languages use indentation this way; but in most, indentation has no semantic meaning. The recommended indent size is four spaces.^[86]

Statements and control flow

Python's statements include the following:

• The assignment statement, using a single equals sign =
• The if statement, which conditionally executes a block of code, along with else and elif (a contraction of else if)
• The for statement, which iterates over an iterable object, capturing each element to a variable for use by the attached block; the variable is not deleted when the loop finishes
• The while statement, which executes a block of code as long as boolean condition is true
• The try statement, which allows exceptions raised in its attached code block to be caught and handled by except clauses (or new syntax except* in Python 3.11 for exception groups);^[87] the try statement also ensures that clean-up code in a finally block is always run regardless of how the block exits
• The raise statement, used to raise a specified exception or re-raise a caught exception
• The class statement, which executes a block of code and attaches its local namespace to a class, for use in object-oriented programming
• The def statement, which defines a function or method
• The with statement, which encloses a code block within a context manager, allowing resource-acquisition-is-initialization (RAII)-like behavior and replacing a common try/finally idiom^[88] Examples of a context include acquiring a lock before some code is run, and then releasing the lock; or opening and then closing a file
• The break statement, which exits a loop
• The continue statement, which skips the rest of the current iteration and continues with the next
• The del statement, which removes a variable—deleting the reference from the name to the value, and producing an error if the variable is referred to before it is redefined^[d]
• The pass statement, serving as a NOP (i.e., no operation), which is syntactically needed to create an empty code block
• The assert statement, used in debugging to check for conditions that should apply
• The yield statement, which returns a value from a generator function (and also an operator); used to implement coroutines
• The return statement, used to return a value from a function
• The import and from statements, used to import modules whose functions or variables can be used in the current program. Python 3.15 adds a new functionality to lazily import with a new keyword: "The lazy keyword works with both import and from ... import statements."^[41]
• The match and case statements, analogous to a switch statement construct, which compares an expression against one or more cases as a control-flow measure

The assignment statement (=) binds a name as a reference to a separate, dynamically allocated object. Variables may subsequently be rebound at any time to any object. In Python, a variable name is a generic reference holder without a fixed data type; however, it always refers to some object with a type. This is called dynamic typing—in contrast to statically-typed languages, where each variable may contain only a value of a certain type.

Python does not support tail call optimization or first-class continuations; according to Van Rossum, the language never will.^[89][90] However, better support for coroutine-like functionality is provided by extending Python's generators.^[91] Before 2.5, generators were lazy iterators; data was passed unidirectionally out of the generator. From Python 2.5 on, it is possible to pass data back into a generator function; and from version 3.3, data can be passed through multiple stack levels.^[92]

Expressions

Python's expressions include the following:

• The +, -, and * operators for mathematical addition, subtraction, and multiplication are similar to other languages, but the behavior of division differs. There are two types of division in Python: floor division (or integer division) //, and floating-point division /.^[93] Python uses the ** operator for exponentiation.
• Python uses the + operator for string concatenation. The language uses the * operator for duplicating a string a specified number of times.
• The @ infix operator is intended to be used by libraries such as NumPy for matrix multiplication.^[94][95]
• The syntax :=, called the "walrus operator", was introduced in Python 3.8. This operator assigns values to variables as part of a larger expression.^[96]
• In Python, == compares two objects by value. Python's is operator may be used to compare object identities (i.e., comparison by reference), and comparisons may be chained—for example, a <= b <= c.
• Python uses and, or, and not as Boolean operators.
• Python has a type of expression called a list comprehension, and a more general expression called a generator expression.^[64]
• Anonymous functions are implemented using lambda expressions; however, there may be only one expression in each body.
• Conditional expressions are written as x if c else y.^[97] (This is different in operand order from the c ? x : y operator common to many other languages.)
• Python makes a distinction between lists and tuples. Lists are written as [1, 2, 3], are mutable, and cannot be used as the keys of dictionaries (since dictionary keys must be immutable in Python). Tuples, written as (1, 2, 3), are immutable and thus can be used as the keys of dictionaries, provided that all of the tuple's elements are immutable. The + operator can be used to concatenate two tuples, which does not directly modify their contents, but produces a new tuple containing the elements of both. For example, given the variable t initially equal to (1, 2, 3), executing t = t + (4, 5) first evaluates t + (4, 5), which yields (1, 2, 3, 4, 5); this result is then assigned back to t—thereby effectively "modifying the contents" of t while conforming to the immutable nature of tuple objects. Parentheses are optional for tuples in unambiguous contexts.^[98]
• Python features sequence unpacking where multiple expressions, each evaluating to something assignable (e.g., a variable or a writable property) are associated just as in forming tuple literal; as a whole, the results are then put on the left-hand side of the equal sign in an assignment statement. This statement expects an iterable object on the right-hand side of the equal sign to produce the same number of values as the writable expressions on the left-hand side; while iterating, the statement assigns each of the values produced on the right to the corresponding expression on the left.^[99]
• Python has a "string format" operator % that functions analogously to printf format strings in the C language—e.g. "spam=%s eggs=%d" % ("blah", 2) evaluates to "spam=blah eggs=2". In Python 2.6+ and 3+, this operator was supplemented by the format() method of the str class, e.g., "spam={0} eggs={1}".format("blah", 2). Python 3.6 added "f-strings": spam = "blah"; eggs = 2; f'spam={spam} eggs={eggs}'.^[100]
• Strings in Python can be concatenated by "adding" them (using the same operator as for adding integers and floats); e.g., "spam" + "eggs" returns "spameggs". If strings contain numbers, they are concatenated as strings rather than as integers, e.g. "2" + "2" returns "22".
• Python supports string literals in several ways:
  • Delimited by single or double quotation marks; single and double quotation marks have equivalent functionality (unlike in Unix shells, Perl, and Perl-influenced languages). Both marks use the backslash (\) as an escape character. String interpolation became available in Python 3.6 as "formatted string literals".^[100]
  • Triple-quoted, i.e., starting and ending with three single or double quotation marks; this may span multiple lines and function like here documents in shells, Perl, and Ruby.
  • Raw string varieties, denoted by prefixing the string literal with r. Escape sequences are not interpreted; hence raw strings are useful where literal backslashes are common, such as in regular expressions and Windows-style paths. (Compare "@-quoting" in C#.)
• Python has array index and array slicing expressions in lists, which are written as a[key], a[start:stop] or a[start:stop:step]. Indexes are zero-based, and negative indexes are relative to the end. Slices take elements from the start index up to, but not including, the stop index. The (optional) third slice parameter, called step or stride, allows elements to be skipped or reversed. Slice indexes may be omitted—for example, a[:] returns a copy of the entire list. Each element of a slice is a shallow copy.

In Python, a distinction between expressions and statements is rigidly enforced, in contrast to languages such as Common Lisp, Scheme, or Ruby. This distinction leads to duplicating some functionality, for example:

• List comprehensions vs. for-loops
• Conditional expressions vs. if blocks
• The eval() vs. exec() built-in functions (in Python 2, exec is a statement); the former function is for expressions, while the latter is for statements

A statement cannot be part of an expression; because of this restriction, expressions such as list and dict comprehensions (and lambda expressions) cannot contain statements. As a particular case, an assignment statement such as a = 1 cannot be part of the conditional expression of a conditional statement.

Typing

Python uses duck typing, and it has typed objects but untyped variable names. Type constraints are not checked at definition time; rather, operations on an object may fail at usage time, indicating that the object is not of an appropriate type. Despite being dynamically typed, Python is strongly typed, forbidding operations that are poorly defined (e.g., adding a number and a string) rather than quietly attempting to interpret them.

Python allows programmers to define their own types using classes, most often for object-oriented programming. New instances of classes are constructed by calling the class, for example, SpamClass() or EggsClass()); the classes are instances of the metaclass type (which is an instance of itself), thereby allowing metaprogramming and reflection.

Before version 3.0, Python had two kinds of classes, both using the same syntax: old-style and new-style.^[101] Current Python versions support the semantics of only the new style.

Python supports optional type annotations.^[5][102] These annotations are not enforced by the language, but may be used by external tools such as mypy to catch errors. Python includes a module typing including several type names for type annotations.^[103][104] Also, mypy supports a Python compiler called mypyc, which leverages type annotations for optimization.^[105]

"""Spanning
multiple
lines"""

Arithmetic operations

Python includes conventional symbols for arithmetic operators (+, -, *, /), the floor-division operator //, and the modulo operator %. (With the modulo operator, a remainder can be negative, e.g., 4 % -3 == -2.) Python also offers the ** symbol for exponentiation, e.g. 5**3 == 125 and 9**0.5 == 3.0, as well as the matrix‑multiplication operator @.^[112] These operators work as in traditional mathematics; with the same precedence rules, the infix operators + and - can also be unary, to represent positive and negative numbers respectively.

Division between integers produces floating-point results. The behavior of division has changed significantly over time:^[113]

• The current version of Python (i.e., since 3.0) changed the / operator to always represent floating-point division, e.g., 5/2 == 2.5.
• The floor division // operator was introduced, meaning that 7//3 == 2, -7//3 == -3, 7.5//3 == 2.0, and -7.5//3 == -3.0. For Python 2.7, adding the from __future__ import division statement allows a module in Python 2.7 to use Python 3.x rules for division (see above).

In Python terms, the / operator represents true division (or simply division), while the // operator represents floor division. Before version 3.0, the / operator represents classic division.^[113]

Rounding towards negative infinity, though a different method than in most languages, adds consistency to Python. For instance, this rounding implies that the equation (a + b)//b == a//b + 1 is always true. Also, the rounding implies that the equation b*(a//b) + a%b == a is valid for both positive and negative values of a. As expected, the result of a%b lies in the half-open interval [0, b), where b is a positive integer; however, maintaining the validity of the equation requires that the result must lie in the interval (b, 0] when b is negative.^[114]

Python provides a round function for rounding a float to the nearest integer. For tie-breaking, Python 3 uses the round to even method: round(1.5) and round(2.5) both produce 2.^[115] Python versions before 3 used the round-away-from-zero method: round(0.5) is 1.0, and round(-0.5) is −1.0.^[116]

Python allows Boolean expressions that contain multiple equality relations to be consistent with general usage in mathematics. For example, the expression a < b < c tests whether a is less than b and b is less than c.^[117] C-derived languages interpret this expression differently: in C, the expression would first evaluate a < b, resulting in 0 or 1, and that result would then be compared with c.^[118]

Python uses arbitrary-precision arithmetic for all integer operations. The Decimal type/class in the decimal module provides decimal floating-point numbers to a pre-defined arbitrary precision with several rounding modes.^[119] The Fraction class in the fractions module provides arbitrary precision for rational numbers.^[120]

Due to Python's extensive mathematics library and the third-party library NumPy, the language is frequently used for scientific scripting in tasks such as numerical data processing and manipulation.^[121][122]

Function syntax

Functions are created in Python by using the def keyword. A function is defined similarly to how it is called, by first providing the function name and then the required parameters. Here is an example of a function that prints its inputs:

def printer(input1, input2 = "already there"):
    print(input1)
    print(input2)

printer("hello")

# Example output:
# hello
# already there

To assign a default value to a function parameter in case no actual value is provided at run time, variable-definition syntax can be used inside the function header.

Reference implementation

CPython is the reference implementation of Python. This implementation is written in C; meeting the C11 standard^[133] since version 3.11. Older versions use the C89 standard with several select C99 features, but third-party extensions are not limited to older C versions—e.g., they can be implemented using C11 or C++.^[134][135] CPython compiles Python programs into an intermediate bytecode,^[136] which is then executed by a virtual machine.^[137] CPython is distributed with a large standard library written in a combination of C and native Python.

CPython is available for many platforms, including Windows and most modern Unix-like systems, including macOS (and Apple M1 Macs, since Python 3.9.1, using an experimental installer). Starting with Python 3.9, the Python installer intentionally fails to install on Windows 7 and 8;^[138][139] Windows XP was supported until Python 3.5. Old Python versions unofficially support VMS (mostly supporting^[140]) and OpenVMS x86-64 has Python 3.10 support.^[141][142] Platform portability was one of Python's earliest priorities.^[143] During development of Python 1 and 2, even OS/2 and Solaris were supported;^[8] since that time, support has been dropped for many platforms.

All current Python versions (since 3.7) support only operating systems that feature multithreading, by now supporting not nearly as many operating systems (dropping many outdated) than in the past.

Limitations of the reference implementation

• The energy usage of Python with CPython for typically written code is much worse than C by a factor of 75.88.^[144]
• The throughput of Python with CPython for typically written code is worse than C by a factor of 71.9.^[144]
• The average memory usage of CPython for typically written code is worse than C by a factor of 2.4.^[144]

Other implementations

All alternative implementations have at least slightly different semantics. For example, an alternative may include unordered dictionaries, in contrast to other current Python versions. As another example in the larger Python ecosystem, PyPy does not support the full C Python API.

Creating an executable with Python often is done by bundling an entire Python interpreter into the executable, which causes binary sizes to be massive for small programs,^[145] yet there exist implementations that are capable of truly compiling Python. Alternative implementations include the following:

• PyPy is a faster, compliant interpreter of Python 2.7 and 3.11.^[146][147] PyPy's just-in-time compiler often improves speed significantly relative to CPython, but PyPy does not support some libraries written in C.^[148] PyPy offers support for the RISC-V instruction-set architecture.
• Codon is an implementation with an ahead-of-time (AOT) compiler, which compiles a statically-typed Python-like language whose "syntax and semantics are nearly identical to Python's, there are some notable differences"^[149] For example, Codon uses 64-bit machine integers for speed, not arbitrarily as with Python; Codon developers claim that speedups over CPython are usually on the order of ten to a hundred times. Codon compiles to machine code (via LLVM) and supports native multithreading.^[150] Codon can also compile to Python extension modules that can be imported and used from Python.
• MicroPython and CircuitPython are Python 3 variants that are optimized for microcontrollers, including the Lego Mindstorms EV3.^[151]
• Pyston is a variant of the Python runtime that uses just-in-time compilation to speed up execution of Python programs.^[152]
• Cinder is a performance-oriented fork of CPython 3.8 that features a number of optimizations, including bytecode inline caching, eager evaluation of coroutines, a method-at-a-time JIT, and an experimental bytecode compiler.^[153]
• The Snek^{[154][155][156]} embedded computing language "is Python-inspired, but it is not Python. It is possible to write Snek programs that run under a full Python system, but most Python programs will not run under Snek."^[157] Snek is compatible with 8-bit AVR microcontrollers such as ATmega 328P-based Arduino, as well as larger microcontrollers that are compatible with MicroPython. Snek is an imperative language that (unlike Python) omits object-oriented programming. Snek supports only one numeric data type, which features 32-bit single precision (resembling JavaScript numbers, though smaller).
• RustPython is an implementation written in Rust language. It aims to be compatible with CPython, including its C-ABI.^[158] Currently, it is used in GrepTimeDB and Ruff among other projects.

Unsupported implementations

Stackless Python is a significant fork of CPython that implements microthreads. This implementation uses the call stack differently, thus allowing massively concurrent programs. PyPy also offers a stackless version.^[159]

Just-in-time Python compilers have been developed, but are now unsupported:

• Google began a project named Unladen Swallow in 2009: this project aimed to speed up the Python interpreter five-fold by using LLVM, and improve multithreading capability for scaling to thousands of cores,^[160] while typical implementations are limited by the global interpreter lock.
• Psyco is a discontinued just-in-time specializing compiler, which integrates with CPython and transforms bytecode to machine code at runtime. The emitted code is specialized for certain data types and is faster than standard Python code. Psyco does not support Python 2.7 or later.
• PyS60 was a Python 2 interpreter for Series 60 mobile phones, which was released by Nokia in 2005. The interpreter implemented many modules from Python's standard library, as well as additional modules for integration with the Symbian operating system. The Nokia N900 also supports Python through the GTK widget library, allowing programs to be written and run on the target device.^[161]

Transpilers to other languages

There are several compilers/transpilers to high-level object languages; the source language is unrestricted Python, a subset of Python, or a language similar to Python:

• Brython^[162] and Transcrypt^[163][164] compile Python to JavaScript.
• Cython compiles a superset of Python to C. The resulting code can be used with Python via direct C-level API calls into the Python interpreter.
• PyJL compiles/transpiles a subset of Python to "human-readable, maintainable, and high-performance Julia source code".^[75] Despite the developers' performance claims, this is not possible for arbitrary Python code; that is, compiling to a faster language or machine code is known to be impossible in the general case. The semantics of Python might potentially be changed, but in many cases speedup is possible with few or no changes in the Python code. The faster Julia source code can then be used from Python or compiled to machine code.
• Nuitka compiles Python into C.^[165] This compiler works with Python 3.4 to 3.13 (and 2.6 and 2.7) for Python's main supported platforms (and Windows 7 or even Windows XP) and for Android. The compiler developers claim full support for Python 3.10, partial support for Python 3.11 and 3.12, and experimental support for Python 3.13. Nuitka supports macOS including Apple Silicon-based versions. The compiler is free of cost, though it has commercial add-ons (e.g., for hiding source code).
• Numba is a JIT compiler that is used from Python; the compiler translates a subset of Python and NumPy code into fast machine code. This tool is enabled by adding a decorator to the relevant Python code.
• Pythran compiles a subset of Python 3 to C++ (C++11).^[166]
• RPython can be compiled to C, and it is used to build the PyPy interpreter for Python.
• The Python → 11l → C++ transpiler^[167] compiles a subset of Python 3 to C++ (C++17).

There are also specialized compilers:

• MyHDL is a Python-based hardware description language (HDL) that converts MyHDL code to Verilog or VHDL code.

Some older projects existed, as well as compilers not designed for use with Python 3.x and related syntax:

• Google's Grumpy transpiles Python 2 to Go.^{[168][169][170]} The latest release was in 2017.
• IronPython allows running Python 2.7 programs with the .NET Common Language Runtime.^[171] An alpha version (released in 2021), is available for "Python 3.4, although features and behaviors from later versions may be included."^[172]
• Jython compiles Python 2.7 to Java bytecode, allowing the use of Java libraries from a Python program.^[173]
• Pyrex (last released in 2010) and Shed Skin (last released in 2013) compile to C and C++ respectively.

Performance

A performance comparison among various Python implementations, using a non-numerical (combinatorial) workload, was presented at EuroSciPy '13.^[174] In addition, Python's performance relative to other programming languages is benchmarked by The Computer Language Benchmarks Game.^[175]

There are several approaches to optimizing Python performance, despite the inherent slowness of an interpreted language. These approaches include the following strategies or tools:

• Just-in-time compilation: Dynamically compiling parts of a Python program during the execution of the program. This technique is used in libraries such as Numba and PyPy.
• Static compilation: Sometimes, Python code can be compiled into machine code sometime before execution. An example of this approach is Cython, which compiles Python into C.
• Concurrency and parallelism: Multiple tasks can be run simultaneously. Python contains modules such as `multiprocessing` to support this form of parallelism. Moreover, this approach helps to overcome limitations of the Global Interpreter Lock (GIL) in CPU tasks.
• Efficient data structures: Performance can also be improved by using data types such as Set for membership tests, or deque from collections for queue operations.
• Performance gains can be observed by utilizing libraries such as NumPy. Most high performance Python libraries use C or Fortran under the hood instead of the Python interpreter.^[176]

Sources

• "Python for Artificial Intelligence". Python Wiki. 19 July 2012. Archived from the original on 1 November 2012. Retrieved 3 December 2012.
• Paine, Jocelyn, ed. (August 2005). "AI in Python". AI Expert Newsletter. Amzi!. Archived from the original on 26 March 2012. Retrieved 11 February 2012.
• "PyAIML 0.8.5: Python Package Index". Pypi.python.org. Retrieved 17 July 2013.
• Russell, Stuart J. & Norvig, Peter (2009). Artificial Intelligence: A Modern Approach (3rd ed.). Upper Saddle River, NJ: Prentice Hall. ISBN 978-0-13-604259-4.