# Iteration

## Iterables & Iteration

### Comprehensions

• Comprehensions can process more than one input sequence
• Multiple input sequences in comprehensions work like nested for-loops
• Comprehensions can also have multiple if-clauses interspersed with the for-clauses
• Later clauses in a comprehension can reference variables bound in earlier clauses
• Comprehension can also appear in the result expression of a comprehension, resulting in nested sequences

#### Example

``````## Comprehensions
>>> points
[(0, 0), (0, 1), (0, 2), (0, 3), (1, 0), (1, 1), (1, 2), (1, 3), (2, 0), (2, 1), (2, 2), (2, 3)]
>>> points_c=[(x,y) for x in range(3) for y in range(4) ]
>>> points_c
[(0, 0), (0, 1), (0, 2), (0, 3), (1, 0), (1, 1), (1, 2), (1, 3), (2, 0), (2, 1), (2, 2), (2, 3)]
>>> points_d=[(x,y)
...     for x in range(3)
...     if x > 0
...     for y in range(4)
...     if y > x ]
>>> points_d
[(1, 2), (1, 3), (2, 3)]
>>> points_e = [ [ y-1 for y in range(x)  ] for x in range(3)]
>>> points_e
[[], [-1], [-1, 0]]
``````

### Functional-style tools

• Python provides a number of functional-style tools for working with iterators

#### Map

• `map()` calls a function for each element in its input sequences
• `map()` returns an iterable object, not a fully-evaluated collection
• `map()` results are lazily evaluated, meaning that you must access them to force their calculation
• `map()` results are typically evaluated through the use of iteration constructs such as for-loops
• You must provide as many input sequences to `map()` as the callable argument has parameters
• `map()` takes one element from each input sequence for each output element it produces
• `map()` stops producing output when its shortest input sequence is exhausted
• `map()` can be used to implement the same behavior as comprehensions in some cases
##### Example
``````>>> sizes =['small','medium', 'large']
>>> colors = ['lavender', 'teal', 'burnt orange']
>>> animals = ['koala', 'platypus', 'salamander']
>>> def combine( size, color, animal):
...     return '{} {} {}'.format(size, color, animal)
...
>>> list(map(combine, sizes, colors, animals ))
['small lavender koala', 'medium teal platypus', 'large burnt orange salamander']
>>> import itertools
>>> def combine2(quantity, size, color, animal):
...     return '{} - {} {} {}'.format(quantity, size, color, animal)
...
>>> list(map(combine2, itertools.count(), sizes, colors, animals ))
['0 - small lavender koala', '1 - medium teal platypus', '2 - large burnt orange salamander']
``````

#### Filter

• `filter()` selects values from an input sequence which match a specified criteria
• `filter()` passes each element in its input sequence to the function argument
• `filter()` returns an iterable over the input elements for which the function argument is truthy
• Like `map()`, `filter()` produces its output lazily If you pass None as the first argument to `filter()`, it yields the input values which evaluate to True in a boolean context `reduce()` cumulatively applies a function to the elements of an input sequence
##### Example
``````## filter
>>> negs = filter(lambda x: x<0 , [3, -1, 2,-4,0,9,-33])
>>> list(negs)
[-1, -4, -33]
>>> notnones = filter(None, [0,1, False, True, [], () , {}, (1,2,), [1,2], '', 'yes'])
>>> list(map(type, list(notnones)))
[<class 'int'>, <class 'bool'>, <class 'tuple'>, <class 'list'>, <class 'str'>]
``````

#### Reduce

• `reduce()` calls the input function with two arguments: the accumulated result so far, and the next element in the sequence
• `reduce()` is a generalization of summation
• `reduce()` returns the accumulated result after all of the input has been processed
• If you pass an empty sequence to `reduce()` it will raise a TypeError
• `reduce()` accepts an optional initial value argument
• This initial value is conceptually added to the front of the input sequence
• The initial value is returned if the input sequence is empty
• The `map()` and `reduce()` functions in Python are related to the ideas in the map-reduce algorithm
##### Example
``````## reduce
>>> from functools import reduce
>>> import operator
15

### x is interim result , y is the next sequence value
>>> def mul(x, y):
...     print(' mul {} {} '.format(x,y))
...     return x * y
...
>>> reduce( mul, range(1, 5))
mul 1 2
mul 2 3
mul 6 4
24
``````

### The next() function

• Python’s next() function calls `__next__()` on its argument
• Iterators in Python must support the `__next__()` method
• `__next__()` should return the next item in the sequence, or raise `StopIteration` if it is exhausted
• Python’s `iter()` function calls `__iter__()` on its argument
• Iterable objects in Python must support the `__iter__()` method
• `__iter__()` should return an iterator for the iterable object
• Objects with a `__getitem__()` method that accepts consecutive integer indices starting at zero are also iterables
• Iterables implemented via `__getitem__()` must raise `IndexError` when they are exhausted
##### Example of Iterator
``````class ExampleIterator:
def __init__(self, data):
self.index = 0
self.data = data

def __iter__(self):
return self

def __next__(self):
if self.index >= len(self.data):
raise StopIteration()

rslt = self.data[self.index]
self.index += 1
return rslt
``````
##### Example of Iterable
``````class ExampleIterable:
def __init__(self):
self.data = [1, 2, 3]

def __iter__(self):
return ExampleIterator(self.data)
``````
##### Example of AnotherIterable
``````class AnotherIterable:
def __init__(self):
self.data = [1, 2, 3]

def __getitem__(self, idx):
return self.data[idx]
``````

### The iter() function

• The extended form of `iter()` accepts a zero-argument callable and a sentinel value
• Extended `iter()` repeatedly calls the callable argument until it returns the sentinel value
• The values produced by extended `iter()` are those returned from the callable
##### Example
``````>>> ts = iter(datetime.datetime.now, None)
>>> next(ts)
datetime.datetime(2017, 7, 14, 14, 38, 10, 752761)
>>> next(ts)
datetime.datetime(2017, 7, 14, 14, 38, 13, 373613)
>>> next(ts)
datetime.datetime(2017, 7, 14, 14, 38, 14, 754588)

## Content of the file file.txt
## the file
## the
## END
## but not see the END above

>>> with open('file.txt', 'rt') as f:
...     for line in iter(lambda: f.readline().strip(), 'END'):
...             print(line)
...
the file
the
``````
• One use case for extended `iter()` is to iterate using simple functions
• Protocol conforming iterators must also be iterable
##### Example of Sensor
``````import datetime
import itertools
import random
import time

class Sensor:
def __iter__(self):
return self

def __next__(self):
return random.random()

sensor = Sensor()
timestamps = iter(datetime.datetime.now, None)

for stamp, value in itertools.islice(zip(timestamps, sensor), 10):
print(stamp, value)
time.sleep(1)

``````