Goulib.itertools2 module

additions to itertools standard library

Goulib.itertools2.identity(x)[source]

Do nothing and return the variable untouched

Goulib.itertools2.isiterable(obj)[source]
Result:bool True if obj is iterable (but not a string)
Goulib.itertools2.iscallable(f)[source]
Goulib.itertools2.anyf(seq, pred=<class 'bool'>)[source]
Result:bool True if pred(x) is True for at least one element in the iterable
Goulib.itertools2.allf(seq, pred=<class 'bool'>)[source]
Result:bool True if pred(x) is True for all elements in the iterable
Goulib.itertools2.no(seq, pred=<class 'bool'>)[source]
Result:bool True if pred(x) is False for every element in the iterable
Goulib.itertools2.index(value, iterable)[source]
Result:integer index of value in iterable
Goulib.itertools2.ith(iterable, i)[source]
Result:i-th element in the iterable
Goulib.itertools2.first(iterable)[source]
Result:first element in the iterable
Goulib.itertools2.last(iterable)[source]
Result:last element in the iterable
Goulib.itertools2.takeevery(n, iterable, start=0)[source]

Take an element from iterator every n elements

Goulib.itertools2.every(n, iterable, start=0)

Take an element from iterator every n elements

Goulib.itertools2.take(n, iterable)[source]
Result:first n items from iterable
Goulib.itertools2.drop(n, iterable)[source]

Drop n elements from iterable and return the rest

Goulib.itertools2.enumerates(iterable)[source]

generalizes enumerate to dicts :result: key,value pair for whatever iterable type

Goulib.itertools2.ilen(it)[source]
Result:int length exhausting an iterator
Goulib.itertools2.irange(start_or_end, optional_end=None)[source]
Result:iterable that counts from start to end (both included).
Goulib.itertools2.arange(start, stop=None, step=1)[source]

range for floats or other types (numpy.arange without numpy)

Parameters:
  • start – optional number. Start of interval. The interval includes this value. The default start value is 0.
  • stop – number. End of interval. The interval does not include this value, except in some cases where step is not an integer and floating point round-off affects the length of out.
  • step – optional number. Spacing between values. For any output out, this is the distance between two adjacent values, out[i+1] - out[i]. The default step size is 1.
Result:

iterator

Goulib.itertools2.linspace(start, end, n=100)[source]

iterator over n values linearly interpolated between (and including) start and end numpy.linspace without numpy

Parameters:
  • start – number, or iterable vector
  • end – number, or iterable vector
  • n – int number of interpolated values
Result:

iterator

Goulib.itertools2.flatten(it, donotrecursein=<class 'str'>)[source]

iterator to flatten (depth-first) structure

Parameters:
  • it – iterable structure
  • donotrecursein – iterable types in which algo doesn’t recurse string type by default
Goulib.itertools2.itemgetter(iterable, i)[source]
Goulib.itertools2.recurse(f, x)[source]
Goulib.itertools2.swap(iterable)[source]
Goulib.itertools2.tee(iterable, n=2, copy=None)[source]

tee or copy depending on type and goal

Parameters:
  • iterable – any iterable
  • n – int number of tees/copies to return
  • copy – optional copy function, for exemple copy.copy or copy.deepcopy
Result:

tee of iterable if it’s an iterator or generator, or (deep)copies for other types

this function is useful to avoid side effects at a lower memory cost depending on the case

Goulib.itertools2.groups(iterable, n, step=None)[source]

Make groups of ‘n’ elements from the iterable advancing ‘step’ elements on each iteration

Goulib.itertools2.pairwise(iterable, op=None, loop=False)[source]

iterates through consecutive pairs

Parameters:
  • iterable – input iterable s1,s2,s3, …. sn
  • op – optional operator to apply to each pair
  • loop – boolean True if last pair should be (sn,s1) to close the loop
Result:

pairs iterator (s1,s2), (s2,s3) … (si,si+1), … (sn-1,sn) + optional pair to close the loop

Goulib.itertools2.select(it1, it2, op)[source]
Goulib.itertools2.shape(iterable)[source]

shape of a mutidimensional array, without numpy

Parameters:iterable – iterable of iterable … of iterable or numpy arrays…
Result:list of n ints corresponding to iterable’s len of each dimension
Warning:if iterable is not a (hyper) rect matrix, shape is evaluated from

the [0,0,…0] element … :see: http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.ndarray.shape.html

Goulib.itertools2.ndim(iterable)[source]

number of dimensions of a mutidimensional array, without numpy

Parameters:iterable – iterable of iterable … of iterable or numpy arrays…
Result:int number of dimensions
Goulib.itertools2.reshape(data, dims)[source]
Result:data as a n-dim matrix
Goulib.itertools2.compose(f, g)[source]

Compose two functions -> compose(f, g)(x) -> f(g(x))

Goulib.itertools2.iterate(func, arg)[source]

After Haskell’s iterate: apply function repeatedly.

Goulib.itertools2.accumulate(iterable, func=<built-in function add>, skip_first=False, modulo=0)[source]

Return running totals. extends python.accumulate

# accumulate([1,2,3,4,5]) –> 1 3 6 10 15 # accumulate([1,2,3,4,5], operator.mul) –> 1 2 6 24 120

Goulib.itertools2.record(iterable, it=count(0), max=0)[source]

return the index and value of iterable which exceed previous max

Goulib.itertools2.record_index(iterable, it=count(0), max=0)[source]
Goulib.itertools2.record_value(iterable, it=count(0), max=0)[source]
Goulib.itertools2.tails(seq)[source]

Get tails of a sequence

tails([1,2,3]) -> [1,2,3], [2,3], [3], [].

Goulib.itertools2.ireduce(func, iterable, init=None)[source]

Like python.reduce but using iterators (a.k.a scanl)

Goulib.itertools2.occurences(iterable)[source]

count number of occurences of each item in a finite iterable

Parameters:iterable – finite iterable
Returns:dict of int count indexed by item
Goulib.itertools2.compress(iterable, key=<function identity>, buffer=None)[source]

generates (item,count) pairs by counting the number of consecutive items in iterable)

Parameters:
  • iterable – iterable, possibly infinite
  • key – optional function defining which elements are considered equal
  • buffer – optional integer. if defined, iterable is sorted with this buffer
Goulib.itertools2.decompress(iterable)[source]
Goulib.itertools2.unique(iterable, key=None, buffer=100)[source]

generate unique elements, preserving order. :param iterable: iterable, possibly infinite :param key: optional function defining which elements are considered equal :param buffer: optional integer defining how many of the last unique elements to keep in memory mandatory if iterable is infinite

# unique(‘AAAABBBCCDAABBB’) –> A B C D # unique(‘ABBCcAD’, str.lower) –> A B C D

Goulib.itertools2.count_unique(iterable, key=None)[source]

Count unique elements

# count_unique(‘AAAABBBCCDAABBB’) –> 4 # count_unique(‘ABBCcAD’, str.lower) –> 4

Goulib.itertools2.combinations_with_replacement(iterable, r)[source]

combinations_with_replacement(‘ABC’, 2) –> AA AB AC BB BC CC same as combinations_with_replacement except it doesn’t generate duplicates

Goulib.itertools2.takenth(n, iterable, default=None)[source]
Result:nth item of iterable
Goulib.itertools2.nth(n, iterable, default=None)
Result:nth item of iterable
Goulib.itertools2.icross(*sequences)[source]

Cartesian product of sequences (recursive version)

Goulib.itertools2.quantify(iterable, pred=<class 'bool'>)[source]
Result:int count how many times the predicate is true
Goulib.itertools2.interleave(l1, l2)[source]
Parameters:
  • l1 – iterable
  • l2 – iterable of same length, or 1 less than l1
Result:

iterable interleaving elements from l1 and l2, starting by l1[0]

Goulib.itertools2.shuffle(ary)[source]
Param:array to shuffle by Fisher-Yates algorithm
Result:shuffled array (IN PLACE!)
See:http://www.drgoulu.com/2013/01/19/comment-bien-brasser-les-cartes/
Goulib.itertools2.rand_seq(size)[source]
Result:range(size) shuffled
Goulib.itertools2.all_pairs(size)[source]

generates all i,j pairs for i,j from 0-size

Goulib.itertools2.index_min(values, key=<function identity>)[source]
Result:min_index, min_value
Goulib.itertools2.index_max(values, key=<function identity>)[source]
Result:max_index, max_value
Goulib.itertools2.best(iterable, key=None, n=1, reverse=False)[source]

generate items corresponding to the n best values of key sort order

Goulib.itertools2.sort_indexes(iterable, key=<function identity>, reverse=False)[source]
Returns:iterator over indexes of iterable that correspond to the sorted iterable
Goulib.itertools2.filter2(iterable, condition)[source]

like python.filter but returns 2 lists : - list of elements in iterable that satisfy condition - list of those that don’t

Goulib.itertools2.ifind(iterable, f, reverse=False)[source]

iterates through items in iterable where f(item) == True.

Goulib.itertools2.iremove(iterable, f)[source]

removes items from an iterable based on condition :param iterable: iterable . will be modified in place :param f: function of the form lambda line:bool returning True if item should be removed :yield: removed items backwards

Goulib.itertools2.removef(iterable, f)[source]

removes items from an iterable based on condition :param iterable: iterable . will be modified in place :param f: function of the form lambda line:bool returning True if item should be removed :result: list of removed items.

Goulib.itertools2.find(iterable, f)[source]

Return first item in iterable where f(item) == True.

Goulib.itertools2.isplit(iterable, sep, include_sep=False)[source]

split iterable by separators or condition :param sep: value or function(item) returning True for items that separate :param include_sep: bool. If True the separators items are included in output, at beginning of each sub-iterator :result: iterates through slices before, between, and after separators

Goulib.itertools2.split(iterable, sep, include_sep=False)[source]

like https://docs.python.org/2/library/stdtypes.html#str.split, but for iterable :param sep: value or function(item) returning True for items that separate :param include_sep: bool. If True the separators items are included in output, at beginning of each sub-iterator :result: list of iterable slices before, between, and after separators

Goulib.itertools2.dictsplit(dic, keys)[source]

extract keys from dic :param dic: dict source :param keys: iterable of dict keys :result: dict,dict : the first contains entries present in source, the second the remaining entries

Goulib.itertools2.next_permutation(seq, pred=<function <lambda>>)[source]

Like C++ std::next_permutation() but implemented as generator. see http://blog.bjrn.se/2008/04/lexicographic-permutations-using.html :param seq: iterable :param pred: a function (a,b) that returns a negative number if a<b, like cmp(a,b) in Python 2.7

class Goulib.itertools2.iter2(iterable)[source]

Bases: object

Takes in an object that is iterable. http://code.activestate.com/recipes/578092-flattening-an-arbitrarily-deep-list-or-any-iterato/ Allows for the following method calls (that should be built into iterators anyway…) calls: - append - appends another iterable onto the iterator. - insert - only accepts inserting at the 0 place, inserts an iterable before other iterables. - adding. an iter2 object can be added to another object that is iterable. i.e. iter2 + iter (not iter + iter2). It’s best to make all objects iter2 objects to avoid syntax errors. :D

__init__(iterable)[source]

Initialize self. See help(type(self)) for accurate signature.

append(iterable)[source]
insert(place, iterable)[source]
__add__(iterable)[source]
__next__()[source]
next()
__iter__()[source]
__class__

alias of builtins.type

__delattr__

Implement delattr(self, name).

__dir__()

Default dir() implementation.

__eq__

Return self==value.

__format__()

Default object formatter.

__ge__

Return self>=value.

__getattribute__

Return getattr(self, name).

__gt__

Return self>value.

__hash__

Return hash(self).

__init_subclass__()

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__le__

Return self<=value.

__lt__

Return self<value.

__ne__

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__reduce__()

Helper for pickle.

__reduce_ex__()

Helper for pickle.

__repr__

Return repr(self).

__setattr__

Implement setattr(self, name, value).

__sizeof__()

Size of object in memory, in bytes.

__str__

Return str(self).

Goulib.itertools2.subdict(d, keys)[source]

extract “sub-dictionary” :param d: dict :param keys: container of keys to extract: :result: dict: :see: http://stackoverflow.com/questions/5352546/best-way-to-extract-subset-of-key-value-pairs-from-python-dictionary-object/5352649#5352649

exception Goulib.itertools2.SortingError[source]

Bases: Exception

__cause__

exception cause

__class__

alias of builtins.type

__context__

exception context

__delattr__

Implement delattr(self, name).

__dir__()

Default dir() implementation.

__eq__

Return self==value.

__format__()

Default object formatter.

__ge__

Return self>=value.

__getattribute__

Return getattr(self, name).

__gt__

Return self>value.

__hash__

Return hash(self).

__init__

Initialize self. See help(type(self)) for accurate signature.

__init_subclass__()

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__le__

Return self<=value.

__lt__

Return self<value.

__ne__

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__reduce__()

Helper for pickle.

__reduce_ex__()

Helper for pickle.

__repr__

Return repr(self).

__setattr__

Implement setattr(self, name, value).

__setstate__()
__sizeof__()

Size of object in memory, in bytes.

__str__

Return str(self).

__suppress_context__
__traceback__
args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

Goulib.itertools2.ensure_sorted(iterable, key=None)[source]

makes sure iterable is sorted according to key

Yields:items of iterable
Raise:SortingError if not
Goulib.itertools2.sorted_iterable(iterable, key=None, buffer=100)[source]

sorts an “almost sorted” (infinite) iterable

Parameters:
  • iterable – iterable
  • key – function used as sort key
  • buffer – int size of buffer. elements to swap should not be further than that
Goulib.itertools2.diff(iterable1, iterable2)[source]

generate items in sorted iterable1 that are not in sorted iterable2

Goulib.itertools2.intersect(*iterables)[source]

generates itersection of N iterables

Parameters:iterables – any number of SORTED iterables
Yields:elements that belong to all iterables
See:http://stackoverflow.com/questions/969709/joining-a-set-of-ordered-integer-yielding-python-iterators
Goulib.itertools2.product(*iterables, **kwargs)[source]

Cartesian product of (infinite) input iterables.

Parameters:
  • iterables – any number of iterables
  • repeat – integer optional number of repetitions
See:

http://stackoverflow.com/questions/12093364/cartesian-product-of-large-iterators-itertools

class Goulib.itertools2.keep(iterable)[source]

Bases: collections.abc.Iterator

iterator that keeps the last value

__init__(iterable)[source]

Initialize self. See help(type(self)) for accurate signature.

__next__()[source]

Return the next item from the iterator. When exhausted, raise StopIteration

next()

Return the next item from the iterator. When exhausted, raise StopIteration

__abstractmethods__ = frozenset()
__class__

alias of abc.ABCMeta

__delattr__

Implement delattr(self, name).

__dir__()

Default dir() implementation.

__eq__

Return self==value.

__format__()

Default object formatter.

__ge__

Return self>=value.

__getattribute__

Return getattr(self, name).

__gt__

Return self>value.

__hash__

Return hash(self).

__init_subclass__()

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__iter__()
__le__

Return self<=value.

__lt__

Return self<value.

__ne__

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__reduce__()

Helper for pickle.

__reduce_ex__()

Helper for pickle.

__repr__

Return repr(self).

__setattr__

Implement setattr(self, name, value).

__sizeof__()

Size of object in memory, in bytes.

__slots__ = ()
__str__

Return str(self).

Goulib.itertools2.first_match(iter1, iter2, limit=None)[source]

” :param limit: int max number of loops :return: integer i first index where iter1[i]==iter2[i]

Goulib.itertools2.floyd(iterable, limit=1000000.0)[source]

Detect a cycle in iterable using Floyd “tortue hand hare” algorithm

See:

https://en.wikipedia.org/wiki/Cycle_detection#Floyd’s_Tortoise_and_Hare

Parameters:
  • iterable – iterable
  • limit – int limit to prevent infinite loop. no limit if None
Result:

(i,l) tuple of integers where i=index of cycle start, l=length if no cycle is found, return (None,None)

Goulib.itertools2.brent(iterable, limit=1000000.0)[source]

Detect a cycle in iterable using Floyd “tortue hand hare” algorithm

See:

https://en.wikipedia.org/wiki/Cycle_detection#Brent’s_algorithm

Parameters:
  • iterable – iterable
  • limit – int limit to prevent infinite loop. no limit if None
Result:

(i,l) tuple of integers where i=index of cycle start, l=length if no cycle is found, return (None,None)

Goulib.itertools2.detect_cycle(iterable, limit=1000000.0)[source]