Functools

Nice function-functions.

class ktz.functools.Cascade(prefix=None, **kwargs)

Cascading cached function execution.

This class is used to iteratively work with data. If a long pipeline requires much data to be processed linearly but some steps are very resoure intensive, this cascade is used to automatically resume the latest step and omit all previous steps. This is heavily used for iterative development of data processing pipelines in ipython notebooks.

Raises:

KeyError

Thrown if a value is requested which has not been cached yet

Examples

>>> # FIRST SESSION:
>>> !mkdir .cache
>>> from ktz.functools import Cascade                                                                                [70/193]
>>> # this defines two to be cached values x and y
>>> # where y depends on x
>>> run = Cascade(prefix='.cache', x='x.pkl', y='y.pkl')
>>> outside = 1
>>> @run.cache("x")
... def f():
...     return outside
...
>>> @run.cache("y")
... def g(a):
...     return a + 1
...
>>> x = f()
>>> x
1
>>> outside += 1
>>> outside
2
>>> # x is cached now!
>>> x = f()
>>> x
1
>>> # executing g is now preventing f
>>> # to be run in the future
>>> y = g(x)
>>> y
2
>>>
Do you really want to exit ([y]/n)? y
>>> # SECOND SESSION:
>>> from ktz.functools import Cascade                                                                                [70/193]
>>> # this defines two to be cached values x and y
>>> # where y depends on x
>>> run = Cascade(prefix='.cache', x='x.pkl', y='y.pkl')
>>> outside = 100
>>> @run.cache("x")
... def f():
...     return outside
...
>>> @run.cache("y")
... def g(a):
...     return a + 1
>>> # f is not executed as g has already
>>> # computed and cached a value
>>> x = f()
>>> x is None
True
>>> # g is not executed and the cached value
>>> # is used instead
>>> y = g(x)
>>> y
2

Methods

cache(name)	Decorate cache functions.
get(name)	Retrieve a cached object.
when(*names)	Decorate conditionally executed function.

cache(name)

Decorate cache functions.

This decorator handles whether a function is called or if data needs to be retrieved from the cache.

Parameters:

namestr

Data key

get(name)

Retrieve a cached object.

Retrieve object of the provided name from the cache.

Parameters:

namestr

Data key

Raises:

KeyError

If no cache file exists yet

when(*names)

Decorate conditionally executed function.

This decorator handles whether a function is invoked at all. It does not work with any returned data. Decorated functions always return None.

class ktz.functools.Maybe(cache, skip, loaded, cached, data=None)

Maybe it contains data, maybe not.

Used for Cascade, to determine whether to execute functions, load data from cache, or ignore supposed data all together.

Parameters:

Generic[T]T

Whatever is saved in the cache

Attributes:

data