mario.plugins package

Submodules

mario.plugins.basic module

mario.plugins.basic.apply(function, items)

Apply code to the iterable of items.

The code should take an iterable and it will be called with the input items. The items iterable will be converted to a list before the code is called, so it doesn’t work well on very large streams.

For example,

$ mario map int apply sum <<EOF
10
20
30
EOF
60

mario.plugins.basic.async_apply(function, items)

Apply code to an async iterable of items.

The code should take an async iterable.

mario.plugins.basic.async_chain(items, exit_stack)

Concatenate a sequence of input async iterables into one long async iterable.

Converts an async iterable of async iterables of items into an async iterable of items, like itertools.chain.from_iterable for async iterables.

mario.plugins.basic.async_filter(function, items, exit_stack, max_concurrent)

Keep input items that satisfy an asynchronous condition.

For example,

$ mario async-filter '(await asks.get(x)).json()["url"].endswith(("1", "3"))'  <<EOF
http://httpbin.org/delay/5
http://httpbin.org/delay/1
http://httpbin.org/delay/2
http://httpbin.org/delay/3
http://httpbin.org/delay/4
EOF
http://httpbin.org/delay/1
http://httpbin.org/delay/3

mario.plugins.basic.async_map(function, items, exit_stack, max_concurrent)

Run code on each input item asynchronously.

The order of inputs is retained in the outputs. However, the order of inputs does not determine the order in which each input is handled, only the order in which its result is emitted. To keep the order in which each input is handled, use the synchronous version, map.

In this example, we make requests that have a server-side delay of specified length. The input order is retained in the output by holding each item until its precedents are ready.

$ mario async-map 'await asks.get ! x.json()["url"]'  <<EOF
http://httpbin.org/delay/5
http://httpbin.org/delay/1
http://httpbin.org/delay/2
http://httpbin.org/delay/3
http://httpbin.org/delay/4
EOF
https://httpbin.org/delay/5
https://httpbin.org/delay/1
https://httpbin.org/delay/2
https://httpbin.org/delay/3
https://httpbin.org/delay/4

mario.plugins.basic.async_map_unordered(function, items, exit_stack, max_concurrent)

Run code on each input item asynchronously, without retaining input order.

Each result is emitted in the order it becomes ready, regardless of input order. Input order is also ignored when determining in which order to start handling each item. Results start emitting as soon as the first one is ready. It also saves memory because it doesn’t require accumulating results while waiting for previous items to become ready. For stricter ordering, see map or async_map.

In this example, we make requests that have a server-side delay of specified length. The input order is lost but the results appear immediately as they are ready (the delay length determines the output order):

$ mario async-map-unordered 'await asks.get ! x.json()["url"]'  <<EOF
http://httpbin.org/delay/5
http://httpbin.org/delay/1
http://httpbin.org/delay/2
http://httpbin.org/delay/3
http://httpbin.org/delay/4
EOF
https://httpbin.org/delay/1
https://httpbin.org/delay/2
https://httpbin.org/delay/3
https://httpbin.org/delay/4
https://httpbin.org/delay/5

mario.plugins.basic.build_callback(sub_command)

mario.plugins.basic.calculate_function(traversal, howcall=None)

mario.plugins.basic.calculate_reduce(traversal)

mario.plugins.basic.chain(items, exit_stack)

Concatenate a sequence of input iterables together into one long iterable.

Converts an iterable of iterables of items into an iterable of items, like itertools.chain.from_iterable.

For example,

$ mario eval '[[1,2]]'
[[1, 2]]


$ mario eval '[[1, 2]]' chain
[1, 2]

mario.plugins.basic.eval(function)

Evaluate a Python expression.

No input items are used.

For example,

$ mario eval 1+1
2

mario.plugins.basic.filter(function, items, exit_stack, max_concurrent)

Keep input items that satisfy a condition.

Order of input items is retained in the output.

For example,

$ mario filter 'x > "c"' <<EOF
a
b
c
d
e
f
EOF
d
e
f

mario.plugins.basic.map(function, items, exit_stack, max_concurrent)

Run code on each input item.

Each item is handled in the order it was received, and also output in the same order. For less strict ordering and asynchronous execution, see async-map and async-map-unordered.

For example,

$ mario map 'x*2' <<EOF
a
b
c
EOF
aa
bb
cc

mario.plugins.basic.reduce(function, items, exit_stack, max_concurrent)

Reduce input items with code that takes two arguments, similar to functools.reduce.

For example,

$ mario reduce map int operator.mul <<EOF
1
2
3
4
5
EOF
120

mario.plugins.read module

Functions for read commands.

mario.plugins.read.read_csv_dicts(file, **kwargs)

Read csv rows into an iterable of dicts.

Return type:Iterable[Mapping[Any, str]]

mario.plugins.read.read_csv_tuples(file, **kwargs)

Read csv rows into an iterable of tuples.

Return type:Iterable[Tuple]

mario.plugins.write module

Functions for write commands.

mario.plugins.write.write_csv_dicts(rows, header, dialect)

Write iterable of dicts to csv.

Return type:str

mario.plugins.write.write_csv_tuples(rows, dialect)

Write iterable of tuples to csv.

Return type:str

mario.plugins.write.write_yaml(data)

Write data to yaml string.

Return type:str

Module contents