mario.plugins package

Submodules

mario.plugins.basic module

async mario.plugins.basic.apply(function, items)

Apply code to the iterable of items.

The code should take an iterable and it will be called with the input items. The items iterable will be converted to a list before the code is called, so it doesn’t work well on very large streams.

For example,

$ mario map int apply sum <<EOF
10
20
30
EOF
60

async mario.plugins.basic.async_filter(function, items, exit_stack, max_concurrent)

Keep input items that satisfy an asynchronous condition.

For example,

$ mario async-filter '(await asks.get(x)).json()["url"].endswith(("1", "3"))'  <<EOF
http://httpbin.org/delay/5
http://httpbin.org/delay/1
http://httpbin.org/delay/2
http://httpbin.org/delay/3
http://httpbin.org/delay/4
EOF
http://httpbin.org/delay/1
http://httpbin.org/delay/3

async mario.plugins.basic.async_map(function, items, exit_stack, max_concurrent)

Run code on each input item asynchronously.

The order of inputs is retained in the outputs. However, the order of inputs does not determine the order in which each input is handled, only the order in which its result is emitted. To keep the order in which each input is handled, use the synchronous version, map.

In this example, we make requests that have a server-side delay of specified length. The input order is retained in the output by holding each item until its precedents are ready.

$ mario async-map 'await asks.get ! x.json()["url"]'  <<EOF
http://httpbin.org/delay/5
http://httpbin.org/delay/1
http://httpbin.org/delay/2
http://httpbin.org/delay/3
http://httpbin.org/delay/4
EOF
https://httpbin.org/delay/5
https://httpbin.org/delay/1
https://httpbin.org/delay/2
https://httpbin.org/delay/3
https://httpbin.org/delay/4

async mario.plugins.basic.async_map_unordered(function, items, exit_stack, max_concurrent)

Run code on each input item asynchronously, without retaining input order.

Each result is emitted in the order it becomes ready, regardless of input order. Input order is also ignored when determining in which order to start handling each item. Results start emitting as soon as the first one is ready. It also saves memory because it doesn’t require accumulating results while waiting for previous items to become ready. For stricter ordering, see map or async_map.

In this example, we make requests that have a server-side delay of specified length. The input order is lost but the results appear immediately as they are ready (the delay length determines the output order):

$ mario async-map-unordered 'await asks.get ! x.json()["url"]'  <<EOF
http://httpbin.org/delay/5
http://httpbin.org/delay/1
http://httpbin.org/delay/2
http://httpbin.org/delay/3
http://httpbin.org/delay/4
EOF
https://httpbin.org/delay/1
https://httpbin.org/delay/2
https://httpbin.org/delay/3
https://httpbin.org/delay/4
https://httpbin.org/delay/5

mario.plugins.basic.build_callback(sub_command)

mario.plugins.basic.calculate_function(traversal, howcall=None)

mario.plugins.basic.calculate_reduce(traversal)

async mario.plugins.basic.chain(items, exit_stack)

Concatenate a sequence of input iterables together into one long iterable.

Converts an iterable of iterables of items into an iterable of items, like itertools.chain.from_iterable.

For example,

$ mario eval '[[1,2]]'
[[1, 2]]


$ mario eval '[[1, 2]]' chain
[1, 2]

async mario.plugins.basic.eval(function)

Evaluate a Python expression.

No input items are used.

For example,

$ mario eval 1+1
2

async mario.plugins.basic.filter(function, items, exit_stack, max_concurrent)

Keep input items that satisfy a condition.

Order of input items is retained in the output.

For example,

$ mario filter 'x > "c"' <<EOF
a
b
c
d
e
f
EOF
d
e
f

async mario.plugins.basic.map(function, items, exit_stack, max_concurrent)

Run code on each input item.

Each item is handled in the order it was received, and also output in the same order. For less strict ordering and asynchronous execution, see async-map and async-map-unordered.

For example,

$ mario map 'x*2' <<EOF
a
b
c
EOF
aa
bb
cc

async mario.plugins.basic.reduce(function, items, exit_stack, max_concurrent)

Reduce input items with code that takes two arguments, similar to functools.reduce.

For example,

$ mario reduce map int operator.mul <<EOF
1
2
3
4
5
EOF
120

mario.plugins.read module

Functions for read commands.

mario.plugins.read.read_csv_dicts(file, **kwargs)

Read csv rows into an iterable of dicts.

Return type

Iterable[Mapping[Any, str]]

mario.plugins.read.read_csv_tuples(file, **kwargs)

Read csv rows into an iterable of tuples.

Return type

Iterable[Tuple]

mario.plugins.write module

Functions for write commands.

mario.plugins.write.write_csv_dicts(rows, header, dialect)

Write iterable of dicts to csv.

Return type

str

mario.plugins.write.write_csv_tuples(rows, dialect)

Write iterable of tuples to csv.

Return type

str

mario.plugins.write.write_yaml(data)

Write data to yaml string.

Return type

str

Module contents