Command reference¶
mario¶
Mario: Python pipelines for your shell.
mario [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
Options
-
--max-concurrent
<max_concurrent>
¶
-
--exec-before
<exec_before>
¶ Python source code to be executed before any stage.
-
--base-exec-before
<base_exec_before>
¶ Python source code to be executed before any stage; typically set in the user config file. Combined with –exec-before value.
-
--version
¶
Show the version and exit.
Traversals¶
apply¶
Apply code to the iterable of items.
The code should take an iterable and it will be called with the input items. The items iterable will be converted to a list before the code is called, so it doesn’t work well on very large streams.
For example,
$ mario map int apply sum <<EOF
10
20
30
EOF
60
mario apply [OPTIONS] CODE
Options
-
--autocall
,
--no-autocall
¶
Automatically call the function if “x” does not appear in the expression. Allows
map len
instead ofmap len(x)
.
-
--exec-before
<exec_before>
¶ Execute code in the function’s global namespace.
Arguments
-
CODE
¶
Required argument
chain¶
Concatenate a sequence of input iterables together into one long iterable.
Converts an iterable of iterables of items into an iterable of items, like itertools.chain.from_iterable.
For example,
$ mario eval '[[1,2]]'
[[1, 2]]
$ mario eval '[[1, 2]]' chain
[1, 2]
mario chain [OPTIONS]
eval¶
Evaluate a Python expression.
No input items are used.
For example,
$ mario eval 1+1
2
mario eval [OPTIONS] CODE
Options
-
--autocall
,
--no-autocall
¶
Automatically call the function if “x” does not appear in the expression. Allows
map len
instead ofmap len(x)
.
-
--exec-before
<exec_before>
¶ Execute code in the function’s global namespace.
Arguments
-
CODE
¶
Required argument
filter¶
Keep input items that satisfy a condition.
Order of input items is retained in the output.
For example,
$ mario filter 'x > "c"' <<EOF
a
b
c
d
e
f
EOF
d
e
f
mario filter [OPTIONS] CODE
Options
-
--autocall
,
--no-autocall
¶
Automatically call the function if “x” does not appear in the expression. Allows
map len
instead ofmap len(x)
.
-
--exec-before
<exec_before>
¶ Execute code in the function’s global namespace.
Arguments
-
CODE
¶
Required argument
map¶
Run code on each input item.
Each item is handled in the order it was received, and also output in the
same order. For less strict ordering and asynchronous execution, see
async-map
and async-map-unordered
.
For example,
$ mario map 'x*2' <<EOF
a
b
c
EOF
aa
bb
cc
mario map [OPTIONS] CODE
Options
-
--autocall
,
--no-autocall
¶
Automatically call the function if “x” does not appear in the expression. Allows
map len
instead ofmap len(x)
.
-
--exec-before
<exec_before>
¶ Execute code in the function’s global namespace.
Arguments
-
CODE
¶
Required argument
reduce¶
Reduce input items with code that takes two arguments, similar to functools.reduce
.
For example,
$ mario reduce map int operator.mul <<EOF
1
2
3
4
5
EOF
120
mario reduce [OPTIONS] FUNCTION_NAME
Options
-
--exec-before
<exec_before>
¶ Execute code in the function’s global namespace.
Arguments
-
FUNCTION_NAME
¶
Required argument
Commands for calling code on data.
Async traversals¶
async-filter¶
Keep input items that satisfy an asynchronous condition.
For example,
$ mario async-filter '(await asks.get(x)).json()["url"].endswith(("1", "3"))' <<EOF
http://httpbin.org/delay/5
http://httpbin.org/delay/1
http://httpbin.org/delay/2
http://httpbin.org/delay/3
http://httpbin.org/delay/4
EOF
http://httpbin.org/delay/1
http://httpbin.org/delay/3
mario async-filter [OPTIONS] CODE
Options
-
--autocall
,
--no-autocall
¶
Automatically call the function if “x” does not appear in the expression. Allows
map len
instead ofmap len(x)
.
-
--exec-before
<exec_before>
¶ Execute code in the function’s global namespace.
Arguments
-
CODE
¶
Required argument
async-map¶
Run code on each input item asynchronously.
The order of inputs is retained in the outputs. However, the order of inputs
does not determine the order in which each input is handled, only the order
in which its result is emitted. To keep the order in which each input is
handled, use the synchronous version, map
.
In this example, we make requests that have a server-side delay of specified length. The input order is retained in the output by holding each item until its precedents are ready.
$ mario async-map 'await asks.get ! x.json()["url"]' <<EOF
http://httpbin.org/delay/5
http://httpbin.org/delay/1
http://httpbin.org/delay/2
http://httpbin.org/delay/3
http://httpbin.org/delay/4
EOF
https://httpbin.org/delay/5
https://httpbin.org/delay/1
https://httpbin.org/delay/2
https://httpbin.org/delay/3
https://httpbin.org/delay/4
mario async-map [OPTIONS] CODE
Options
-
--autocall
,
--no-autocall
¶
Automatically call the function if “x” does not appear in the expression. Allows
map len
instead ofmap len(x)
.
-
--exec-before
<exec_before>
¶ Execute code in the function’s global namespace.
Arguments
-
CODE
¶
Required argument
async-map-unordered¶
Run code on each input item asynchronously, without retaining input order.
Each result is emitted in the order it becomes ready, regardless of input
order. Input order is also ignored when determining in which order to
start handling each item. Results start emitting as soon as the first one
is ready. It also saves memory because it doesn’t require accumulating
results while waiting for previous items to become ready. For stricter
ordering, see map
or async_map
.
In this example, we make requests that have a server-side delay of specified length. The input order is lost but the results appear immediately as they are ready (the delay length determines the output order):
$ mario async-map-unordered 'await asks.get ! x.json()["url"]' <<EOF
http://httpbin.org/delay/5
http://httpbin.org/delay/1
http://httpbin.org/delay/2
http://httpbin.org/delay/3
http://httpbin.org/delay/4
EOF
https://httpbin.org/delay/1
https://httpbin.org/delay/2
https://httpbin.org/delay/3
https://httpbin.org/delay/4
https://httpbin.org/delay/5
mario async-map-unordered [OPTIONS] CODE
Options
-
--autocall
,
--no-autocall
¶
Automatically call the function if “x” does not appear in the expression. Allows
map len
instead ofmap len(x)
.
-
--exec-before
<exec_before>
¶ Execute code in the function’s global namespace.
Arguments
-
CODE
¶
Required argument
Commands for asynchronously calling code on data.
Read¶
read-bytes¶
Read input as a block of bytes, joining lines with a line separator.
For example,
$ mario read-bytes <<EOF
Lorem ipsum dolor sit amet,
consectetur adipiscing elit,
EOF
b'Lorem ipsum dolor sit amet,\nconsectetur adipiscing elit,'
$ mario read-bytes map len <<EOF
Lorem ipsum dolor sit amet,
consectetur adipiscing elit,
EOF
56
mario read-bytes [OPTIONS]
Options
-
--sep
<sep>
¶ Separator to join input lines with
read-csv-dicts¶
Read a csv file into Python dicts. Given a csv like this:
name,age
Alice,21
Bob,22
try:
$ mario read-csv-dicts <<EOF
name,age
Alice,21
Bob,22
EOF
{'name': 'Alice', 'age': '21'}
{'name': 'Bob', 'age': '22'}
mario read-csv-dicts [OPTIONS]
Options
-
--dialect
<dialect>
¶ CSV dialect (See https://docs.python.org/3/library/csv.html)
- Options
excel|excel-tab|unix
read-csv-tuples¶
Read a csv file into Python tuples. Given a csv like this:
name,age
Alice,21
Bob,22
Carol,23
try:
$ mario read-csv-tuples <<EOF
Alice,21
Bob,22
Carol,23
EOF
('Alice', '21')
('Bob', '22')
('Carol', '23')
mario read-csv-tuples [OPTIONS]
Options
-
--dialect
<dialect>
¶ CSV dialect (See https://docs.python.org/3/library/csv.html)
- Options
excel|excel-tab|unix
read-json¶
Read a single json string into a Python object.
For example,
$ mario read-json <<EOF
[
{"name": "Alice", "age": 21},
{"name": "Bob", "age": 22}
]
EOF
[{'name': 'Alice', 'age': 21}, {'name': 'Bob', 'age': 22}]
mario read-json [OPTIONS]
read-json-array¶
Read a single json string into a Python object.
For example,
$ mario read-json-array <<EOF
[
{"name": "Alice", "age": 21},
{"name": "Bob", "age": 22}
]
EOF
{'name': 'Alice', 'age': 21}
{'name': 'Bob', 'age': 22}
$ mario read-json-array map 'x["age"]' <<EOF
[
{"name": "Alice", "age": 21},
{"name": "Bob", "age": 22}
]
EOF
21
22
mario read-json-array [OPTIONS]
read-jsonl¶
Read a sequence of json entities into Python objects.
For example,
$ mario read-jsonl <<EOF
{"a":1, "b":2}
{"a": 5, "b":9}
EOF
{'a': 1, 'b': 2}
{'a': 5, 'b': 9}
mario read-jsonl [OPTIONS]
read-text¶
Read input lines as a block of text, joining lines with a line separator.
For example,
$ mario read-text <<EOF
Lorem ipsum dolor sit amet,
consectetur adipiscing elit,
EOF
Lorem ipsum dolor sit amet,
consectetur adipiscing elit,
$ mario read-text map len <<EOF
Lorem ipsum dolor sit amet,
consectetur adipiscing elit,
EOF
56
mario read-text [OPTIONS]
Options
-
--sep
<sep>
¶ Separator to join input lines with
read-toml¶
Read a toml document into a Python object.
For example,
$ mario read-toml <<EOF
[[persons]]
name = "Alice"
age = 21
[[persons]]
name = "Bob"
age = 22
EOF
{'persons': [{'name': 'Alice', 'age': 21}, {'name': 'Bob', 'age': 22}]}
mario read-toml [OPTIONS]
read-xml¶
Read xml into a Python object.
For example,
$ mario read-xml <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<message>
<warning>
Hello World
</warning>
</message>
EOF
{'message': {'warning': 'Hello World'}}
mario read-xml [OPTIONS]
Options
-
--process-namespaces
¶
read-yaml¶
Read a yaml document into a Python object.
For example,
$ mario read-yaml <<EOF
- Cat: "foo"
- Dog: "bar"
- Goldfish: "baz"
EOF
[{'Cat': 'foo'}, {'Dog': 'bar'}, {'Goldfish': 'baz'}]
mario read-yaml [OPTIONS]
read-yaml-array¶
Read a yaml document into a Python object.
For example,
$ mario read-yaml-array <<EOF
- Cat: "foo"
- Dog: "bar"
- Goldfish: "baz"
EOF
{'Cat': 'foo'}
{'Dog': 'bar'}
{'Goldfish': 'baz'}
mario read-yaml-array [OPTIONS]
Write¶
write-csv-dicts¶
Write a list of dicts to csv.
For example,
$ mario read-json write-csv-dicts --no-header <<EOF
[
{
"name": "Alice",
"age": 21
},
{
"name": "Bob",
"age": 22
}
]
EOF
Alice,21
Bob,22
mario write-csv-dicts [OPTIONS]
Options
-
--header
,
--no-header
¶
Whether to write the dict keys as the first row
-
--dialect
<dialect>
¶ CSV dialect (See https://docs.python.org/3/library/csv.html)
- Options
excel|excel-tab|unix
write-csv-tuples¶
Write a list of tuples to csv.
For example,
$ mario read-json write-csv-tuples <<EOF
[
["name", "age"],
["Alice", 21],
["Bob", 22]
]
EOF
name,age
Alice,21
Bob,22
mario write-csv-tuples [OPTIONS]
Options
-
--dialect
<dialect>
¶ CSV dialect (See https://docs.python.org/3/library/csv.html)
- Options
excel|excel-tab|unix
write-json¶
Serialize each input item to its json representation.
For example,
$ mario eval "[1, 2, 'foo']" write-json --no-pretty
[1, 2, "foo"]
Use the --indent
option to set the indentation level:
$ mario read-toml write-json --pretty <<EOF
[[persons]]
name = "Alice"
age = 21
[[persons]]
name = "Bob"
age = 22
EOF
{
"persons": [
{
"name": "Alice",
"age": 21
},
{
"name": "Bob",
"age": 22
}
]
}
mario write-json [OPTIONS]
Options
-
--pretty
,
--no-pretty
¶
write-json-array¶
Write the input sequence into a json array.
$ mario read-json-array write-json-array map str.rstrip <<EOF
[
{
"name": "Alice",
"age": 21
},
{
"name": "Bob",
"age": 22
}
]
EOF
[
{
"name": "Alice",
"age": 21
},
{
"name": "Bob",
"age": 22
}
]
mario write-json-array [OPTIONS]
Options
-
--pretty
,
--no-pretty
¶
write-jsonl¶
Write a sequence to newline-separated json.
$ mario read-json write-jsonl <<EOF
[
{"name": "Alice", "age": 21},
{"name": "Bob", "age": 22}
]
EOF
{"name": "Alice", "age": 21}
{"name": "Bob", "age": 22}
mario write-jsonl [OPTIONS]
write-toml¶
Write each input item to its toml representation.
For example,
$ mario read-json write-toml map str.rstrip <<EOF
{
"persons": [
{
"name": "Alice",
"age": 21
},
{
"name": "Bob",
"age": 22
}
]
}
EOF
[[persons]]
name = "Alice"
age = 21
[[persons]]
name = "Bob"
age = 22
mario write-toml [OPTIONS]
write-xml¶
Write a mapping to xml string.
For example,
$ mario eval '{"foo": {"bar": 1}}' write-xml
<?xml version="1.0" encoding="utf-8"?>
<foo>
<bar>1</bar>
</foo>
mario write-xml [OPTIONS]
Options
-
--pretty
,
--no-pretty
¶
Pretty-print the output
write-yaml¶
Write a yaml document.
$ mario read-json write-yaml map str.rstrip <<EOF
[
{
"name": "Alice",
"age": 21
},
{
"name": "Bob",
"age": 22
}
]
EOF
- age: 21
name: Alice
- age: 22
name: Bob
mario write-yaml [OPTIONS]
meta¶
Commands about using mario.
mario meta [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
pip¶
Run pip
in the environment that mario is installed into.
Arguments are forwarded to pip
.
mario meta pip [OPTIONS] [PIP_ARGS]...
Arguments
-
PIP_ARGS
¶
Optional argument(s)
test¶
Run all declarative command tests from plugins and config.
Executes each test in the command.tests
field with pytest.
Default pytest args: -vvv --tb=short
mario meta test [OPTIONS] [PYTEST_ARGS]...
Arguments
-
PYTEST_ARGS
¶
Optional argument(s)
Commands about using mario.
mario meta [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
Mario: Python pipelines for your shell.
mario [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
Options
-
--max-concurrent
<max_concurrent>
¶
-
--exec-before
<exec_before>
¶ Python source code to be executed before any stage.
-
--base-exec-before
<base_exec_before>
¶ Python source code to be executed before any stage; typically set in the user config file. Combined with –exec-before value.
-
--version
¶
Show the version and exit.