259 lines
6.3 KiB
Markdown
259 lines
6.3 KiB
Markdown
# Shlax: Pythonic automation tool
|
|
|
|
Shlax is a Python framework for system automation, initially with the purpose
|
|
of replacing docker, docker-compose and ansible with a single tool with the
|
|
purpose of code-reuse made possible by target abstraction.
|
|
|
|
The pattern resolves around two moving parts: Actions and Targets.
|
|
|
|
## Action
|
|
|
|
An action is a function that takes a target argument, it may execute nested
|
|
actions by passing over the target argument which collects the results.
|
|
|
|
Example:
|
|
|
|
```python
|
|
async def hello_world(target):
|
|
"""Bunch of silly commands to demonstrate action programming."""
|
|
await target.mkdir('foo')
|
|
python = await target.which('python3', 'python')
|
|
await target.exec(f'{python} --version > foo/test')
|
|
version = target.exec('cat foo/test').output
|
|
print('version')
|
|
```
|
|
|
|
### Recursion
|
|
|
|
An action may call other actions recursively. There are two ways:
|
|
|
|
```python
|
|
async def something(target):
|
|
# just run the other action code
|
|
hello_world(target)
|
|
|
|
# or delegate the call to target
|
|
target(hello_world)
|
|
```
|
|
|
|
In the first case, the resulting count of ran actions will remain 1:
|
|
"something" action.
|
|
|
|
In the second case, the resulting count of ran actions will be 2: "something"
|
|
and "hello_world".
|
|
|
|
### Callable classes
|
|
|
|
Actually in practice, Actions are basic callable Python classes, here's a basic
|
|
example to run a command:
|
|
|
|
```python
|
|
class Run:
|
|
def __init__(self, cmd):
|
|
self.cmd = cmd
|
|
|
|
async def __call__(self, target):
|
|
return await target.exec(self.cmd)
|
|
```
|
|
|
|
This allows to create callable objects which may be called just like functions
|
|
and as such be appropriate actions, instead of:
|
|
|
|
```python
|
|
async def one(target):
|
|
target.exec('one')
|
|
|
|
async def two(target):
|
|
target.exec('two')
|
|
```
|
|
|
|
You can do:
|
|
|
|
```python
|
|
one = Run('one')
|
|
two = Run('two')
|
|
```
|
|
|
|
### Parallel execution
|
|
|
|
Actions may be executed in parallel with an action named ... Parallel. This
|
|
defines an action that will execute three actions in parallel:
|
|
|
|
```python
|
|
action = Parallel(
|
|
hello_world,
|
|
something,
|
|
Run('echo hi'),
|
|
)
|
|
```
|
|
|
|
In this case, all actions must succeed for the parallel action to be considered
|
|
a success.
|
|
|
|
### Methods
|
|
|
|
An action may also be a method, as long as it just takes a target argument, for
|
|
example:
|
|
|
|
```python
|
|
class Thing:
|
|
def start(self, target):
|
|
"""Starts thing"""
|
|
|
|
def stop(self, target):
|
|
"""Stops thing"""
|
|
|
|
action = Thing().start
|
|
```
|
|
|
|
### Cleaning
|
|
|
|
If an action defines a `clean` method, it will always be called wether or not
|
|
the action succeeded. Example:
|
|
|
|
```python
|
|
class Thing:
|
|
def __call__(self, target):
|
|
"""Do some thing"""
|
|
|
|
def clean(self, target):
|
|
"""Clean-up target after __call__"""
|
|
```
|
|
|
|
### Colorful actions
|
|
|
|
If an action defines a `colorize` method, it will be called with the colorset
|
|
as argument for every output, this allows to code custom output rendering.
|
|
|
|
## Target
|
|
|
|
A Target is mainly an object providing an abstraction layer over the system we
|
|
want to automate with actions. It defines functions to execute a command, mount
|
|
a directory, copy a file, manage environment variables and so on.
|
|
|
|
### Pre-configuration
|
|
|
|
A Target can be pre-configured with a list of Actions in which case calling the
|
|
target without argument will execute its Actions until one fails by raising an
|
|
Exception:
|
|
|
|
```python
|
|
say_hello = Localhost(
|
|
hello_world,
|
|
Run('echo hi'),
|
|
)
|
|
await say_hello()
|
|
```
|
|
|
|
### Results
|
|
|
|
Every time a target execute an action, it will set the "status" attribute on it
|
|
to "success" or "failure", and add it to the "results" attribute:
|
|
|
|
```python
|
|
say_hello = Localhost(Run('echo hi'))
|
|
await say_hello()
|
|
say_hello.results # contains the action with status="success"
|
|
```
|
|
|
|
## Targets as Actions: the nesting story
|
|
|
|
We've seen that any callable taking a target argument is good to be considered
|
|
an action, and that targets are callables.
|
|
|
|
To make a Target runnable like any action, all we had to do is add the target
|
|
keyword argument to `Target.__call__`.
|
|
|
|
But `target()` fills `self.results`, so nested action results would not
|
|
propagate to the parent target.
|
|
|
|
That's why if Target receives a non-None target argument, it will has to set
|
|
`self.parent` with it.
|
|
|
|
This allows nested targets to traverse parents and get to the root Target
|
|
with `target.caller`, where it can then attach results to.
|
|
|
|
This opens the nice side effect that a target implementation may call the
|
|
parent target if any, you could write a Docker target as such:
|
|
|
|
```python
|
|
class Docker(Target):
|
|
def __init__(self, *actions, name):
|
|
self.name = name
|
|
super().__init__(*actions)
|
|
|
|
async def exec(self, *args):
|
|
return await self.parent.exec(*['docker', 'exec', self.name] + args)
|
|
```
|
|
|
|
This also means that you always need a parent with an exec implementation,
|
|
there are two:
|
|
|
|
- Localhost, executes on localhost
|
|
- Stub, for testing
|
|
|
|
The result of that design is that the following use cases are available:
|
|
|
|
```python
|
|
# This action installs my favorite package on any distro
|
|
action = Packages('python3')
|
|
|
|
# Run it right here: apt install python3
|
|
Localhost()(action)
|
|
|
|
# Or remotely: ssh yourhost apt install python3
|
|
Ssh(host='yourhost')(action)
|
|
|
|
# Let's make a container build receipe with that action
|
|
build = Buildah(package)
|
|
|
|
# Run it locally: buildah exec apt install python3
|
|
Localhost()(build)
|
|
|
|
# Or on a server: ssh yourhost build exec apt install python3
|
|
Ssh(host='yourhost')(build)
|
|
|
|
# Or on a server behingh a bastion:
|
|
# ssh yourbastion ssh yourhost build exec apt install python3
|
|
Localhost()(Ssh(host='bastion')(Ssh(host='yourhost')(build))
|
|
|
|
# That's going to do the same
|
|
Localhost(Ssh(
|
|
Ssh(
|
|
build,
|
|
host='yourhost'
|
|
),
|
|
host='bastion'
|
|
))()
|
|
```
|
|
|
|
## CLI
|
|
|
|
You can execute Shlax actions directly on the command line with the `shlax` CLI
|
|
command.
|
|
|
|
For your own Shlaxfiles, you can build your CLI with your favorite CLI
|
|
framework. If you decide to use `cli2`, then Shlax provides a thin layer on top
|
|
of it: Group and Command objects made for Shlax objects.
|
|
|
|
For example:
|
|
|
|
```python
|
|
yourcontainer = Container(
|
|
build=Buildah(
|
|
User('app', '/app', 1000),
|
|
Packages('python', 'unzip', 'findutils'),
|
|
Copy('setup.py', 'yourdir', '/app'),
|
|
base='archlinux',
|
|
commit='yourimage',
|
|
),
|
|
)
|
|
|
|
|
|
if __name__ == '__main__':
|
|
print(Group(doc=__doc__).load(yourcontainer).entry_point())
|
|
```
|
|
|
|
The above will execute a cli2 command with each method of yourcontainer as a
|
|
sub-command.
|