Trying to get somewhere with traefik module

.gitlab-ci.yml

@@ -2,22 +2,14 @@ build:
   cache:
     key: cache
     paths: [.cache]
-  image: quay.io/buildah/stable
-  script:
-  - dnf install -y python3-pip
-  - pip3 install -U --user -e .[cli]
-  - CACHE_DIR=$(pwd)/.cache python3 ./shlaxfile.py build
+  image: yourlabs/shlax
+  script: pip install -U --user -e . && CACHE_DIR=$(pwd)/.cache ./shlaxfile.py -d
+    shlax build push
   stage: build
-
-test:
-  image: yourlabs/python
-  stage: build
-  script:
-  - pip install -U --user -e .[test]
-  - py.test -sv tests
-
 pypi:
   image: yourlabs/python
   only: [tags]
   script: pypi-release
   stage: deploy
+test: {image: yourlabs/python, script: 'pip install -U --user -e .[test] && py.test
+    -svv tests', stage: build}

README.md

@@ -1,258 +0,0 @@
-# Shlax: Pythonic automation tool
-
-Shlax is a Python framework for system automation, initially with the purpose
-of replacing docker, docker-compose and ansible with a single tool with the
-purpose of code-reuse made possible by target abstraction.
-
-The pattern resolves around two moving parts: Actions and Targets.
-
-## Action
-
-An action is a function that takes a target argument, it may execute nested
-actions by passing over the target argument which collects the results.
-
-Example:
-
-```python
-async def hello_world(target):
-    """Bunch of silly commands to demonstrate action programming."""
-    await target.mkdir('foo')
-    python = await target.which('python3', 'python')
-    await target.exec(f'{python} --version > foo/test')
-    version = target.exec('cat foo/test').output
-    print('version')
-```
-
-### Recursion
-
-An action may call other actions recursively. There are two ways:
-
-```python
-async def something(target):
-    # just run the other action code
-    hello_world(target)
-
-    # or delegate the call to target
-    target(hello_world)
-```
-
-In the first case, the resulting count of ran actions will remain 1:
-"something" action.
-
-In the second case, the resulting count of ran actions will be 2: "something"
-and "hello_world".
-
-### Callable classes
-
-Actually in practice, Actions are basic callable Python classes, here's a basic
-example to run a command:
-
-```python
-class Run:
-    def __init__(self, cmd):
-        self.cmd = cmd
-
-    async def __call__(self, target):
-        return await target.exec(self.cmd)
-```
-
-This allows to create callable objects which may be called just like functions
-and as such be appropriate actions, instead of:
-
-```python
-async def one(target):
-    target.exec('one')
-
-async def two(target):
-    target.exec('two')
-```
-
-You can do:
-
-```python
-one = Run('one')
-two = Run('two')
-```
-
-### Parallel execution
-
-Actions may be executed in parallel with an action named ... Parallel. This
-defines an action that will execute three actions in parallel:
-
-```python
-action = Parallel(
-    hello_world,
-    something,
-    Run('echo hi'),
-)
-```
-
-In this case, all actions must succeed for the parallel action to be considered
-a success.
-
-### Methods
-
-An action may also be a method, as long as it just takes a target argument, for
-example:
-
-```python
-class Thing:
-    def start(self, target):
-        """Starts thing"""
-
-    def stop(self, target):
-        """Stops thing"""
-
-action = Thing().start
-```
-
-### Cleaning
-
-If an action defines a `clean` method, it will always be called wether or not
-the action succeeded. Example:
-
-```python
-class Thing:
-    def __call__(self, target):
-        """Do some thing"""
-
-    def clean(self, target):
-        """Clean-up target after __call__"""
-```
-
-### Colorful actions
-
-If an action defines a `colorize` method, it will be called with the colorset
-as argument for every output, this allows to code custom output rendering.
-
-## Target
-
-A Target is mainly an object providing an abstraction layer over the system we
-want to automate with actions. It defines functions to execute a command, mount
-a directory, copy a file, manage environment variables and so on.
-
-### Pre-configuration
-
-A Target can be pre-configured with a list of Actions in which case calling the
-target without argument will execute its Actions until one fails by raising an
-Exception:
-
-```python
-say_hello = Localhost(
-    hello_world,
-    Run('echo hi'),
-)
-await say_hello()
-```
-
-### Results
-
-Every time a target execute an action, it will set the "status" attribute on it
-to "success" or "failure", and add it to the "results" attribute:
-
-```python
-say_hello = Localhost(Run('echo hi'))
-await say_hello()
-say_hello.results  # contains the action with status="success"
-```
-
-## Targets as Actions: the nesting story
-
-We've seen that any callable taking a target argument is good to be considered
-an action, and that targets are callables.
-
-To make a Target runnable like any action, all we had to do is add the target
-keyword argument to `Target.__call__`.
-
-But `target()` fills `self.results`, so nested action results would not
-propagate to the parent target.
-
-That's why if Target receives a non-None target argument, it will has to set
-`self.parent` with it.
-
-This allows nested targets to traverse parents and get to the root Target
-with `target.caller`, where it can then attach results to.
-
-This opens the nice side effect that a target implementation may call the
-parent target if any, you could write a Docker target as such:
-
-```python
-class Docker(Target):
-    def __init__(self, *actions, name):
-        self.name = name
-        super().__init__(*actions)
-
-    async def exec(self, *args):
-        return await self.parent.exec(*['docker', 'exec', self.name] + args)
-```
-
-This also means that you always need a parent with an exec implementation,
-there are two:
-
-- Localhost, executes on localhost
-- Stub, for testing
-
-The result of that design is that the following use cases are available:
-
-```python
-# This action installs my favorite package on any distro
-action = Packages('python3')
-
-# Run it right here: apt install python3
-Localhost()(action)
-
-# Or remotely: ssh yourhost apt install python3
-Ssh(host='yourhost')(action)
-
-# Let's make a container build receipe with that action
-build = Buildah(package)
-
-# Run it locally: buildah exec apt install python3
-Localhost()(build)
-
-# Or on a server: ssh yourhost build exec apt install python3
-Ssh(host='yourhost')(build)
-
-# Or on a server behingh a bastion:
-# ssh yourbastion ssh yourhost build exec apt install python3
-Localhost()(Ssh(host='bastion')(Ssh(host='yourhost')(build))
-
-# That's going to do the same
-Localhost(Ssh(
-    Ssh(
-        build,
-        host='yourhost'
-    ),
-    host='bastion'
-))()
-```
-
-## CLI
-
-You can execute Shlax actions directly on the command line with the `shlax` CLI
-command.
-
-For your own Shlaxfiles, you can build your CLI with your favorite CLI
-framework. If you decide to use `cli2`, then Shlax provides a thin layer on top
-of it: Group and Command objects made for Shlax objects.
-
-For example:
-
-```python
-yourcontainer = Container(
-    build=Buildah(
-        User('app', '/app', 1000),
-        Packages('python', 'unzip', 'findutils'),
-        Copy('setup.py', 'yourdir', '/app'),
-        base='archlinux',
-        commit='yourimage',
-    ),
-)
-
-
-if __name__ == '__main__':
-    print(Group(doc=__doc__).load(yourcontainer).entry_point())
-```
-
-The above will execute a cli2 command with each method of yourcontainer as a
-sub-command.

setup.py

@@ -5,9 +5,10 @@ setup(
     name='shlax',
     versioning='dev',
     setup_requires='setupmeta',
+    install_requires=['cli2>=1.1.6'],
     extras_require=dict(
-        cli=[
-            'cli2>=2.2.2',
+        full=[
+            'pyyaml',
         ],
         test=[
             'pytest',
@@ -24,7 +25,7 @@ setup(
     python_requires='>=3',
     entry_points={
         'console_scripts': [
-            'shlax = shlax.cli:cli.entry_point',
+            'shlax = shlax.cli:cli',
         ],
     },
 )

shlax/__init__.py

@@ -0,0 +1,7 @@
+from .actions import *
+from .image import Image
+from .strategies import *
+from .output import Output
+from .proc import Proc
+from .targets import *
+from .shlaxfile import Shlaxfile

shlax/actions/__init__.py

@@ -0,0 +1,7 @@
+from .copy import Copy
+from .packages import Packages  # noqa
+from .base import Action  # noqa
+from .htpasswd import Htpasswd
+from .run import Run  # noqa
+from .pip import Pip
+from .service import Service

shlax/actions/base.py

@@ -1,2 +1,225 @@
+from copy import deepcopy
+import functools
+import inspect
+import importlib
+import sys
+
+from ..output import Output
+from ..exceptions import WrongResult
+
+
 class Action:
-    pass
+    parent = None
+    contextualize = []
+    regexps = {
+        r'([\w]+):': '{cyan}\\1{gray}:{reset}',
+        r'(^|\n)( *)\- ': '\\1\\2{red}-{reset} ',
+    }
+    options = dict(
+        debug=dict(
+            alias='d',
+            default='visit',
+            help='''
+            Display debug output. Supports values (combinable): cmd,out,visit
+            '''.strip(),
+            immediate=True,
+        ),
+    )
+
+    def __init__(self, *args, doc=None, **kwargs):
+        self.args = args
+        self.kwargs = kwargs
+        self.call_args = []
+        self.call_kwargs = {}
+        self._doc = doc
+        self.menu = {
+            name: value
+            for name, value in kwargs.items()
+            if isinstance(value, Action)
+        }
+
+    @property
+    def context(self):
+        if not self.parent:
+            if '_context' not in self.__dict__:
+                self._context = dict()
+            return self._context
+        else:
+            return self.parent.context
+
+    def actions_filter(self, results, f=None, **filters):
+        if f:
+            def ff(a):
+                try:
+                    return f(a)
+                except:
+                    return False
+            results = [*filter(ff, results)]
+
+        for k, v in filters.items():
+            if k == 'type':
+                results = [*filter(
+                    lambda s: type(s).__name__.lower() == str(v).lower(),
+                    results
+                )]
+            else:
+                results = [*filter(
+                    lambda s: getattr(s, k, None) == v,
+                    results
+                )]
+
+        return results
+
+
+    def sibblings(self, f=None, **filters):
+        if not self.parent:
+            return []
+        return self.actions_filter(
+            [a for a in self.parent.actions if a is not self],
+            f, **filters
+        )
+
+    def parents(self, f=None, **filters):
+        if self.parent:
+            return self.actions_filter(
+                [self.parent] + self.parent.parents(),
+                f, **filters
+            )
+        return []
+
+    def children(self, f=None, **filters):
+        children = []
+        def add(parent):
+            if parent != self:
+                children.append(parent)
+            if 'actions' not in parent.__dict__:
+                return
+
+            for action in parent.actions:
+                add(action)
+        add(self)
+        return self.actions_filter(children, f, **filters)
+
+    def __getattr__(self, name):
+        for a in self.parents() + self.sibblings() + self.children():
+            if name in a.contextualize:
+                return getattr(a, name)
+        raise AttributeError(f'{type(self).__name__} has no {name}')
+
+    async def call(self, *args, **kwargs):
+        print(f'{self}.call(*args, **kwargs) not implemented')
+        sys.exit(1)
+
+    def output_factory(self, *args, **kwargs):
+        kwargs.setdefault('regexps', self.regexps)
+        return Output(**kwargs)
+
+    async def __call__(self, *args, **kwargs):
+        self.call_args = list(self.call_args) + list(args)
+        self.call_kwargs.update(kwargs)
+        self.output = self.output_factory(*args, **kwargs)
+        self.output_start()
+        self.status = 'running'
+        try:
+            result = await self.call(*args, **kwargs)
+        except Exception as e:
+            self.output_fail(e)
+            self.status = 'fail'
+            proc = getattr(e, 'proc', None)
+            if proc:
+                result = proc.rc
+            else:
+                raise
+        else:
+            self.output_success()
+            if self.status == 'running':
+                self.status = 'success'
+        finally:
+            clean = getattr(self, 'clean', None)
+            if clean:
+                self.output.clean(self)
+                await clean(*args, **kwargs)
+        return result
+
+    def output_start(self):
+        if self.kwargs.get('quiet', False):
+            return
+        self.output.start(self)
+
+    def output_fail(self, exception=None):
+        if self.kwargs.get('quiet', False):
+            return
+        self.output.fail(self, exception)
+
+    def output_success(self):
+        if self.kwargs.get('quiet', False):
+            return
+        self.output.success(self)
+
+    def __repr__(self):
+        return ' '.join([type(self).__name__] + list(self.args) + [
+            f'{k}={v}'
+            for k, v in self.kwargs.items()
+        ])
+
+    def colorized(self):
+        return ' '.join([
+            self.output.colors['pink1']
+            + type(self).__name__
+            + self.output.colors['yellow']
+        ] + list(self.args) + [
+            f'{self.output.colors["blue"]}{k}{self.output.colors["gray"]}={self.output.colors["green2"]}{v}'
+            for k, v in self.kwargs_output().items()
+        ] + [self.output.colors['reset']])
+
+    def callable(self):
+        from ..targets import Localhost
+        async def cb(*a, **k):
+            from shlax.cli import cli
+            script = Localhost(self, quiet=True)
+            result = await script(*a, **k)
+
+            success = functools.reduce(
+                lambda a, b: a + b,
+                [1 for c in script.children() if c.status == 'success'] or [0])
+            if success:
+                script.output.success(f'{success} PASS')
+
+            failures = functools.reduce(
+                lambda a, b: a + b,
+                [1 for c in script.children() if c.status == 'fail'] or [0])
+            if failures:
+                script.output.fail(f'{failures} FAIL')
+            cli.exit_code = failures
+            return result
+        return cb
+
+    def kwargs_output(self):
+        return self.kwargs
+
+    def action(self, action, *args, **kwargs):
+        if isinstance(action, str):
+            import cli2
+            a = cli2.Callable.factory(action).target
+            if not a:
+                a = cli2.Callable.factory(
+                    '.'.join(['shlax', action])
+                ).target
+            if a:
+                action = a
+
+        p = action(*args, **kwargs)
+        for parent in self.parents():
+            if hasattr(parent, 'actions'):
+                break
+        p.parent = parent
+        if 'actions' not in self.__dict__:
+            # "mutate" to Strategy
+            from ..strategies.script import Actions
+            self.actions = Actions(self, [p])
+        return p
+
+    def bind(self, *args):
+        clone = deepcopy(self)
+        clone.call_args = args
+        return clone

shlax/actions/copy.py

@@ -1,56 +1,6 @@
-import asyncio
-import binascii
-import os
+from .base import Action
 
 
-class Copy:
-    def __init__(self, *args):
-        self.src = args[:-1]
-        self.dst = args[-1]
-
-    def listfiles(self):
-        if getattr(self, '_listfiles', None):
-            return self._listfiles
-
-        result = []
-        for src in self.src:
-            if os.path.isfile(src):
-                result.append(src)
-                continue
-
-            for root, dirs, files in os.walk(src):
-                if '__pycache__' in root:
-                    continue
-                result += [
-                    os.path.join(root, f)
-                    for f in files
-                    if not f.endswith('.pyc')
-                ]
-        self._listfiles = result
-        return result
-
-    async def __call__(self, target):
-        await target.mkdir(self.dst)
-
-        for path in self.listfiles():
-            if os.path.isdir(path):
-                await target.mkdir(os.path.join(self.dst, path))
-            elif '/' in path:
-                dirname = os.path.join(
-                    self.dst,
-                    '/'.join(path.split('/')[:-1])
-                )
-                await target.mkdir(dirname)
-                await target.copy(path, dirname)
-            else:
-                await target.copy(path, self.dst)
-
-    def __str__(self):
-        return f'Copy({", ".join(self.src)}, {self.dst})'
-
-    async def cachekey(self):
-        async def chksum(path):
-            with open(path, 'rb') as f:
-                return (path, str(binascii.crc32(f.read())))
-        results = await asyncio.gather(*[chksum(f) for f in self.listfiles()])
-        return {path: chks for path, chks in results}
+class Copy(Action):
+    async def call(self, *args, **kwargs):
+        await self.copy(*self.args)

shlax/actions/htpasswd.py

@@ -0,0 +1,29 @@
+import hashlib
+import secrets
+import string
+
+from .base import Action
+
+
+class Htpasswd(Action):
+    def __init__(self, path, user, *args, **kwargs):
+        self.path = path
+        self.user = user
+        super().__init__(*args, **kwargs)
+
+    async def call(self, *args, **kwargs):
+        found = False
+        htpasswd = await self.exec('cat', self.path, raises=False)
+        if htpasswd.rc == 0:
+            for line in htpasswd.out.split('\n'):
+                if line.startswith(self.user + ':'):
+                    found = True
+                    break
+
+        if not found:
+            self.password = ''.join(secrets.choice(
+                string.ascii_letters + string.digits
+            ) for i in range(20))
+            hashed = hashlib.sha1(self.password.encode('utf8'))
+            line = f'{self.user}:\\$sha1\\${hashed.hexdigest()}'
+            await self.exec(f'echo {line} >> {self.path}')

shlax/actions/packages.py

@@ -7,16 +7,19 @@ import os
 import subprocess
 from textwrap import dedent
 
+from .base import Action
 
-class Packages:
+
+class Packages(Action):
     """
-    Package manager abstract layer with caching.
+    The Packages visitor wraps around the container's package manager.
 
     It's a central piece of the build process, and does iterate over other
     container visitors in order to pick up packages. For example, the Pip
     visitor will declare ``self.packages = dict(apt=['python3-pip'])``, and the
     Packages visitor will pick it up.
     """
+    contextualize = ['mgr']
     regexps = {
         #r'Installing ([\w\d-]+)': '{cyan}\\1',
         r'Installing': '{cyan}lol',
@@ -52,12 +55,12 @@ class Packages:
 
     installed = []
 
-    def __init__(self, *packages, upgrade=True):
+    def __init__(self, *packages, **kwargs):
         self.packages = []
-        self.upgrade = upgrade
         for package in packages:
             line = dedent(package).strip().replace('\n', ' ')
             self.packages += line.split(' ')
+        super().__init__(*packages, **kwargs)
 
     @property
     def cache_root(self):
@@ -66,9 +69,9 @@ class Packages:
         else:
             return os.path.join(os.getenv('HOME'), '.cache')
 
-    async def update(self, target):
+    async def update(self):
         # run pkgmgr_setup functions ie. apk_setup
-        cachedir = await getattr(self, self.mgr + '_setup')(target)
+        cachedir = await getattr(self, self.mgr + '_setup')()
 
         lastupdate = None
         if os.path.exists(cachedir + '/lastupdate'):
@@ -92,7 +95,7 @@ class Packages:
                     f.write(str(os.getpid()))
 
                 try:
-                    await target.rexec(self.cmds['update'])
+                    await self.rexec(self.cmds['update'])
                 finally:
                     os.unlink(lockfile)
 
@@ -100,15 +103,15 @@ class Packages:
                     f.write(str(now))
             else:
                 while os.path.exists(lockfile):
-                    print(f'{self.target} | Waiting for {lockfile} ...')
+                    print(f'{self.container.name} | Waiting for update ...')
                     await asyncio.sleep(1)
 
-    async def __call__(self, target):
-        cached = getattr(target, 'pkgmgr', None)
+    async def call(self, *args, **kwargs):
+        cached = getattr(self, '_pagkages_mgr', None)
         if cached:
             self.mgr = cached
         else:
-            mgr = await target.which(*self.mgrs.keys())
+            mgr = await self.which(*self.mgrs.keys())
             if mgr:
                 self.mgr = mgr[0].split('/')[-1]
 
@@ -116,9 +119,11 @@ class Packages:
             raise Exception('Packages does not yet support this distro')
 
         self.cmds = self.mgrs[self.mgr]
-        await self.update(target)
-        if self.upgrade:
-            await target.rexec(self.cmds['upgrade'])
+        if not getattr(self, '_packages_upgraded', None):
+            await self.update()
+            if self.kwargs.get('upgrade', True):
+                await self.rexec(self.cmds['upgrade'])
+            self._packages_upgraded = True
 
         packages = []
         for package in self.packages:
@@ -131,22 +136,22 @@ class Packages:
             else:
                 packages.append(package)
 
-        await target.rexec(*self.cmds['install'].split(' ') + packages)
+        await self.rexec(*self.cmds['install'].split(' ') + packages)
 
-    async def apk_setup(self, target):
+    async def apk_setup(self):
         cachedir = os.path.join(self.cache_root, self.mgr)
-        await target.mount(cachedir, '/var/cache/apk')
+        await self.mount(cachedir, '/var/cache/apk')
         # special step to enable apk cache
-        await target.rexec('ln -sf /var/cache/apk /etc/apk/cache')
+        await self.rexec('ln -sf /var/cache/apk /etc/apk/cache')
         return cachedir
 
-    async def dnf_setup(self, target):
+    async def dnf_setup(self):
         cachedir = os.path.join(self.cache_root, self.mgr)
-        await target.mount(cachedir, f'/var/cache/{self.mgr}')
-        await target.rexec('echo keepcache=True >> /etc/dnf/dnf.conf')
+        await self.mount(cachedir, f'/var/cache/{self.mgr}')
+        await self.rexec('echo keepcache=True >> /etc/dnf/dnf.conf')
         return cachedir
 
-    async def apt_setup(self, target):
+    async def apt_setup(self):
         codename = (await self.rexec(
             f'source {self.mnt}/etc/os-release; echo $VERSION_CODENAME'
         )).out
@@ -158,8 +163,8 @@ class Packages:
         await self.mount(cache_lists, f'/var/lib/apt/lists')
         return cachedir
 
-    async def pacman_setup(self, target):
+    async def pacman_setup(self):
         return self.cache_root + '/pacman'
 
-    def __str__(self):
-        return f'Packages({self.packages}, upgrade={self.upgrade})'
+    def __repr__(self):
+        return f'Packages({self.packages})'

shlax/actions/parallel.py

@@ -1,11 +0,0 @@
-import asyncio
-
-
-class Parallel:
-    def __init__(self, *actions):
-        self.actions = actions
-
-    async def __call__(self, target):
-        return await asyncio.gather(*[
-            target(action) for action in self.actions
-        ])

shlax/actions/pip.py

@@ -1,69 +1,57 @@
 from glob import glob
 import os
-from urllib import request
 
 from .base import Action
 
 
 class Pip(Action):
-    """Pip abstraction layer."""
-    def __init__(self, *pip_packages):
-        self.pip_packages = pip_packages
+    def __init__(self, *pip_packages, pip=None, requirements=None):
+        self.requirements = requirements
+        super().__init__(*pip_packages, pip=pip, requirements=requirements)
 
-    async def __call__(self, target):
-        # ensure python presence
-        results = await target.which('python3', 'python')
-        if results:
-            python = results[0]
+    async def call(self, *args, **kwargs):
+        pip = self.kwargs.get('pip', None)
+        if not pip:
+            pip = await self.which('pip3', 'pip', 'pip2')
+            if pip:
+                pip = pip[0]
             else:
-            raise Exception('Could not find pip nor python')
-
-        # ensure pip module presence
-        result = await target.exec(python, '-m', 'pip', raises=False)
-        if result.rc != 0:
-            if not os.path.exists('get-pip.py'):
-                req = request.urlopen(
-                    'https://bootstrap.pypa.io/get-pip.py'
+                from .packages import Packages
+                action = self.action(
+                    Packages,
+                    'python3,apk', 'python3-pip,apt',
+                    args=args, kwargs=kwargs
                 )
-                content = req.read()
-                with open('get-pip.py', 'wb+') as f:
-                    f.write(content)
+                await action(*args, **kwargs)
+                pip = await self.which('pip3', 'pip', 'pip2')
+                if not pip:
+                    raise Exception('Could not install a pip command')
+                else:
+                    pip = pip[0]
 
-            await target.copy('get-pip.py', '.')
-            await target.exec(python, 'get-pip.py')
-
-        # choose a cache directory
         if 'CACHE_DIR' in os.environ:
             cache = os.path.join(os.getenv('CACHE_DIR'), 'pip')
         else:
             cache = os.path.join(os.getenv('HOME'), '.cache', 'pip')
 
-        # and mount it
-        if getattr(target, 'mount', None):
+        if getattr(self, 'mount', None):
             # we are in a target which shares a mount command
-            await target.mount(cache, '/root/.cache/pip')
+            await self.mount(cache, '/root/.cache/pip')
+        await self.exec(f'{pip} install --upgrade pip')
 
-        source = []
-        nonsource = []
-        for package in self.pip_packages:
-            if os.path.exists(package):
-                source.append(package)
-            else:
-                nonsource.append(package)
-
-        if nonsource:
-            await target.exec(
-                python, '-m', 'pip',
-                'install', '--upgrade',
-                *nonsource
-            )
+        # https://github.com/pypa/pip/issues/5599
+        if 'pip' not in self.kwargs:
+            pip = 'python3 -m pip'
 
+        source = [p for p in self.args if p.startswith('/') or p.startswith('.')]
         if source:
-            await target.exec(
-                python, '-m', 'pip',
-                'install', '--upgrade', '--editable',
-                *source
+            await self.exec(
+                f'{pip} install --upgrade --editable {" ".join(source)}'
             )
 
-    def __str__(self):
-        return f'Pip({", ".join(self.pip_packages)})'
+        nonsource = [p for p in self.args if not p.startswith('/')]
+        if nonsource:
+            await self.exec(f'{pip} install --upgrade {" ".join(nonsource)}')
+
+        if self.requirements:
+            await self.exec(f'{pip} install --upgrade -r {self.requirements}')

shlax/actions/run.py

@@ -1,11 +1,18 @@
+from ..targets.buildah import Buildah
+from ..targets.docker import Docker
+
+from .base import Action
 
 
-class Run:
-    def __init__(self, cmd):
-        self.cmd = cmd
+class Run(Action):
+    async def call(self, *args, **kwargs):
+        image = self.kwargs.get('image', None)
+        if not image:
+            return await self.exec(*self.args, **self.kwargs)
+        if isinstance(image, Buildah):
+            breakpoint()
+            result = await self.action(image, *args, **kwargs)
 
-    async def __call__(self, target):
-        self.proc = await target.exec(self.cmd)
-
-    def __str__(self):
-        return f'Run({self.cmd})'
+        return await Docker(
+            image=image,
+        ).exec(*args, **kwargs)

shlax/actions/service.py

@@ -0,0 +1,16 @@
+import asyncio
+
+from .base import Action
+
+
+class Service(Action):
+    def __init__(self, *names, state=None):
+        self.state = state or 'started'
+        self.names = names
+        super().__init__()
+
+    async def call(self, *args, **kwargs):
+        return asyncio.gather(*[
+            self.exec('systemctl', 'start', name, user='root')
+            for name in self.names
+        ])

shlax/actions/user.py

@@ -1,42 +0,0 @@
-import os
-import re
-
-from .packages import Packages
-
-
-class User:
-    """
-    Create a user.
-
-    Example:
-
-        User('app', '/app', getenv('_CONTAINERS_ROOTLESS_UID', 1000)),
-
-    _CONTAINERS_ROOTLESS_UID allows to get your UID during build, which happens
-    in buildah unshare.
-    """
-    def __init__(self, username, home, uid):
-        self.username = username
-        self.home = home
-        self.uid = uid
-
-    def __str__(self):
-        return f'User({self.username}, {self.home}, {self.uid})'
-
-    async def __call__(self, target):
-        result = await target.rexec('id', self.uid)
-        if result.rc == 0:
-            old = re.match('.*\(([^)]*)\).*', result.out).group(1)
-            await target.rexec(
-                'usermod',
-                '-d', self.home,
-                '-l', self.username,
-                old
-            )
-        else:
-            await target.rexec(
-                'useradd',
-                '-d', self.home,
-                '-u', self.uid,
-                self.username
-            )

shlax/cli.py

@@ -1,80 +1,86 @@
-"""
-Shlax executes mostly in 3 ways:
-- Execute actions on targets with the command line
-- With your shlaxfile as first argument: offer defined Actions
-- With the name of a module in shlax.repo: a community maintained shlaxfile
-"""
-import ast
+'''
+shlax is a micro-framework to orchestrate commands.
+
+  shlax yourfile.py: to list actions you have declared.
+  shlax yourfile.py <action>: to execute a given action
+  #!/usr/bin/env shlax: when making yourfile.py an executable.
+'''
+
 import asyncio
 import cli2
-import glob
+import copy
 import inspect
 import importlib
+import glob
 import os
 import sys
 
-
-class Group(cli2.Group):
-    def __init__(self, *args, **kwargs):
-        super().__init__(*args, **kwargs)
-        self.cmdclass = Command
+from .exceptions import *
+from .shlaxfile import Shlaxfile
+from .targets import Localhost
 
 
-class Command(cli2.Command):
-    def call(self, *args, **kwargs):
-        return self.shlax_target(self.target)
+class ConsoleScript(cli2.ConsoleScript):
+    def __call__(self, *args, **kwargs):
+        self.shlaxfile = None
+        shlaxfile = sys.argv.pop(1) if len(sys.argv) > 1 else ''
+        if shlaxfile:
+            if not os.path.exists(shlaxfile):
+                try:    # missing shlaxfile, what are we gonna do !!
+                    mod = importlib.import_module('shlax.repo.' + shlaxfile)
+                except ImportError:
+                    print('Could not find ' + shlaxfile)
+                    self.exit_code = 1
+                    return
+                shlaxfile = mod.__file__
 
-    def __call__(self, *argv):
-        from shlax.targets.base import Target
-        self.shlax_target = Target()
-        result = super().__call__(*argv)
-        self.shlax_target.output.results(self.shlax_target)
-        return result
+            self.shlaxfile = Shlaxfile()
+            self.shlaxfile.parse(shlaxfile)
+            self._doc = inspect.getdoc(mod)
+            if 'main' in self.shlaxfile.actions:
+                action = self.shlaxfile.actions['main']
+                for name, child in self.shlaxfile.actions['main'].menu.items():
+                    self[name] = cli2.Callable(
+                        name,
+                        child.callable(),
+                        options={
+                            k: cli2.Option(name=k, **v)
+                            for k, v in action.options.items()
+                        },
+                        color=getattr(action, 'color', cli2.YELLOW),
+                    )
+            for name, action in self.shlaxfile.actions.items():
+                self[name] = cli2.Callable(
+                    name,
+                    action.callable(),
+                    options={
+                        k: cli2.Option(name=k, **v)
+                        for k, v in action.options.items()
+                    },
+                    color=getattr(action, 'color', cli2.YELLOW),
+                    doc=inspect.getdoc(getattr(action, name, None)) or action._doc,
+                )
+        else:
+            from shlax import repo
+            path = repo.__path__._path[0]
+            for shlaxfile in glob.glob(os.path.join(path, '*.py')):
+                name = shlaxfile.split('/')[-1].split('.')[0]
+                mod = importlib.import_module('shlax.repo.' + name)
+                self[name] = cli2.Callable(name, mod)
+
+        return super().__call__(*args, **kwargs)
+
+    def call(self, command):
+        kwargs = copy.copy(self.parser.funckwargs)
+        kwargs.update(self.parser.options)
+        try:
+            return command(*self.parser.funcargs, **kwargs)
+        except WrongResult as e:
+            print(e)
+            self.exit_code = e.proc.rc
+        except ShlaxException as e:
+            print(e)
+            self.exit_code = 1
 
 
-class ActionCommand(Command):
-    def call(self, *args, **kwargs):
-        self.target = self.target(*args, **kwargs)
-        return super().call(*args, **kwargs)
-
-
-class ConsoleScript(Group):
-    def __call__(self, *argv):
-        self.load_actions()
-        #self.load_shlaxfiles()  # wip
-        return super().__call__(*argv)
-
-    def load_shlaxfiles(self):
-        filesdir = os.path.dirname(__file__) + '/shlaxfiles/'
-        for filename in os.listdir(filesdir):
-            filepath = filesdir + filename
-            if not os.path.isfile(filepath):
-                continue
-
-            with open(filepath, 'r') as f:
-                tree = ast.parse(f.read())
-            group = self.group(filename[:-3])
-
-            main = Group(doc=__doc__).load(shlax)
-
-    def load_actions(self):
-        actionsdir = os.path.dirname(__file__) + '/actions/'
-        for filename in os.listdir(actionsdir):
-            filepath = actionsdir + filename
-            if not os.path.isfile(filepath):
-                continue
-            with open(filepath, 'r') as f:
-                tree = ast.parse(f.read())
-            cls = [
-                node
-                for node in tree.body
-                if isinstance(node, ast.ClassDef)
-            ]
-            if not cls:
-                continue
-            mod = importlib.import_module('shlax.actions.' + filename[:-3])
-            cls = getattr(mod, cls[0].name)
-            self.add(cls, name=filename[:-3], cmdclass=ActionCommand)
-
-
-cli = ConsoleScript(doc=__doc__)
+cli = ConsoleScript(__doc__).add_module('shlax.cli')

shlax/container.py

@@ -1,32 +0,0 @@
-import os
-
-from .image import Image
-
-
-class Container:
-    def __init__(self, build=None, image=None):
-        self.build = build
-        self.image = self.build.image
-        prefix = os.getcwd().split('/')[-1]
-        repo = self.image.repository.replace('/', '-')
-        if prefix == repo:
-            self.name = repo
-        else:
-            self.name = '-'.join([prefix, repo])
-
-    async def start(self, target):
-        """Start the container"""
-        await target.rexec(
-            'podman',
-            'run',
-            '--name',
-            self.name,
-            str(self.image),
-        )
-
-    async def stop(self, target):
-        """Start the container"""
-        await target.rexec('podman', 'stop', self.name)
-
-    def __str__(self):
-        return f'Container(name={self.name}, image={self.image})'

shlax/contrib/gitlab.py

@@ -0,0 +1,37 @@
+from copy import deepcopy
+import yaml
+
+from shlax import *
+
+
+class GitLabCI(Script):
+    async def call(self, *args, write=True, **kwargs):
+        output = dict()
+        for key, value in self.kwargs.items():
+            if isinstance(value, dict):
+                output[key] = deepcopy(value)
+                image = output[key].get('image', 'alpine')
+                if hasattr(image, 'image'):
+                    output[key]['image'] = image.image.repository + ':$CI_COMMIT_SHORT_SHA'
+            else:
+                output[key] = value
+
+        output = yaml.dump(output)
+        if kwargs['debug'] is True:
+            self.output(output)
+        if write:
+            with open('.gitlab-ci.yml', 'w+') as f:
+                f.write(output)
+
+        from shlax.cli import cli
+        for arg in args:
+            job = self.kwargs[arg]
+            _args = []
+            if not isinstance(job['image'], str):
+                image = str(job['image'].image)
+            else:
+                image = job['image']
+            await self.action('Docker', Run(job['script']), image=image)(*_args, **kwargs)
+
+    def colorized(self):
+        return type(self).__name__

shlax/exceptions.py

@@ -0,0 +1,22 @@
+class ShlaxException(Exception):
+    pass
+
+
+class Mistake(ShlaxException):
+    pass
+
+
+class WrongResult(ShlaxException):
+    def __init__(self, proc):
+        self.proc = proc
+
+        msg = f'FAIL exit with {proc.rc} ' + proc.args[0]
+
+        if not proc.debug or 'cmd' not in str(proc.debug):
+            msg += '\n' + proc.cmd
+
+        if not proc.debug or 'out' not in str(proc.debug):
+            msg += '\n' + proc.out
+            msg += '\n' + proc.err
+
+        super().__init__(msg)

shlax/image.py

@@ -1,9 +1,19 @@
-import copy
 import os
 import re
 
-
 class Image:
+    ENV_TAGS = (
+        # gitlab
+        'CI_COMMIT_SHORT_SHA',
+        'CI_COMMIT_REF_NAME',
+        'CI_COMMIT_TAG',
+        # CircleCI
+        'CIRCLE_SHA1',
+        'CIRCLE_TAG',
+        'CIRCLE_BRANCH',
+        # contributions welcome here
+    )
+
     PATTERN = re.compile(
         '^((?P<backend>[a-z]*)://)?((?P<registry>[^/]*[.][^/]*)/)?((?P<repository>[^:]+))?(:(?P<tags>.*))?$'  # noqa
         , re.I
@@ -33,6 +43,12 @@ class Image:
         if self.registry == 'docker.io':
             self.format = 'docker'
 
+        # figure tags from CI vars
+        for name in self.ENV_TAGS:
+            value = os.getenv(name)
+            if value:
+                self.tags.append(value)
+
         # filter out tags which resolved to None
         self.tags = [t for t in self.tags if t]
 
@@ -40,6 +56,10 @@ class Image:
         if not self.tags:
             self.tags = ['latest']
 
+    async def __call__(self, action, *args, **kwargs):
+        args = list(args)
+        return await action.exec(*args, **self.kwargs)
+
     def __str__(self):
         return f'{self.repository}:{self.tags[-1]}'
 
@@ -53,8 +73,3 @@ class Image:
 
         for tag in self.tags:
             await action.exec('buildah', 'push', f'{self.repository}:{tag}')
-
-    def layer(self, key):
-        layer = copy.deepcopy(self)
-        layer.tags = ['layer-' + key]
-        return layer

shlax/output.py

@@ -1,6 +1,5 @@
 import re
 import sys
-import types
 
 from .colors import colors
 
@@ -26,23 +25,7 @@ class Output:
     def colorize(self, code, content):
         return self.color(code) + content + self.color()
 
-    def colorized(self, action):
-        if hasattr(action, 'colorized'):
-            return action.colorized(self.colors)
-        elif isinstance(action, types.MethodType):
-            return f'{action.__self__}.{action.__name__}'
-        else:
-            return str(action)
-
-    def __init__(
-            self,
-            prefix=None,
-            regexps=None,
-            debug='cmd,visit,out',
-            write=None,
-            flush=None,
-            **kwargs
-        ):
+    def __init__(self, prefix=None, regexps=None, debug=False, write=None, flush=None, **kwargs):
         self.prefix = prefix
         self.debug = debug
         self.prefix_length = 0
@@ -116,21 +99,22 @@ class Output:
         return line
 
     def test(self, action):
+        if self.debug is True:
             self(''.join([
                 self.colors['purplebold'],
                 '!  TEST   ',
                 self.colors['reset'],
-            self.colorized(action),
+                action.colorized(),
                 '\n',
             ]))
 
     def clean(self, action):
-        if self.debug:
+        if self.debug is True:
             self(''.join([
                 self.colors['bluebold'],
                 '+  CLEAN  ',
                 self.colors['reset'],
-                self.colorized(action),
+                action.colorized(),
                 '\n',
             ]))
 
@@ -140,27 +124,7 @@ class Output:
                 self.colors['orangebold'],
                 '⚠  START  ',
                 self.colors['reset'],
-                self.colorized(action),
-                '\n',
-            ]))
-
-    def info(self, text):
-        if self.debug is True or 'visit' in str(self.debug):
-            self(''.join([
-                self.colors['cyanbold'],
-                '➤   INFO  ',
-                self.colors['reset'],
-                text,
-                '\n',
-            ]))
-
-    def skip(self, action):
-        if self.debug is True or 'visit' in str(self.debug):
-            self(''.join([
-                self.colors['yellowbold'],
-                '↪️   SKIP  ',
-                self.colors['reset'],
-                self.colorized(action),
+                action.colorized(),
                 '\n',
             ]))
 
@@ -170,7 +134,7 @@ class Output:
                 self.colors['greenbold'],
                 '✔ SUCCESS ',
                 self.colors['reset'],
-                self.colorized(action),
+                action.colorized() if hasattr(action, 'colorized') else str(action),
                 '\n',
             ]))
 
@@ -180,32 +144,6 @@ class Output:
                 self.colors['redbold'],
                 '✘  FAIL   ',
                 self.colors['reset'],
-                self.colorized(action),
-                '\n',
-            ]))
-
-    def results(self, action):
-        success = 0
-        fail = 0
-        for result in action.results:
-            if result.status == 'success':
-                success += 1
-            if result.status == 'failure':
-                fail += 1
-
-        self(''.join([
-            self.colors['greenbold'],
-            '✔ SUCCESS REPORT: ',
-            self.colors['reset'],
-            str(success),
-            '\n',
-        ]))
-
-        if fail:
-            self(''.join([
-                self.colors['redbold'],
-                '✘  FAIL REPORT: ',
-                self.colors['reset'],
-                str(fail),
+                action.colorized() if hasattr(action, 'colorized') else str(action),
                 '\n',
             ]))

shlax/pod.py

@@ -1,26 +0,0 @@
-import cli2
-
-from shlax.targets.base import Target
-from shlax.actions.parallel import Parallel
-
-
-class Pod:
-    """Help text"""
-    def __init__(self, **containers):
-        self.containers = containers
-
-    async def _call(self, target, method, *names):
-        methods = [
-            getattr(container, method)
-            for name, container in self.containers.items()
-            if not names or name in names
-        ]
-        await target(Parallel(*methods))
-
-    async def build(self, target, *names):
-        """Build container images"""
-        await self._call(target, 'build', *names)
-
-    async def start(self, target, *names):
-        """Start container images"""
-        await self._call(target, 'start', *names)

shlax/proc.py

@@ -7,46 +7,31 @@ import os
 import shlex
 import sys
 
+from .exceptions import WrongResult
 from .output import Output
 
 
-class ProcFailure(Exception):
-    def __init__(self, proc):
-        self.proc = proc
-
-        msg = f'FAIL exit with {proc.rc} ' + proc.args[0]
-
-        if not proc.output.debug or 'cmd' not in str(proc.output.debug):
-            msg += '\n' + proc.cmd
-
-        if not proc.output.debug or 'out' not in str(proc.output.debug):
-            msg += '\n' + proc.out
-            msg += '\n' + proc.err
-
-        super().__init__(msg)
-
-
 class PrefixStreamProtocol(asyncio.subprocess.SubprocessStreamProtocol):
     """
     Internal subprocess stream protocol to add a prefix in front of output to
     make asynchronous output readable.
     """
 
-    def __init__(self, proc, *args, **kwargs):
-        self.proc = proc
+    def __init__(self, output, *args, **kwargs):
+        self.output = output
         super().__init__(*args, **kwargs)
 
     def pipe_data_received(self, fd, data):
-        if self.proc.output.debug is True or 'out' in str(self.proc.output.debug):
+        if self.output.debug is True or 'out' in str(self.output.debug):
             if fd in (1, 2):
-                self.proc.output(data)
+                self.output(data)
         super().pipe_data_received(fd, data)
 
 
-def protocol_factory(proc):
+def protocol_factory(output):
     def _p():
         return PrefixStreamProtocol(
-            proc,
+            output,
             limit=asyncio.streams._DEFAULT_LIMIT,
             loop=asyncio.events.get_event_loop()
         )
@@ -69,10 +54,8 @@ class Proc:
     """
     test = False
 
-    def __init__(self, *args, prefix=None, raises=True, output=None, quiet=False):
-        if quiet:
-            self.output = Output(debug=False)
-        else:
+    def __init__(self, *args, prefix=None, raises=True, debug=None, output=None):
+        self.debug = debug if not self.test else False
         self.output = output or Output()
         self.cmd = ' '.join(args)
         self.args = args
@@ -104,7 +87,7 @@ class Proc:
         if self.called:
             raise Exception('Already called: ' + self.cmd)
 
-        if 'cmd' in str(self.output.debug):
+        if self.debug is True or 'cmd' in str(self.debug):
             self.output.cmd(self.cmd)
 
         if self.test:
@@ -115,7 +98,7 @@ class Proc:
 
         loop = asyncio.events.get_event_loop()
         transport, protocol = await loop.subprocess_exec(
-            protocol_factory(self), *self.args)
+            protocol_factory(self.output), *self.args)
         self.proc = asyncio.subprocess.Process(transport, protocol, loop)
         self.called = True
 
@@ -140,7 +123,7 @@ class Proc:
         if not self.communicated:
             await self.communicate()
         if self.raises and self.proc.returncode:
-            raise ProcFailure(self)
+            raise WrongResult(self)
         return self
 
     @property

shlax/repo/traefik.py

@@ -0,0 +1,41 @@
+#!/usr/bin/env shlax
+"""
+Manage a traefik container maintained by Shlax community.
+"""
+
+from shlax import *
+
+main = Docker(
+    name='traefik',
+    image='traefik:v2.0.0',
+    networks=['web'],
+    command=[
+        '--entrypoints.web.address=:80',
+        '--providers.docker',
+        '--api',
+    ],
+    ports=[
+        '80:80',
+        '443:443',
+    ],
+    volumes=[
+        '/var/run/docker.sock:/var/run/docker.sock:ro',
+        '/etc/traefik/acme/:/etc/traefik/acme/',
+        '/etc/traefik/htpasswd:/htpasswd:ro',
+    ],
+    labels=[
+        'traefik.http.routers.traefik.rule=Host(`{{ url.split("/")[2] }}`)',
+        'traefik.http.routers.traefik.service=api@internal',
+        'traefik.http.routers.traefik.entrypoints=web',
+    ],
+    doc='Current traefik instance',
+)
+
+install = Script(
+    Htpasswd('./htpasswd', 'root'),
+    main.bind('up'),
+    doc='Deploy a Traefik instance',
+)
+
+up = main.bind('up')
+rm = main.bind('rm')

shlax/result.py

@@ -1,13 +0,0 @@
-class Result:
-    def __init__(self, target, action):
-        self.target = target
-        self.action = action
-        self.status = 'pending'
-        self.exception = None
-
-
-class Results(list):
-    def new(self, target, action):
-        result = Result(target, action)
-        self.append(result)
-        return result

shlax/shlaxfile.py

@@ -0,0 +1,28 @@
+import importlib
+import os
+
+from .actions.base import Action
+
+
+class Shlaxfile:
+    def __init__(self, actions=None, tests=None):
+        self.actions = actions or {}
+        self.tests = tests or {}
+        self.paths = []
+
+    def parse(self, path):
+        spec = importlib.util.spec_from_file_location('shlaxfile', path)
+        mod = importlib.util.module_from_spec(spec)
+        spec.loader.exec_module(mod)
+        for name, value in mod.__dict__.items():
+            if isinstance(value, Action):
+                value.name = name
+                self.actions[name] = value
+            elif callable(value) and getattr(value, '__name__', '').startswith('test_'):
+                self.tests[value.__name__] = value
+
+        self.paths.append(path)
+
+    @property
+    def path(self):
+        return self.paths[0]

shlax/shortcuts.py

@@ -1,18 +0,0 @@
-from .targets.base import Target
-from .targets.buildah import Buildah
-from .targets.localhost import Localhost
-from .targets.stub import Stub
-
-from .actions.copy import Copy
-from .actions.packages import Packages
-from .actions.run import Run
-from .actions.pip import Pip
-from .actions.parallel import Parallel
-from .actions.user import User
-
-from .cli import Command, Group
-
-from .container import Container
-from .pod import Pod
-
-from os import getenv, environ

shlax/strategies/__init__.py

@@ -0,0 +1,3 @@
+from .asyn import Async
+from .script import Script
+from .pod import Pod, Container

shlax/strategies/asyn.py

@@ -0,0 +1,11 @@
+import asyncio
+
+from .script import Script
+
+
+class Async(Script):
+    async def call(self, *args, **kwargs):
+        return await asyncio.gather(*[
+            action(*args, **kwargs)
+            for action in self.actions
+        ])

shlax/strategies/pod.py

@@ -0,0 +1,44 @@
+import os
+from .script import Script
+from ..image import Image
+
+
+class Container(Script):
+    """
+    Wolcome to crazy container control cli
+
+    Such wow
+    """
+    def __init__(self, *args, **kwargs):
+        kwargs.setdefault('start', dict())
+        super().__init__(*args, **kwargs)
+
+    async def call(self, *args, **kwargs):
+        if step('build'):
+            await self.kwargs['build'](**kwargs)
+            self.image = self.kwargs['build'].image
+        else:
+            self.image = kwargs.get('image', 'alpine')
+        if isinstance(self.image, str):
+            self.image = Image(self.image)
+
+        if step('install'):
+            await self.install(*args, **kwargs)
+
+        if step('test'):
+            self.output.test(self)
+            await self.action('Docker',
+                *self.kwargs['test'].actions,
+                image=self.image,
+                mount={'.': '/app'},
+                workdir='/app',
+            )(**kwargs)
+
+        if step('push'):
+            await self.image.push(action=self)
+
+        #name = kwargs.get('name', os.getcwd()).split('/')[-1]
+
+
+class Pod(Script):
+    pass

shlax/strategies/script.py

@@ -0,0 +1,41 @@
+import copy
+import os
+
+from ..exceptions import WrongResult
+from ..actions.base import Action
+from ..proc import Proc
+
+
+class Actions(list):
+    def __init__(self, owner, actions):
+        self.owner = owner
+        super().__init__()
+        for action in actions:
+            self.append(action)
+
+    def append(self, value):
+        action = copy.deepcopy(value)
+        action.parent = self.owner
+        action.status = 'pending'
+        super().append(action)
+
+
+class Script(Action):
+    contextualize = ['shargs', 'exec', 'rexec', 'env', 'which', 'copy']
+
+    def __init__(self, *actions, **kwargs):
+        self.home = kwargs.pop('home', os.getcwd())
+        super().__init__(**kwargs)
+        self.actions = Actions(self, actions)
+
+    async def call(self, *args, **kwargs):
+        for action in self.actions:
+            result = await action(*args, **kwargs)
+            if action.status != 'success':
+                break
+
+    def pollute(self, gbls):
+        for name, script in self.kwargs.items():
+            if not isinstance(script, Script):
+                continue
+            gbls[name] = script

shlax/targets/__init__.py

@@ -0,0 +1,4 @@
+from .buildah import Buildah
+from .docker import Docker
+from .localhost import Localhost
+from .ssh import Ssh

shlax/targets/base.py

@@ -1,146 +0,0 @@
-import asyncio
-import copy
-from pathlib import Path
-import os
-import re
-import sys
-
-from ..output import Output
-from ..proc import Proc
-from ..result import Result, Results
-
-
-class Target:
-    def __init__(self, *actions, root=None):
-        self.actions = actions
-        self.results = []
-        self.output = Output()
-        self.parent = None
-        self.root = root or os.getcwd()
-
-    @property
-    def parent(self):
-        return self._parent or Target()
-
-    @parent.setter
-    def parent(self, value):
-        self._parent = value
-
-    @property
-    def caller(self):
-        """Traverse parents and return the top-levels Target."""
-        if not self._parent:
-            return self
-        caller = self._parent
-        while caller._parent:
-            caller = caller._parent
-        return caller
-
-    async def __call__(self, *actions, target=None):
-        if target:
-            # that's going to be used by other target methods, to access
-            # the calling target
-            self.parent = target
-
-        for action in actions or self.actions:
-            if await self.action(action, reraise=bool(actions)):
-                break
-
-    async def action(self, action, reraise=False):
-        result = Result(self, action)
-        self.output.start(action)
-        try:
-            await action(target=self)
-        except Exception as e:
-            self.output.fail(action, e)
-            result.status = 'failure'
-            result.exception = e
-            if reraise:
-                # nested call, re-raise
-                raise
-            else:
-                import traceback
-                traceback.print_exception(type(e), e, sys.exc_info()[2])
-                return True
-        else:
-            self.output.success(action)
-            result.status = 'success'
-        finally:
-            self.caller.results.append(result)
-
-            clean = getattr(action, 'clean', None)
-            if clean:
-                self.output.clean(action)
-                await clean(self, result)
-
-    async def rexec(self, *args, **kwargs):
-        kwargs['user'] = 'root'
-        return await self.exec(*args, **kwargs)
-
-    async def which(self, *cmd):
-        """
-        Return the first path to the cmd in the container.
-
-        If cmd argument is a list then it will try all commands.
-        """
-        proc = await self.exec('type ' + ' '.join(cmd), raises=False)
-        result = []
-        for res in proc.out.split('\n'):
-            match = re.match('([^ ]+) is ([^ ]+)$', res.strip())
-            if match:
-                result.append(match.group(1))
-        return result
-
-    def shargs(self, *args, **kwargs):
-        user = kwargs.pop('user', None)
-        args = [str(arg) for arg in args if args is not None]
-
-        if args and ' ' in args[0]:
-            if len(args) == 1:
-                args = ['sh', '-euc', args[0]]
-            else:
-                args = ['sh', '-euc'] + list(args)
-
-        if user == 'root':
-            args = ['sudo'] + args
-        elif user:
-            args = ['sudo', '-u', user] + args
-
-        return args, kwargs
-
-        if self.parent:
-            return self.parent.shargs(*args, **kwargs)
-        else:
-            return args, kwargs
-
-    async def exec(self, *args, **kwargs):
-        kwargs['output'] = self.output
-        args, kwargs = self.shargs(*args, **kwargs)
-        proc = await Proc(*args, **kwargs)()
-        if kwargs.get('wait', True):
-            await proc.wait()
-        return proc
-
-    @property
-    def root(self):
-        return self._root
-
-    @root.setter
-    def root(self, value):
-        self._root = Path(value or os.getcwd())
-
-    def path(self, path):
-        if str(path).startswith('/'):
-            path = str(path)[1:]
-        return self.root / path
-
-    async def mkdir(self, path):
-        if '_mkdir' not in self.__dict__:
-            self._mkdir = []
-        path = str(path)
-        if path not in self._mkdir:
-            await self.exec('mkdir', '-p', path)
-            self._mkdir.append(path)
-
-    async def copy(self, *args):
-        return await self.exec('cp', '-a', *args)

shlax/targets/buildah.py

@@ -1,224 +1,156 @@
 import asyncio
-import copy
-import hashlib
-import json
 import os
-import sys
+import asyncio
 from pathlib import Path
+import signal
+import shlex
+import subprocess
+import sys
+import textwrap
 
-from .base import Target
-
-from ..image import Image
+from ..actions.base import Action
+from ..exceptions import Mistake
 from ..proc import Proc
+from ..image import Image
+from .localhost import Localhost
 
 
-class Buildah(Target):
-    """Build container image with buildah"""
+class Buildah(Localhost):
+    """
+    The build script iterates over visitors and runs the build functions, it
+    also provides wrappers around the buildah command.
+    """
+    contextualize = Localhost.contextualize + ['mnt', 'ctr', 'mount', 'image']
 
-    def __init__(self,
-                 *actions,
-                 base=None, commit=None,
-                 cmd=None):
-        self.base = base or 'alpine'
-        self.image = Image(commit) if commit else None
-
-        self.ctr = None
-        self.root = None
+    def __init__(self, base, *args, commit=None, push=False, cmd=None, **kwargs):
+        if isinstance(base, Action):
+            args = [base] + list(args)
+            base = 'alpine'  # default selection in case of mistake
+        super().__init__(*args, **kwargs)
+        self.base = base
         self.mounts = dict()
-
-        self.config = dict(
+        self.ctr = None
+        self.mnt = None
+        self.image = Image(commit) if commit else None
+        self.config= dict(
             cmd=cmd or 'sh',
         )
 
-        # Always consider localhost as parent for now
-        self.parent = Target()
+    def shargs(self, *args, user=None, buildah=True, **kwargs):
+        if not buildah or args[0].startswith('buildah'):
+            return super().shargs(*args, user=user, **kwargs)
 
-        super().__init__(*actions)
-
-    def is_runnable(self):
-        return Proc.test or os.getuid() == 0
-
-    def __str__(self):
-        if not self.is_runnable():
-            return 'Replacing with: buildah unshare ' + ' '.join(sys.argv)
-        return f'Buildah({self.image})'
-
-    async def __call__(self, *actions, target=None):
-        if target:
-            self.parent = target
-
-        if not self.is_runnable():
-            os.execvp('buildah', ['buildah', 'unshare'] + sys.argv)
-            # program has been replaced
-
-        layers = await self.layers()
-        keep = await self.cache_setup(layers, *actions)
-        keepnames = [*map(lambda x: 'localhost/' + str(x), keep)]
-        self.invalidate = [name for name in layers if name not in keepnames]
-        if self.invalidate:
-            self.output.info('Invalidating old layers')
-            await self.parent.exec(
-                'buildah', 'rmi', *self.invalidate, raises=False)
-
-        if actions:
-            actions = actions[len(keep):]
-            if not actions:
-                return self.uptodate()
-        else:
-            self.actions = self.actions[len(keep):]
-            if not self.actions:
-                return self.uptodate()
-
-        self.ctr = (await self.parent.exec('buildah', 'from', self.base)).out
-        self.root = Path((await self.parent.exec('buildah', 'mount', self.ctr)).out)
-
-        return await super().__call__(*actions)
-
-    def uptodate(self):
-        self.clean = None
-        self.output.success('Image up to date')
-        return
-
-    async def layers(self):
-        ret = set()
-        results = await self.parent.exec(
-            'buildah images --json',
-            quiet=True,
-        )
-        results = json.loads(results.out)
-
-        prefix = 'localhost/' + self.image.repository + ':layer-'
-        for result in results:
-            if not result.get('names', None):
-                continue
-            for name in result['names']:
-                if name.startswith(prefix):
-                    ret.add(name)
-        return ret
-
-    async def cache_setup(self, layers, *actions):
-        keep = []
-        self.image_previous = Image(self.base)
-        for action in actions or self.actions:
-            action_image = await self.action_image(action)
-            name = 'localhost/' + str(action_image)
-            if name in layers:
-                self.base = self.image_previous = action_image
-                keep.append(action_image)
-                self.output.skip(
-                    f'Found layer for {action}: {action_image.tags[0]}'
-                )
-            else:
-                break
-        return keep
-
-    async def action_image(self, action):
-        prefix = str(self.image_previous)
-        for tag in self.image_previous.tags:
-            if tag.startswith('layer-'):
-                prefix = tag
-                break
-        if hasattr(action, 'cachekey'):
-            action_key = action.cachekey()
-            if asyncio.iscoroutine(action_key):
-                action_key = str(await action_key)
-        else:
-            action_key = str(action)
-        key = prefix + action_key
-        sha1 = hashlib.sha1(key.encode('ascii'))
-        return self.image.layer(sha1.hexdigest())
-
-    async def action(self, action, reraise=False):
-        stop = await super().action(action, reraise)
-        if not stop:
-            action_image = await self.action_image(action)
-            self.output.info(f'Commiting {action_image} for {action}')
-            await self.parent.exec(
-                'buildah',
-                'commit',
-                '--format=' + action_image.format,
-                self.ctr,
-                action_image,
-            )
-            self.image_previous = action_image
-        return stop
-
-    async def clean(self, target, result):
-        for src, dst in self.mounts.items():
-            await self.parent.exec('umount', self.root / str(dst)[1:])
-
-        if self.root is not None:
-            await self.parent.exec('buildah', 'umount', self.ctr)
-
-        if self.ctr is not None:
-            if result.status == 'success':
-                await self.commit()
-
-            await self.parent.exec('buildah', 'rm', self.ctr)
-
-            if result.status == 'success' and os.getenv('BUILDAH_PUSH'):
-                await self.image.push(target)
-
-    async def mount(self, src, dst):
-        """Mount a host directory into the container."""
-        target = self.root / str(dst)[1:]
-        await self.parent.exec(f'mkdir -p {src} {target}')
-        await self.parent.exec(f'mount -o bind {src} {target}')
-        self.mounts[src] = dst
-
-    async def exec(self, *args, user=None, **kwargs):
         _args = ['buildah', 'run']
         if user:
             _args += ['--user', user]
         _args += [self.ctr, '--', 'sh', '-euc']
-        _args += [' '.join([str(a) for a in args])]
-        return await self.parent.exec(*_args, **kwargs)
-
-    async def commit(self, image=None):
-        image = image or self.image
-        if not image:
-            return
-
-        if not image:
-            # don't go through that if layer commit
-            for key, value in self.config.items():
-                await self.parent.exec(f'buildah config --{key} "{value}" {self.ctr}')
-
-        self.sha = (await self.parent.exec(
-            'buildah',
-            'commit',
-            '--format=' + image.format,
-            self.ctr,
-        )).out
-
-        ENV_TAGS = (
-            # gitlab
-            'CI_COMMIT_SHORT_SHA',
-            'CI_COMMIT_REF_NAME',
-            'CI_COMMIT_TAG',
-            # CircleCI
-            'CIRCLE_SHA1',
-            'CIRCLE_TAG',
-            'CIRCLE_BRANCH',
-            # contributions welcome here
+        return super().shargs(
+            *(
+                _args
+                + [' '.join([str(a) for a in args])]
+            ),
+            **kwargs
         )
 
-        # figure tags from CI vars
-        for name in ENV_TAGS:
-            value = os.getenv(name)
-            if value:
-                self.image.tags.append(value)
+    def __repr__(self):
+        return f'Base({self.base})'
 
-        if image.tags:
-            tags = [f'{image.repository}:{tag}' for tag in image.tags]
-        else:
-            tags = [image.repository]
-
-        for tag in tags:
-            await self.parent.exec('buildah', 'tag', self.sha, tag)
-
-    async def mkdir(self, path):
-        return await self.parent.mkdir(self.path(path))
+    async def config(self, line):
+        """Run buildah config."""
+        return await self.exec(f'buildah config {line} {self.ctr}', buildah=False)
 
     async def copy(self, *args):
-        return await self.parent.copy(*args[:-1], self.path(args[-1]))
+        """Run buildah copy to copy a file from host into container."""
+        src = args[:-1]
+        dst = args[-1]
+        await self.mkdir(dst)
+
+        procs = []
+        for s in src:
+            if Path(s).is_dir():
+                target = self.mnt / s
+                if not target.exists():
+                    await self.mkdir(target)
+                args = ['buildah', 'copy', self.ctr, s, Path(dst) / s]
+            else:
+                args = ['buildah', 'copy', self.ctr, s, dst]
+            procs.append(self.exec(*args, buildah=False))
+
+        return await asyncio.gather(*procs)
+
+    async def mount(self, src, dst):
+        """Mount a host directory into the container."""
+        target = self.mnt / str(dst)[1:]
+        await self.exec(f'mkdir -p {src} {target}', buildah=False)
+        await self.exec(f'mount -o bind {src} {target}', buildah=False)
+        self.mounts[src] = dst
+
+    def is_runnable(self):
+        return (
+            Proc.test
+            or os.getuid() == 0
+        )
+
+    async def call(self, *args, **kwargs):
+        if self.is_runnable():
+            self.ctr = (await self.exec('buildah', 'from', self.base, buildah=False)).out
+            self.mnt = Path((await self.exec('buildah', 'mount', self.ctr, buildah=False)).out)
+            result = await super().call(*args, **kwargs)
+            return result
+
+        from shlax.cli import cli
+        debug = kwargs.get('debug', False)
+        # restart under buildah unshare environment
+        argv = [
+            'buildah', 'unshare',
+            sys.argv[0],  # current script location
+            cli.shlaxfile.path,  # current shlaxfile location
+        ]
+        if debug is True:
+            argv.append('-d')
+        elif isinstance(debug, str) and debug:
+            argv.append('-d=' + debug)
+        argv += [
+            cli.parser.command.name,  # script name ?
+        ]
+
+        await self.exec(*argv)
+
+    async def commit(self):
+        if not self.image:
+            return
+
+        for key, value in self.config.items():
+            await self.exec(f'buildah config --{key} "{value}" {self.ctr}')
+
+        self.sha = (await self.exec(
+            'buildah',
+            'commit',
+            '--format=' + self.image.format,
+            self.ctr,
+            buildah=False,
+        )).out
+
+        if self.image.tags:
+            tags = [f'{self.image.repository}:{tag}' for tag in self.image.tags]
+        else:
+            tags = [self.image.repository]
+
+        for tag in tags:
+            await self.exec('buildah', 'tag', self.sha, tag, buildah=False)
+
+    async def clean(self, *args, **kwargs):
+        if self.is_runnable():
+            for src, dst in self.mounts.items():
+                await self.exec('umount', self.mnt / str(dst)[1:], buildah=False)
+
+            if self.status == 'success':
+                await self.commit()
+                if 'push' in args:
+                    await self.image.push(action=self)
+
+            if self.mnt is not None:
+                await self.exec('buildah', 'umount', self.ctr, buildah=False)
+
+            if self.ctr is not None:
+                await self.exec('buildah', 'rm', self.ctr, buildah=False)

shlax/targets/docker.py

@@ -0,0 +1,99 @@
+import asyncio
+from pathlib import Path
+import os
+
+from ..image import Image
+from .localhost import Localhost
+
+
+class Docker(Localhost):
+    contextualize = Localhost.contextualize + ['mnt', 'ctr', 'mount']
+
+    def __init__(self, *args, **kwargs):
+        self.image = kwargs.get('image', 'alpine')
+        self.name = kwargs.get('name', os.getcwd().split('/')[-1])
+
+        if not isinstance(self.image, Image):
+            self.image = Image(self.image)
+
+        super().__init__(*args, **kwargs)
+
+    def shargs(self, *args, daemon=False, **kwargs):
+        if args[0] == 'docker':
+            return args, kwargs
+
+        extra = []
+        if 'user' in kwargs:
+            extra += ['--user', kwargs.pop('user')]
+
+        args, kwargs = super().shargs(*args, **kwargs)
+
+        if self.name:
+            executor = 'exec'
+            extra = [self.name]
+            return [self.kwargs.get('docker', 'docker'), executor, '-t'] + extra + list(args), kwargs
+
+        executor = 'run'
+        cwd = os.getcwd()
+        if daemon:
+            extra += ['-d']
+        extra = extra + ['-v', f'{cwd}:{cwd}', '-w', f'{cwd}']
+        return [self.kwargs.get('docker', 'docker'), executor, '-t'] + extra + [str(self.image)] + list(args), kwargs
+
+    async def call(self, *args, **kwargs):
+        def step(step):
+            return not args or step in args
+
+        # self.name = (
+        #     await self.exec(
+        #         'docker', 'ps', '-aq', '--filter',
+        #         'name=' + self.name,
+        #         raises=False
+        #     )
+        # ).out.split('\n')[0]
+
+        if step('rm'):
+            await self.rm(*args, **kwargs)
+
+        if step('down') and self.name:
+            await self.exec('docker', 'down', '-f', self.name)
+
+        if step('up'):
+            await self.up(*args, **kwargs)
+        return await super().call(*args, **kwargs)
+
+    async def rm(self, *args, **kwargs):
+        return await self.exec('docker', 'rm', '-f', self.name)
+
+    async def down(self, *args, **kwargs):
+        """Remove instance, except persistent data if any"""
+        if self.name:
+            self.name = (await self.exec('docker', 'start', self.name)).out
+        else:
+            self.name = (await self.exec('docker', 'run', '-d', '--name', self.name)).out
+
+    async def up(self, *args, **kwargs):
+        """Perform start or run"""
+        if self.name:
+            self.name = (await self.exec('docker', 'start', self.name)).out
+        else:
+            self.id = (await self.exec('docker', 'run', '-d', '--name', self.name)).out
+
+    async def copy(self, *args):
+        src = args[:-1]
+        dst = args[-1]
+        await self.mkdir(dst)
+
+        procs = []
+        for s in src:
+            '''
+            if Path(s).is_dir():
+                await self.mkdir(s)
+                args = ['docker', 'copy', self.ctr, s, Path(dst) / s]
+            else:
+                args = ['docker', 'copy', self.ctr, s, dst]
+            '''
+            args = ['docker', 'cp', s, self.name + ':' + dst]
+            procs.append(self.exec(*args))
+
+        return await asyncio.gather(*procs)

shlax/targets/localhost.py

@@ -1,14 +1,19 @@
-import copy
+import os
 import re
 
-from ..output import Output
-from ..proc import Proc
-from ..result import Result, Results
+from shlax.proc import Proc
 
-from .base import Target
+from ..strategies.script import Script
 
 
-class Localhost(Target):
+class Localhost(Script):
+    root = '/'
+    contextualize = Script.contextualize + ['home']
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.home = kwargs.pop('home', os.getcwd())
+
     def shargs(self, *args, **kwargs):
         user = kwargs.pop('user', None)
         args = [str(arg) for arg in args if args is not None]
@@ -24,17 +29,54 @@ class Localhost(Target):
         elif user:
             args = ['sudo', '-u', user] + args
 
-        return args, kwargs
-
         if self.parent:
             return self.parent.shargs(*args, **kwargs)
         else:
             return args, kwargs
 
     async def exec(self, *args, **kwargs):
-        kwargs['output'] = self.output
+        if 'debug' not in kwargs:
+            kwargs['debug'] = getattr(self, 'call_kwargs', {}).get('debug', False)
+        kwargs.setdefault('output', self.output)
         args, kwargs = self.shargs(*args, **kwargs)
         proc = await Proc(*args, **kwargs)()
         if kwargs.get('wait', True):
             await proc.wait()
         return proc
+
+    async def rexec(self, *args, **kwargs):
+        kwargs['user'] = 'root'
+        return await self.exec(*args, **kwargs)
+
+    async def env(self, name):
+        return (await self.exec('echo $' + name)).out
+
+    async def exists(self, *paths):
+        proc = await self.exec('type ' + ' '.join(cmd), raises=False)
+
+    async def which(self, *cmd):
+        """
+        Return the first path to the cmd in the container.
+
+        If cmd argument is a list then it will try all commands.
+        """
+        proc = await self.exec('type ' + ' '.join(cmd), raises=False)
+        result = []
+        for res in proc.out.split('\n'):
+            match = re.match('([^ ]+) is ([^ ]+)$', res.strip())
+            if match:
+                result.append(match.group(1))
+        return result
+
+    async def copy(self, *args):
+        if args[-1].startswith('./'):
+            args = list(args)
+            args[-1] = self.home + '/' + args[-1][2:]
+        args = ['cp', '-rua'] + list(args)
+        return await self.exec(*args)
+
+    async def mount(self, *dirs):
+        pass
+
+    async def mkdir(self, *dirs):
+        return await self.exec(*['mkdir', '-p'] + list(dirs))

shlax/targets/ssh.py

@@ -0,0 +1,17 @@
+import os
+
+from shlax.proc import Proc
+
+from .localhost import Localhost
+
+
+class Ssh(Localhost):
+    root = '/'
+
+    def __init__(self, host, *args, **kwargs):
+        self.host = host
+        super().__init__(*args, **kwargs)
+
+    def shargs(self, *args, **kwargs):
+        args, kwargs = super().shargs(*args, **kwargs)
+        return (['ssh', self.host] + list(args)), kwargs

shlax/targets/stub.py

@@ -1,23 +0,0 @@
-from ..proc import Proc
-
-from .base import Target
-
-
-class ProcStub(Proc):
-    async def __call__(self, wait=True):
-        return self
-
-    async def communicate(self):
-        self.communicated = True
-        return self
-
-    async def wait(self):
-        return self
-
-
-class Stub(Target):
-    async def exec(self, *args, **kwargs):
-        proc = await ProcStub(*args, **kwargs)()
-        if kwargs.get('wait', True):
-            await proc.wait()
-        return proc

shlaxfile.py

@@ -1,20 +1,55 @@
-#!/usr/bin/env python
-"""
-Shlaxfile for shlax itself.
-"""
+#!/usr/bin/env shlax
+from shlax.contrib.gitlab import *
 
-from shlax.shortcuts import *
+PYTEST = 'py.test -svv tests'
 
-shlax = Container(
-    build=Buildah(
-        Packages('python38', 'buildah', 'unzip', 'findutils', upgrade=False),
-        Copy('setup.py', 'shlax', '/app'),
-        Pip('/app'),
-        base='quay.io/podman/stable',
-        commit='shlax',
-    ),
+test = Script(
+    Pip('.[test]'),
+    Run(PYTEST),
 )
 
+build = Buildah(
+    'quay.io/podman/stable',
+    Packages('python38', 'buildah', 'unzip', 'findutils', 'python3-yaml', upgrade=False),
+    Async(
+        # python3.8 on centos with pip dance ...
+        Run('''
+        curl -o setuptools.zip https://files.pythonhosted.org/packages/42/3e/2464120172859e5d103e5500315fb5555b1e908c0dacc73d80d35a9480ca/setuptools-45.1.0.zip
+        unzip setuptools.zip
+        mkdir -p /usr/local/lib/python3.8/site-packages/
+        sh -c "cd setuptools-* && python3.8 setup.py install"
+        easy_install-3.8 pip
+        echo python3.8 -m pip > /usr/bin/pip
+        chmod +x /usr/bin/pip
+        '''),
+        Copy('shlax/', 'setup.py', '/app'),
+    ),
+    Pip('/app[full]'),
+    commit='docker.io/yourlabs/shlax',
+    workdir='/app',
+)
 
-if __name__ == '__main__':
-    print(Group(doc=__doc__).load(shlax).entry_point())
+shlax = Container(
+    build=build,
+    test=Script(Run('./shlaxfile.py -d test')),
+)
+
+gitlabci = GitLabCI(
+    test=dict(
+        stage='build',
+        script='pip install -U --user -e .[test] && ' + PYTEST,
+        image='yourlabs/python',
+    ),
+    build=dict(
+        stage='build',
+        image='yourlabs/shlax',
+        script='pip install -U --user -e . && CACHE_DIR=$(pwd)/.cache ./shlaxfile.py -d shlax build push',
+        cache=dict(paths=['.cache'], key='cache'),
+    ),
+    pypi=dict(
+        stage='deploy',
+        only=['tags'],
+        image='yourlabs/python',
+        script='pypi-release',
+    ),
+)

tests/actions/test_base.py

@@ -0,0 +1,53 @@
+from shlax import *
+
+inner = Run()
+other = Run('ls')
+middle = Buildah('alpine', inner, other)
+outer = Localhost(middle)
+middle = outer.actions[0]
+other = middle.actions[1]
+inner = middle.actions[0]
+
+
+def test_action_init_args():
+    assert other.args == ('ls',)
+
+
+def test_action_parent_autoset():
+    assert list(outer.actions) == [middle]
+    assert middle.parent == outer
+    assert inner.parent == middle
+    assert other.parent == middle
+
+
+def test_action_context():
+    assert outer.context is inner.context
+    assert middle.context is inner.context
+    assert middle.context is outer.context
+    assert other.context is outer.context
+
+
+def test_action_sibblings():
+    assert inner.sibblings() == [other]
+    assert inner.sibblings(lambda s: s.args[0] == 'ls') == [other]
+    assert inner.sibblings(lambda s: s.args[0] == 'foo') == []
+    assert inner.sibblings(type='run') == [other]
+    assert inner.sibblings(args=('ls',)) == [other]
+
+
+def test_actions_parents():
+    assert other.parents() == [middle, outer]
+    assert other.parents(lambda p: p.base == 'alpine') == [middle]
+    assert inner.parents(type='localhost') == [outer]
+    assert inner.parents(type='buildah') == [middle]
+
+
+def test_action_childrens():
+    assert middle.children() == [inner, other]
+    assert middle.children(lambda a: a.args[0] == 'ls') == [other]
+    assert outer.children() == [middle, inner, other]
+
+
+def test_action_getattr():
+    assert other.exec == middle.exec
+    assert other.shargs == middle.shargs

tests/test_image.py

@@ -1,7 +1,7 @@
 import pytest
 import os
 
-from shlax.image import Image
+from shlax import Image
 
 
 tests = {
@@ -25,3 +25,9 @@ def test_args(arg, expected):
     im = Image(arg)
     for k, v in expected.items():
         assert getattr(im, k) == v
+
+def test_args_env():
+    os.environ['IMAGE_TEST_ARGS_ENV'] = 'foo'
+    Image.ENV_TAGS = ['IMAGE_TEST_ARGS_ENV']
+    im = Image('re/po:x,y')
+    assert im.tags == ['x', 'y', 'foo']

tests/test_output.py

@@ -1,5 +1,5 @@
 import pytest
-from shlax.output import Output
+from shlax import Output
 
 
 class Write:

tests/test_proc.py

@@ -0,0 +1,65 @@
+import pytest
+
+from unittest.mock import patch
+
+from shlax import *
+from shlax import proc
+
+
+test_args_params = [
+    (
+        Localhost(Run('echo hi')),
+        [('sh', '-euc', 'echo hi')]
+    ),
+    (
+        Localhost(Run('echo hi', user='jimi')),
+        [('sudo', '-u', 'jimi', 'sh', '-euc', 'echo hi')]
+    ),
+    (
+        Localhost(Run('echo hi', user='root')),
+        [('sudo', 'sh', '-euc', 'echo hi')]
+    ),
+    (
+        Ssh('host', Run('echo hi', user='root')),
+        [('ssh', 'host', 'sudo', 'sh', '-euc', 'echo hi')]
+    ),
+    (
+        Buildah('alpine', Run('echo hi')),
+        [
+            ('buildah', 'from', 'alpine'),
+            ('buildah', 'mount', ''),
+            ('buildah', 'run', '', '--', 'sh', '-euc', 'echo hi'),
+            ('buildah', 'umount', ''),
+            ('buildah', 'rm', ''),
+        ]
+    ),
+    (
+        Buildah('alpine', Run('echo hi', user='root')),
+        [
+            ('buildah', 'from', 'alpine'),
+            ('buildah', 'mount', ''),
+            ('buildah', 'run', '--user', 'root', '', '--', 'sh', '-euc', 'echo hi'),
+            ('buildah', 'umount', ''),
+            ('buildah', 'rm', ''),
+        ]
+    ),
+    (
+        Ssh('host', Buildah('alpine', Run('echo hi', user='root'))),
+        [
+            ('ssh', 'host', 'buildah', 'from', 'alpine'),
+            ('ssh', 'host', 'buildah', 'mount', ''),
+            ('ssh', 'host', 'buildah', 'run', '--user', 'root', '', '--', 'sh', '-euc', 'echo hi'),
+            ('ssh', 'host', 'buildah', 'umount', ''),
+            ('ssh', 'host', 'buildah', 'rm', ''),
+        ]
+    ),
+]
+@pytest.mark.parametrize(
+    'script,commands',
+    test_args_params
+)
+@pytest.mark.asyncio
+async def test_args(script, commands):
+    with Proc.mock():
+        await script()
+        assert commands == Proc.test

tests/test_target.py

@@ -1,101 +0,0 @@
-import pytest
-
-from shlax.targets.stub import Stub
-from shlax.actions.run import Run
-from shlax.actions.parallel import Parallel
-from shlax.result import Result
-
-
-class Error:
-    async def __call__(self, target):
-        raise Exception('lol')
-
-
-@pytest.mark.asyncio
-async def test_success():
-    action = Run('echo hi')
-    target = Stub(action)
-    await target()
-    assert target.results[0].action == action
-    assert target.results[0].status == 'success'
-
-
-@pytest.mark.asyncio
-async def test_error():
-    action = Error()
-    target = Stub(action)
-    await target()
-    assert target.results[0].action == action
-    assert target.results[0].status == 'failure'
-
-
-@pytest.mark.asyncio
-async def test_nested():
-    nested = Error()
-
-    class Nesting:
-        async def __call__(self, target):
-            await target(nested)
-    nesting = Nesting()
-
-    target = Stub(nesting)
-    await target()
-
-    assert len(target.results) == 2
-    assert target.results[0].status == 'failure'
-    assert target.results[0].action == nested
-    assert target.results[1].status == 'failure'
-    assert target.results[1].action == nesting
-
-
-@pytest.mark.asyncio
-async def test_parallel():
-    winner = Run('echo hi')
-    looser = Error()
-    parallel = Parallel(winner, looser)
-
-    target = Stub(parallel)
-    await target()
-    assert len(target.results) == 3
-    assert target.results[0].status == 'success'
-    assert target.results[0].action == winner
-    assert target.results[1].status == 'failure'
-    assert target.results[1].action == looser
-    assert target.results[2].status == 'failure'
-    assert target.results[2].action == parallel
-
-
-@pytest.mark.asyncio
-async def test_function():
-    async def hello(target):
-        await target.exec('hello')
-    await Stub()(hello)
-
-
-@pytest.mark.asyncio
-async def test_method():
-    class Example:
-        def __init__(self):
-            self.was_called = False
-        async def test(self, target):
-            self.was_called = True
-
-    example = Example()
-    action = example.test
-    target = Stub()
-    await target(action)
-    assert example.was_called
-
-
-@pytest.mark.asyncio
-async def test_target_action():
-    child = Stub(Run('echo hi'))
-    parent = Stub(child)
-
-    grandpa = Stub()
-    await grandpa(parent)
-    assert len(grandpa.results) == 3
-
-    grandpa = Stub(parent)
-    await grandpa()
-    assert len(grandpa.results) == 3