Framework for command-line devops and personal productivity tools

See the sample project to get started with WizLib. Browse related projects for real-world examples of WizLib usage. Then consult the reference guide for details.

Logo by Freepik-Flaticon

Capabilities

WizLib wraps the built-in ArgumentParser with a set of functions, classes, and conventions that enables:

• Easy addition of loosely coupled new subcommands
• Unified access to configuration files with a normalized approach
• Standardized use of stdin, stdout, and stderr
• Plugin-type system for handling alternate UIs (such as curses or even a web UI)
• Usable completion support for user input
• Abstracts some of the argparse complexity
• Applies conventions to application code structure
• Supports test-driven development and CICD

History

The WizLib project emerged from a difficult period in my life. Unemployed and recovering from a major health condition, I threw myself into one activity that soothed me: coding in Python. Because of a vision impairment (and personal preference) I focused on simple, easy-to-use command-line applications to help with daily life such as a todo list manager and a utility for organizing files in Dropbox. Some of the tools required similar parts, so I started collecting them into a library, giving birth to WizLib itself.

(I called the original collection "Steampunk Wizard", but that's a story for friends.)

Title: Reference Slug: reference

Reference

Approach

Think of WizLib as a framework, in that it provides a library of code that, when used in a conventional manner, provides a structure and capabilities to simplify application development.

Commands exist independently. To add a new command, simply add a Python file in the command directory with a class definition that inherits from the base command. The command will automatically appear as an option in usage, and the implementation has access to handlers for arguments, inputs, user interfaces, and values from a configuration file for the application.

A WizLib application has the following directory structure at a mimimum. In this case, the app is called Sample with the main command sample and one subcommand doit.

sample
 ├─ .git
 └─ sample
     ├─ __init__.py
     ├─ __main__.py
     └─ command
         ├─ __init__.py
         └─ doit_command.py

API

WizLib itself defines several Python classes and functions for inclusion in projects. They include:

• WizApp - Base class for a WizLib app
• Command - Root class for the app-specific command class, which forms the base class for other commands
• ConfigHandler - handles configuration, either through environment variables or a YAML configuration file
• StreamHandler - simplifies handling of input via stdin for non-tty inputs such as pipes
• ClassFamily - a primitive class that loads all subclasses in a directory into a "family" which can be queried a lookup, avoiding the need to include or reference every member of the family independently
• SuperWrapper - a primitive class that "wraps" subclass methods, so that the superclass method gets calls before and after the subclass method - like an inversion of super()

WizApp

WIP

Command

WIP

ConfigHandler

Enables easy configuration across multiple levels. Tries each of the following approaches in order until one finds the required config option

1. Attributes of the instance (subclass of ConfigHandler) itself (e.g. gitlab_host)
2. Then look for a specific env variable for that config setting in all caps, e.g. GITLAB_HOST
3. If those both fail, then look for a YAML configuration file:
   • First identified with a --config / -c option on the command line
   • Then with a path in the APPNAME_CONFIG environment variable - note all caps
   • Then look in the local working directory for .appname.yml
   • Then look for ~/.appname.yml in the user's home directory

Config files are in YAML, and look something like this:

gitlab:
  host: gitlab.com
local:
  root: $HOME/git

Note that nested labels in the config map to hyphenated command line options.

StreamHandler

When enabled, simplifies handling of input via stdin for non-tty inputs such as pipes. Optionally, users can specify a file other than stdin using the --stream option.

ClassFamily

A class family is a set of class definitions that use single inheritance (each subclass inherits from only one parent) and often multiple inheritance (subclasses can inherit from subclasses). So it's a hierarchy of classes, with one super-parent (termed the "atriarch") at the top.

We offer a way for members of the family to declare themselves simply by living in the right package location. Then those classes can be instantiated using keys or names, without having to be specifically called. The members act independently of each other.

What we get, after importing everything and loading it all, is essentially a little database of classes, where class-level properties become keys for looking up member classes. So, for example, we can have a family of commands, and use a command string to look up the right command.

Ultimately, the atriarch of the family -- the class at the top of the hierarchy -- holds the database, actually a list, in the property called "family". So that class can be queried to find appropriate family member classes or instances thereof.

This utility provides functions for importing family members, loading the "families" property of the super-parent, and querying the family.

In the process of loading and querying the class family, we need to avoid inheritance of attributes. There might be abstract intermediary classes that don't want to play. So we use __dict__ to ensure we're only seeing the atttributes that are defined on that specific class.

SuperWrapper

Provide a decorator to wrap a method so that it's called within the inherited version of that method.

Example of use:

class Parent(SuperWrapper):
    def execute(self, method, *args, **kwargs):
        print(f"Parent execute before")
        method(self, *args, **kwargs)
        print(f"Parent execute after")

class InBetween(Parent):
    @Parent.wrap
    def execute(self, method, *args, **kwargs):
        print(f"IB execute before")
        method(self, *args, **kwargs)
        print(f"IB execute after")

class NewChild(InBetween):
    @InBetween.wrap
    def execute(self, name):
        print(f"Hello {name}")

c = NewChild()
c.execute("Jane")

Note that for a method to be "wrappable" it must take the form shown above, and explicitly call the method that's handed into it. So strictly, this is different from regular inheritance, where the parent class method has the same signature as the child class method.

Test Helpers

The framework includes a few nuggets to support unit testing using unittest.

WizLibTestCase

A subclass of unittest.TestCase to simplify patching inputs and outputs.

Because WizLib applications sometimes depend on user interaction, and other times depend on a stream on standard input, unit testing can require complicated patching. Inherit from WizLibTestCase to make it easier.

Here's an example of how to use it. In this example, we send the word 'laughter' to the standard input stream and capture standard output in a variable for assertion.

# Inherit from WizLibTestCase - get everything in TestCase plus some
class DummyTest(WizLibTestCase):

    # Test function - ass
    def test_input_stdin(self):

        # Use with instead of decorators (to take advantage of 'as') and combine them (if you like that style)
        with \
                self.patch_stream('laughter'), \
                self.patchout() as o:

            # Get going with the test
            DummyApp.start('dance')
            o.seek(0)
        self.assertIn('laughter', o.read())

The actual methods that can be used are:

• patch_stream(val: str) - Patch stream input such as pipes for stream handler
• patch_ttyin(val: str) - Patch input typed by a user in shell ui
• patcherr() - Capture output from standard error
• patchout() - Capture output from standard output

They are convenience methods; feel free to patch those objects separately if you prefer.

Fake values

Framework-aware mock objects

ConfigHandler.fake

Generates a fake WizLib configuration for testing. Example:

    def test_fake_config(self):
        a = DummyApp()
        a.config = ConfigHandler.fake(dummy_vehicle='boat')
        c = DriveCommand(a)
        r = c.execute()
        self.assertIn('Driving a boat', r)

Note that the keys passed to the fake method contain hyphens, where values are referenced using hyphens, for example:

self.app.config.get('dummy-vehicle')

StreamHandler.fake

Generates a fake standard input stream. Example:

    def test_fake_input(self):
        a = DummyApp()
        a.stream = StreamHandler.fake('madly')
        c = DanceCommand(a)
        r = c.execute()
        self.assertEqual('Dancing madly', r)

Testing commands

Since much of the framework-specific functionality in a WizLib app lives in commands, some special guidelines and provisions apply.

Testing command execution only

To test only the execution of a command's functionality (independent of argument parsing), instantiate the command directly. It still requires a WizApp object as the first parameter, and arguments as successive parameters. Example:

    def test_only_command(self):
        a = DummyApp()
        c = MathCommand(a, value=10.0)
        r = c.execute()
        self.assertEqual(6.0, r)

Testing a command with parsing

The WizLibApp.parse_run method provides a quick entry point into the parsing and execution of a full command, without having to go through the class-level start/initialize steps. Just pass in the command and its arguments as function arguments, and the selected command will run. Used to test not just the functionality of the command itself, but also that arguments are correctly parsed. Example:

    def test_parse_run(self):
        with self.patchout() as o:
            DummyApp().parse_run('draw', '-c', 'straight')
        o.seek(0)
        self.assertIn('Curve was straight', o.read())