Python Tips: Dynamic Tab Completion

Do you want to have dynamic, nested, subcommands that can do tab complete? Me either.

Intro

Tab complete is a special feature that most cherish in a CLI application. It can improve user interaction, especially for us lazy typists. It can also reduce errors. Implementing tab complete is not difficult but what if you want it to be dynamic? What about having tab complete for commands and their subcommands? If you start thinking about this, it can quickly add to needing a lot of static methods in an application.

What I'm showing isn't a novel or new approach, but rather a high level understanding of how I implemented. I am using Python's dynamic method creation capability to generate completion methods at runtime and when certain classes get called. It rebuilds the potential tab complete options each time the class is called.

The Problem: A Complex CLI

Build a CLI application with the below capabilities:

• Dynamic commands: commands and subcommands are not fixed and can be added to a database or file and reloaded at anytime

• Nested subcommands: subcommands can be nested under multiple levels

• Tab complete: All levels of the command hierarchy need the ability to have tab completion and the necessary arguments passed to the appropriate subcommand

One might ask why I wanted to make such a complex CLI interface. The problem, at the core, is more about making the CLI interface appear simple, despite however complex the backend might be. I also wanted the codebase to not require any changes when someone adds commands to the application, since these are maintained by a file and/or database.

The Solution: Dynamic Methods

Dynamic method creation allows the generation of methods at runtime or when a class is called, depending on how it is implemented. It is using Python's ability to manipulate class attributes dynamically. What I am going to show will have some missing components. You can see the full code at in my application repo on github/3d6564/artifactor. In the menu.py file you can see how it is implemented. I point to that repository because there are varying levels of complexity and what I implemented is more complex than to do a basic dynamic method for tab complete.

Key libraries needed

I use two libraries to accomplish everything. One is the capabilities of CLI interaction and the second is to build the method map for the commands that get loaded from a file.

• cmd

• inspect

Key methods needed

This is a subset of methods and what their overall purpose is for the dynamic tab complete design.

• dynamic_complete: This is a generic function that handles the logic of tab complete for commands and subcommands. Essentially it fetches possible commands for what has been typed and returns the appropriate potential tab complete results.

• create_complete_methods: This function creates the completion methods dynamically and binds them to the respective class they are called through. If you notice, this will call a method made within itself and it will return those results passed to the dynamic_complete function.

• fetch_subclasses This is a basic function to retrieve the subclasses that meet a certain criteria for a class object

• fetch_nested_submethods: This function will retrieve the possible subcommands that have been made for a class and subclass called. The nested subcommands will need to be made prior to calling this function.

1. Dynamic Complete

In a perfect world, the final return statement is basically never reached.

def dynamic_complete(self, text, line, begidx, endidx, command_name, subcommand_fetchers):
     """dynamic method for tab completion of subcommands and nested subcommands."""
     try:
          remaining_text = line[len(command_name):].strip()
     except Exception:
          return []

     # fetch possible primary subcommands
     possible_matches = subcommand_fetchers.get(command_name)

     # if remaining_text is empty, suggest primary subcommands
     if not remaining_text:
          return [sc + ' ' for sc in possible_matches]

     # split remaining_text to handle subcommands and nested subcommands
     split_text = remaining_text.split(maxsplit=1)
     primary_subcommand = split_text[0]
     remaining_subtext = split_text[1] if len(split_text) > 1 else ''

     # check matching subcommand and not only a space typed
     if ' ' not in remaining_text and primary_subcommand not in possible_matches:
          filtered_matches = [sc for sc in possible_matches if sc.startswith(remaining_text)]
          return [sc + ' ' for sc in filtered_matches]

     # get nested subcommands if subcommand fully typed
     if primary_subcommand in subcommand_fetchers:
          subcommands = subcommand_fetchers[primary_subcommand]
          return [sc for sc in subcommands if sc.startswith(remaining_subtext)]

     return []

2. Create Complete Methods

This binds the generated methods to the respective class when called.

def create_complete_methods(command_name, subcommand_fetchers):
     """Create a complete_<command_name> method dynamically with support for nested subcommands."""
     def complete_method(self, text, line, begidx, endidx):
          return dynamic_complete(self, text, line, begidx, endidx, command_name, subcommand_fetchers)
     return complete_method

3. Fetch Subclasses

This one is pretty straight forward.

def fetch_subclasses(cls):
    """Primary subcommands under 'configure'"""
    subcommands = [subclass.__name__.lower() for subclass in cls.__subclasses__()]
    return subcommands + ['help']

4. Fetch Nested Submethods

This one is very similar to fetch_subclasses but it does it for subcommands.

def fetch_nested_submethods(cls, sub_cls):
     """Nested subcommands under 'configure hosts'"""
     methods = list(set(dir(sub_cls)) - set(dir(cls)))
     if cls == Run:
          methods = [method for method in methods if method.startswith('do_')]
          methods = sorted([method[3:] if method.startswith('do_') else method for method in methods])
     return methods + ['help']

Implementation

The implementation, when I think about it, feels like a migraine. I still want to try to find a way to simplify it, but I just don't know if there is one. Everything gets called under a Cmd class object. The class object dynamically builds a command map when its initialized and that is what will build the basic command map.

It should also be noted that I override the tab completion capability to remove trailing spaces using the command map made during initialization. This all happens in Artifactor's MainCmd class. Also, because of the inherent dynamic nature of the nested subcommands, I have to generate a dynamic help.

A more simple implementation of the dynamic subcommands can be viewed in the Run class. This one is easier to see how the dynamic tab complete works, because it does not have nested subcommand menus like the main menu of the CLI tool.

Conclusion

I have probably done a terrible job of explaining this. I hope I've at least inspired some thoughts on how you could implement something similar, and maybe more simple as well. Feel free to reach out if you have questions or want to collaborate and provide a more simple approach to dynamic nested subcommands in Python!

Happy coding!