From 3fb492930979e630d29935f68f3c861148770efb Mon Sep 17 00:00:00 2001 From: Kyle Bowman Date: Sat, 16 Mar 2024 15:44:14 -0400 Subject: [PATCH] Update documentation. --- README.md | 41 ++++++++++++ design.md => develop.md | 141 +++++++++++++++++++++++++++++++++------- findings.md | 85 ------------------------ specs.md | 18 ----- 4 files changed, 159 insertions(+), 126 deletions(-) rename design.md => develop.md (52%) delete mode 100644 findings.md delete mode 100644 specs.md diff --git a/README.md b/README.md index e69de29..5815efc 100644 --- a/README.md +++ b/README.md @@ -0,0 +1,41 @@ +# Proto Overview + +The proto library simplifies the creation of command line applications by +inferring parser options from type signatures. + +## Design Philosophy + +The core proto design belief is that a CLI app writer should focus on writing +functions, not interfaces. The interface you create matters. But so does the library that you use to create that interface. You shouldn't have to bend to idiosyncratic command line library to produce a conventional CLI app. Especially when much of it is boilerplate. From that core belief manifests these design principles: + +1. Proto treats functions as first class citizens. (Really `Callable`s). +2. (To do) Proto does not modify your function. (E.g. the decorator returns the function unchanged.) +3. Proto respects your type annotations to a fault. + +Unrelated, but important: +1. Keep the library thin. CLI apps shouldn't be weighty. Avoid using third party tools in this library. +2. Let users extend proto from within their app where they need to. (Example: to create a custom type from the command line, register a `get_argspecs()` function.) + + +### About "To a Fault" + +Type annotations are a contract. Proto will abide by the contract. +If you want proto to work well, you must abide by the contract that you define. +If you specify, for example, `Optional[int]`, proto guarantees that, when +parsed, it will return either an `int` or `None`. It is your responsibility +to handle the both cases in your function. + +# Usage + +1. Define your function with type annotations. +2. Decorate the function with `@command`. +3. Use `fn.parse()` to load arguments. +4. Use `fn.run()` to invoke the command. + +Here is an idea of how it might look when done: + +``` python +@command(aliases=['alias1', 'alias2'], help="help message") +def function(arg1: type1 = default1, arg2: type2 = default 2, ...) + definition +``` \ No newline at end of file diff --git a/design.md b/develop.md similarity index 52% rename from design.md rename to develop.md index db8a470..d786017 100644 --- a/design.md +++ b/develop.md @@ -1,19 +1,103 @@ -# Interface Design Goals -* Make a nice interface that is built on and compliant with argparse -* No dependencies outside stdlib +# Overview -## Core Ideas -* Use a command decorator to turn a function into a command. -* Leverage the function signature to autopopulate command info. +The main functions are defined in `utils.py`. They all accept an annotated +function. -Example: -``` python -@command(aliases=['alias1', alias2], help="help message") -def function(arg1: type1 = default1, arg2: type2 = default 2, ...) - definition -``` +* `get_defaults` +* `get_types` +* `get_parser` + +The most intricate function is `get_parser`, which produces an `argparse.ArgumentParser`. + +It does so by reading a function annotations and calling `parser.add_argument(options)` for each argument. There is a lot of conditional logic to determine which options are appropriate. + +The module `prototype.py` exposes the main interface. It creates a class `Command` which invokes the functions from `utils.py` and stores them in a data structure. It exposes the `parse()` and `run()` methods. + +# Codebase Conventions + +## Comment Keywords +Don't commit anything that starts with "temporarily". + +Search by using `git grep "# [A-Z]*:"`. + +* TODO - temporarily bookmark a TBD task +* BUG - temporarily bookmark something that needs to be fixed and tested. +* NOTE - describe an important consideration about the code. May be a design consideration, reminder, etc.. +* ASSUME - describes an assumption that must be true for the code to behave as intended. +* HACK - describes a non-obvious solution; typically indicates a "misuse" of another bit of code +* QUESTION - temporarily indicates something that should be further researched + +# Testing + +## Utils.py + +* Each function in `utils.py` operates on `Callable` +* They gets information from the `Callable`'s signature +* Signatures are optional in Python. + +Let's ensure that each function works on: +* A function +* A method +* A minimal custom `Callable` +* W/ and w/o types declared in the signature +* W/ and w/o defaults declared in the signature +* W/ positional and keyword args. +* Using more than one type +* (?) I don't think anything uses return type yet. +* (?) I don't use any mandatory keywords yet. + +The most typical case is probably a function with types and defaults specified +in the signature. + +## Proto.py -## MainCLI Overview +Use `args=['list', 'of', 'arguments']` to supply command line arguments. + +## REPL testing +In test files, use a `if __main__ == __name__` guard to setup typical things +used for interactive debugging. +# Roadmap + +## Buglike +These aren't necessarily bugs in the sense of something being wrong. +They are bugs in the sense that they might scurry when you shine a light. +They are potential problems for a yet unsupported domain. + +### Functions that annotate with ABCs/Protocols +(This buglike goes hand-in-hand with the next one.) + +When you define a function with an ABC/Protocol, you don't have obvious access +to an initializer. There are two problems with this: + +1. Proto might need to get creative with dispatching `get_argspecs`. Using +`type(prm.annotation)` doesn't get an initializer. But you technically don't need +an initializer to create the parser. +2. You do, however, need an initializer when it comes time to parse the command +line arguments and create objects for the function. + +Some musings: +There are two problems: +* How do you catch a Protocol with single dispatch? + * [Single dispatch isn't great with protocols](https://stackoverflow.com/questions/70986620/combining-single-dispatch-and-protocols) + * For ABCs, you could require user defines a get_argspec that instantiates object. Then at least, they are responsible for their own assumptions. +* How do you use a string (command line argument) to create an *instance* of a Protocol? (Protocols don't have initializers.) +* If you could catch a Protocol, you could add a hook so that users define a default initializer. (You could, of course, annotate with a concrete type. But I don't want people to have to rewrite functions to accomodate the command line interface.) +* Maybe there is a way to define a default implentation of an ABC or a Protocol? +* (Promising) Maybe we can cop out. If it's a Protocol, dispatch to a proto-defined hook that passes the buck to the function writer. I.e. add a placeholder "type" to arg_specs. That "type" requires the user to handle the initializer themselves. + +### Parser definition vs Runtime arguments + +* If you look through Union and define the parser based on the first type... +* But then at runtime, the CLI looks like the second one... +* You can't go back and redefine the parser. + +* Possible solution - define a type placeholder for Union. +* The type placeholder could itself dispatch to runtime parse/validators? +* The important thing is that `fn.parse()` returns types that the function expects. + +## Features + +### MainCLI Overview * We want to support, but not mandate subparsers. * The `MainCLI` class provides the top level parser and automatically registers commands and subparsers. @@ -24,6 +108,19 @@ designate a function for use by a subparser. then invoking `Command.run()`. * using a convenience `dispatch()` method that carries out the above steps for you. +``` python +class MainCLI(): + + # Signature? + def __init__(self): + # self.parser = # default parser + # self.commands = get_commands() + pass + + def parse(self)->Command: + pass +``` + ### Parser Construction: Designating a function for a subparser * Unless you need nesting; avoid mentioning an explicit parser. * If you need subparsing, add `subparser=cmd_parser` to the `@command` decorator. @@ -113,9 +210,7 @@ if __name__ == "__main__": dispatch_cli() ``` -# Possible Enhancements - -## Command aliases +### Command aliases ```python @command(aliases=['alias']) @@ -123,7 +218,7 @@ def long_name(*args, **kwargs): pass ``` -## Validation Ideas: +### Validation Ideas: 1. Define a predicate data type: Callable[Any, bool] 2. Define a validate decorator for Cmd objects * Allow for multiple validators and use any(list[bool]) @@ -141,14 +236,14 @@ def is_valid(arg1, arg2): return True if X else False ``` -## (?) Make get_parser() more flexible +### (?) Make get_parser() more flexible See also the section about Help documentation. I'm not sure what must be done when invoking `parser.add_argument()` and what can be done post construction. If something must be done during `add_argument()` is there a way to remove then re-add an argument? -## (?) Help documentation +### (?) Help documentation This deals with two things: * How to manually specify command help? * How to automatically generate some kind of help? @@ -163,23 +258,23 @@ Idea One: Define as dictionary and pass unpacked to @command. Idea Two: Define helper functions around idea Zero -## (?) Distribute a CLI testing library +### (?) Distribute a CLI testing library * Mock sys.argv? (Use a context manager?) * Shell out? * Promote making useful test functions available for REPL based development. -## (?) Aggregate Commands +### (?) Aggregate Commands We need a way to fetch Cmds from their modules and register with CLI. Like `instances = fetch_subclasses(Cmd)` Make generic? fetch(subclasses, Cmd) -## (?) Serializing Ideas +### (?) Serializing Ideas cmd.to_json() - packs command and args into json add a to_json Cmd? add by default to MainCLI() parser? => Mirror with from_json -## (?) Autoimport flexibility +### (?) Autoimport flexibility Enhancement to MainCLI - add a param that enables users to define the command search space. \ No newline at end of file diff --git a/findings.md b/findings.md deleted file mode 100644 index 0ca9873..0000000 --- a/findings.md +++ /dev/null @@ -1,85 +0,0 @@ -# Mental Stack -Usage: -* Decorate an annotated function -* Use fn.parse() -* use fn.run -* For testing, haphazardly use sys.argv. - -## Comment Keywords -Don't commit anything that starts with "temporarily". - -Search by using `git grep "# [A-Z]*:"`. - -* TODO - temporarily bookmark a TBD task -* BUG - temporarily bookmark something that needs to be fixed and tested. -* NOTE - describe an important consideration about the code. May be a design consideration, reminder, etc.. -* ASSUME - describes an assumption that must be true for the code to behave as intended. -* HACK - describes a non-obvious solution; typically indicates a "misuse" of another bit of code -* QUESTION - temporarily indicates something that should be further researched - -# Findings from experiments - -## About Parsers -* Subparsers are of same type as parser (`argparse.ArgumentParser`) -* Corollary: You can nest subparsers indefinitely -* ~~The `argparse.Namespace` object always stores values as strings.~~ -* Unless you explictly used `type` in `add_argument`, a CLI argument is parsed as a string. -* The `--` is stripped from optional flags when parsed into an `argparse.Namespace` -* An argument is stored as an `Action`. - * Try to reach into the `parser._actions` list to set attributes between `add_argument` and `parser_args`. - * For reference, `parser.add_arguments` seems to be defined in `argparse._ActionsContainer.add_argments` - -## About Decorators -* You can define `@instance.method` wrappers -* This might be useful to do lower level argparse stuff, like setting help info. - -Example: -With `Command.wrap` defined as: -``` python -# Part of class Command: -def wrap(self, fn: Callable, *args, **kwargs): - return str(fn(*args, **kwargs)).upper() -``` - -You can enhance the function decorate the function `new_name` : -``` python -@test.wrap -def new_name(): - return "kyle" -``` - -Invoke it as follows: -``` python -if __name__ == "__main__": - print(new_name) # "KYLE" -``` - -## About Signatures -* Argnames = Signature - Kwargs; this is a good way to report missing required arg - -## About Singledispatch - -* Use `@singledispatch` decorator to mark function`foo` a generic function. -* Use `@foo.register` decorator to mark a function as an implemntation of `foo`. -* Dispatch is based on the type of the first argument. -* Method resolution is from specific to generic. -* Use `foo.dispatch()` to determine which function is dispatched by ``. -* Use `foo.registry` to see all registered functions. - -# About MainCLI -Purpose: -1. Search for other commands to register with it -2. Serve as main parser - Build the parser -3. Parse and dispatch to the command - -Some additional notes: -* Commands are *instances* of Cmd in this framework; -they are not subclasses -* Idiomatically, to get instances, register each instance -with the class at initialization. -* To get instances, they must be initialized (at runtime), -meaning that Cmd() (or the decorator) must have been run -* Each function that defines a command must be imported -*prior* to being able to be registered by MainCLI() -* For now, let's define a module registering standard. -(Enhancement) Look into a look-before-import mechanism (pluggy?) \ No newline at end of file diff --git a/specs.md b/specs.md deleted file mode 100644 index 8e7ea0f..0000000 --- a/specs.md +++ /dev/null @@ -1,18 +0,0 @@ -## Stories -1. Create `Command` object and `@command` decorator. - * What data does `Command` require? (Look at argparse requirements) - * What methods does `Command` require? - * get_parser(fn: Callable)->argparse.ArgParser - * get_types(fn: Callable)->dict[type] - * get_defaults(fn: Callable)->dict - * init(fn: Callable, ?)->Self - * run()->Any: - * What additional info doese `@command` need? - * Tests: - * Check each builder method happy and sad paths. - * Check `Command.run()` output. - * Maybe shell out at least once. -2. Add subparser arg to `@command` decorator (and `Command`?). -3. Add `MainCLI()` (and dispatch convenience method.) - * Add `Command` registering here. -4. Look into enhancements -- 2.39.5