Taking Command Line Arguments in Python

Submitted by nanodano on Fri, 02/14/2020 - 17:59

Introduction

When processing command line arguments in Python, you have a few options available:

sys.argv - a raw list of arguments passed to the Python interpreter
argparse - Python's standard package for argument parsing (replaces getopt and optparse)
fileinput.input() - A special function, similar to Ruby's ARGF, that returns the lines of input from standard input or from the list of filenames provided as arguments.
docopt - A third-party package that uses the docstring as documentation and configuration for arg parsing
click - A third-party package that uses decorators to map command arguments

Let's look at examples of each option.

Methods of parsing command line arguments

sys.argv

The sys.argv object is a Python list of arguments passed to the Python interpreter. It will include the arguments passed after python.

For example, if you had the file test.py:

import sys
print(type(sys.argv))
print(sys.argv)

Example run:

C:\Users\NanoDano>python test.py --a --b c d
<class 'list'>
['test.py', '--a', '--b', 'c', 'd']

Note if you invoke the Python file directly, without python in the command, then the arguments may be different. For example, if you ran ./test.py instead of python test.py you may get different results.

Also note if you run a script on the command line directly with the -c option of Python, it won't actually include the whole script. Here is an example:

C:\Users\NanoDano>python -c "import sys; print(sys.argv)" --one two three
['-c', '--one', 'two', 'three']

Since sys.argv is just a basic list, you can do all the expected things like checking the length and getting elements by index.

Here is an example of checking if the first argument is the --help:

# example.py
import sys

if len(sys.argv) == 2 and sys.argv[1] == '--help':
    print("Help activated.")

Then run the program with:

python example.py --help

argparse.ArgumentParser

If you want a higher level interface to the arguments aside from directly accessing the sys.argv list, you can use Python's standard library package argparse. This package replaces the older getopt and optparse.

argparse allows for many things:

Required or optional arguments
Automatic -h/--help option with usage instructions
Multiple versions of flags, e.g. short and long flags (-h/--help)
Type checking
Required argument checking

You can learn more by reading PEP 389 argparse.

Here are all the options when creating an argparse.ArgumentParser:

class argparse.ArgumentParser(
    prog=None, usage=None, description=None, epilog=None,
    parents=[], formatter_class=argparse.HelpFormatter,
    prefix_chars='-', fromfile_prefix_chars=None,
    argument_default=None, conflict_handler='error',
    add_help=True, allow_abbrev=True)

You can leave all the options blank and they have sane defaults. After you create an argparse.ArgumentParser, you can start adding arguments with the add_argument() function. Here are all the options for argparse.ArgumentParser.add_argument():

argparse.ArgumentParser.add_argument(name or flags...[, action][, nargs][, const][, default][, type][, choices][, required][, help][, metavar][, dest])

Let's look at some example usage in a script:

# args.py
import argparse

# Initialize a default arg parser
arg_parser = argparse.ArgumentParser()

# Simple optional argument. Example usage `--filename test.txt`
arg_parser.add_argument('--filename')
# Boolean, true or false (exists or not)
arg_parser.add_argument('--turbo', action='store_true')
# Multiple variations
arg_parser.add_argument('--verbose', '-v', action='store_true')
# With help text
arg_parser.add_argument('--debug', help='Raise log levels to debug', action='store_true')
# Required arguments
arg_parser.add_argument('--run', required=True, action='store_true')
# Arg with default provided
arg_parser.add_argument('--user', default='anonymous')
# Multiple arguments. Example usage `--numbers 2 4 6 8 10`
arg_parser.add_argument('--numbers', metavar='num', type=int, nargs='+', help='Provide a list of numbers')

# Now that the arg parser knows what arguments to look for,
# parse all the command line arguments in to a convenient object
arguments = arg_parser.parse_args()

print(type(arguments))  # <class 'argparse.Namespace'>
print(arguments)
print(arguments.filename)
print(arguments.turbo)
print(arguments.verbose)
print(arguments.debug)
print(arguments.run)
print(arguments.user)
print(arguments.numbers)

Then try running this script various ways to see how it behaves:

python args.py
python args.py -h
python args.py --help
python args.py --run
python args.py --run --debug
python args.py --run -v
python args.py --run --verbose
python args.py --run --user nanodano
python args.py --run --numbers 2 4 6 8 10 --user nanodano

fileinput.input()

fileinput is a special function, similar to Ruby's ARGF, that returns the lines of input from standard input or from the list of filenames provided as arguments.

For example, you can either pipe in files via standard input or provide a list of filenames as arguments to the application:

python my_app.py < file1.txt
python my_app.py file1.txt file2.txt file3.txt

Using fileinput.input() you will receive a list containing all the lines provided by the files, whether they came from standard input or they came from filenames provided as arguments (it will automatically open each and read the lines).

Let's look at an example of how to use it:

# fileinput_example.py
import fileinput

lines_of_data = fileinput.input()

print(type(lines_of_data))  # fileinput.FileInput

print('Data:')

for line in lines_of_data:
    print(line.strip())

# Or join it all together to a single string
# print(''.join(lines_of_data))

Here is how you can run the program to pipe in files or provide file names:

# Pipe file in via stdin
python fileinput_example.py < file1.txt

# Provide list of files as arguments
python fileinput_example.py file1.txt file2.txt file3.txt

Learn more about the fileinput.FileInput object.

docopt

docopt is nice because it ties your help message documentation and usage information directly to the code. It uses the docstring itself as the documentation and the configuration for the argument parsing.

It has a lot of strict conventions though and can be tough and confusing to use. Personally, I don't reach for it often now, but I have used it in past with success and is worth a mention.

To use it, you will need to install/require the docopt package:

python -m pip install docopt

Here is an example script using it:

"""My cool app

Usage:
  myapp.py <username> [--verbose]
  myapp.py (-h | --help)
  myapp.py --version

Options:
  -h --help   Show help info
  --version   Show version info
  --verbose   Print verbose output
"""
from docopt import docopt  # pip install docopt

args = docopt(__doc__)

if __name__ == '__main__':
    print(args)

You can try running this in several ways:

python myapp.py
python myapp.py -h
python myapp.py --help
python myapp.py --version
python myapp.py nanodano
python myapp.py nanodano --verbose

click

Click is third-party package that uses decorators to map command arguments. It also generates help documentation automatically and has a set of nice features. One cool feature is that it can prompt you for input if you did not provide it as an argument. It also supports sub-com

Since it is a third-party package you will need to install it first. The easiest way is to use pip to install the package:

python -m pip install click

Let's look at a simple example of how it works:

import click

@click.command()
@click.option('--verbose/--quiet', default=False, help='Toggle verbose')
@click.option('--username', prompt='Enter username')
@click.option('--password', prompt=True, hide_input=True, confirmation_prompt=True)
def main(verbose, username, password):
    print(verbose)
    print(username)
    print(password)


if __name__ == '__main__':
    main()

The example above shows you how to do a boolean variable (--verbose), an argument that will prompt if no value is provided (username and password), and how to get a hidden input with confirmation (the password).

Conclusion

After reading this guide, you should have a good understanding of how to process command line arguments in Python using sys.argv, argparse.ArgumentParser, fileinput.input(), docopt, and click.