pip from behind a proxy

python -m pip install --proxy="{fully qualified domain name}:{port number}" {item to be pipped}

Change the colours of the console

To change the colouring of the console within which Python runs you can use system calls. Actually I know of no other way to change the colours of the console completely. I have found some usages of the ctypes module but suspected I would only change the colour of newly printed output. TODO: Create an os aware function.

Please be aware that copy and paste of the code below will most probably lead to indention error, as the code was more or less just a copy of respective passages in real code.

from os import system   # NOTE system calls usually are not portable!

system('color 3e')

Get user input

Please be aware that copy and paste of the code below will most probably lead to indention error, as the code was more or less just a copy of respective passages in real code.

import re

RE_ENV_NON_DEV = re.compile('^INT|KONS|PROD$')

            env = input('Please enter the instance ( INT KONS PROD ) ' +
                        'to deploy to: ')
            while env is None or RE_ENV_NON_DEV.match(env) is None:
                env = input('Please enter the instance ( INT KONS PROD ) ' +
                            'to deploy to: ')

Set up logging

A simple logger using the logging module. TODO: Convert code into a method returning a logger object providing the method to unregister/remove and close the file handler (see the other post).

Please be aware that copy and paste of the code below will most probably lead to indention error, as the code was more or less just a copy of respective passages in real code.

from datetime import datetime
import logging
from logging.handlers import RotatingFileHandler
from pathlib import Path

    scriptPathParts = Path(__file__).resolve().parts
    logDirectoryName = 'logs'
    scriptDirectory = Path(__file__).parent.resolve()
    logDirectory = scriptDirectory.joinpath(logDirectoryName)
    logDirectoryCreated = False
    if not logDirectory.is_dir():
        logDirectory.mkdir(parents = True, exist_ok = True)
        logDirectoryCreated = True
    startTimestamp = datetime.now().strftime("%Y%m%d%H%M%S")
    logger = logging.getLogger('deploy')
    logLevel = 'INFO'
    logger.setLevel(logLevel)
    formatter = logging.Formatter(
            '%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    logHandlerStream = logging.StreamHandler()
    logHandlerStream.setFormatter(formatter)
    logger.addHandler(logHandlerStream)
    logFileName = scriptPathParts[-1].replace('.py', '')
    # for debugging it is easier if the log file name stays the same
    if (not logger.isEnabledFor(logging.DEBUG)):
        logFileName = logFileName + '_' + startTimestamp
    logFileName = logFileName + '.log'
    logFile = logDirectory.joinpath(logFileName)
    # FIXME for some unknown reason to me the mode does not get respected, the log
    # file is appended. Maybe this is a "feature" of Windoof.
    logHandlerFile = RotatingFileHandler(
            filename = str(logFile),
            mode = 'w',
            maxBytes = 10485760,
            backupCount = 10,
            encoding = 'utf-8',
            delay = True)
    logHandlerFile.setFormatter(formatter)
    logger.addHandler(logHandlerFile)

Compressing the file a logger has written

In the case that it is in the same script log file gets written and you want to remove the original log file retaining only the compressed one, do (the files are available as pathlib path objects). For the set up of the logger see the other post.

Please be aware that copy and paste of the code below will most probably lead to indention error, as the code was more or less just a copy of respective passages in real code.

        # Doing some log file handling so the file handler of the logger better
        # be removed. Explicit close is required. Othe wise the file does not
        # get released causing (on Windows) 'PermissionError: [WinError 32]'
        logHandlerFile.close()
        logger.removeHandler(logHandlerFile)
        createLZMACompressedFile(fileNameIn = str(logFile),
                                 fileNameOut = str(logFileNew),
                                 logger = logger,
                                 removeIn = True)

Execute external programme and log its results

Of course you need to have set up a logger using the logging module (see the other post).

Please be aware that copy and paste of the code below will most probably lead to indention error, as the code was more or less just a copy of respective passages in real code.

import subprocess

def executeExternalAndLog(cmd, logger, reErrorLine):
    u"""Executes given 'exploded' command and logs its results to given logger.

    Given compiled regular expression is used to log lines as errors.
    """
    try:
        result = \
          subprocess.run(args=cmd,
                         check = True,
                         stdout = subprocess.PIPE, # to capture output
                         stderr = subprocess.PIPE, # to capture error
                         encoding = 'utf-8',
                         universal_newlines=True)
    except subprocess.CalledProcessError as e:
        logger.debug('Command: {}'.format(e.cmd)) # DEBUG because can contain password
        logger.info('Return code: {}'.format(e.returncode))
        logger.info("Stdout")
        for line in e.stdout.splitlines():
            if reErrorLine.match(line):
                logger.error(line)
            else:
                logger.info(line)
        logger.critical("Stderr")
        for line in e.stderr.splitlines():
            logger.critical(line)
        raise
    except:
        logger.critical(sys.exc_info())
        raise
    else:
        for line in result.stdout.splitlines():
            logger.info(line)

A call could be done like this.

        executeExternalAndLog(["sqlplus",
                               "-L",
                               '{}/{}@{}'.format(schema, password, instance),
                               "@install_master.sql"],
                              logger,
                              RE_ORACLE_ERROR_LINE)

Compress a file with LZMA

Of course you need to have set up a logger using the logging module (see the other post).

Please be aware that copy and paste of the code below will most probably lead to indention error, as the code was more or less just a copy of respective passages in real code.

import lzma

def createLZMACompressedFile(fileNameIn, fileNameOut, logger, removeIn = False):
    u"""Compresses in file to out file using LZMA.

    The in file gets removed optionally.
    """
    logger.debug("fileNameIn: {}".format(fileNameIn))
    logger.debug("fileNameOut: {}".format(fileNameOut))
    logger.debug("Current path: {}".format(os.getcwd()))

    # Treat data as bytes to compress (wb and rb respectively)
    with lzma.open(filename = fileNameOut,
                   mode = 'wb',
                   format = lzma.FORMAT_ALONE,
                   preset = lzma.PRESET_EXTREME) as fileOut:
        with open(fileNameIn, 'rb') as fileIn:
            data = fileIn.read()

        fileOut.write(data)

    if removeIn:
        os.remove(fileNameIn)

A call could be done like this.

            createLZMACompressedFile(fileNameIn = str(logFile),
                                     fileNameOut = str(logFileNew),
                                     logger = logger,
                                     removeIn = True)