Introduction

Software creeping slowly into every day devices and everybody carrying around a device that easily outperforms a couple of 90s supercomputers is a clear indicator for the increased presence of software. The Number of projects hosted on a well known „developer-platform“ also indicates that the amount of software has increaseda lot in the last years. Long story made short: As a software developer there is no need to re-invent the wheel every now and then. Normally there is a library available for one or the other task that one has to solve. Question is how to pull foreign projects into your own python project? There is a couple of options!

pip

pip allows for installation of packages from the python package index which is an extensive repository of python packages. I cannot say what pip stands for, but it is an extremly usefull tool. Using opensuse I had to install it first. The package is called python3-pip for me. Then I can start pulling in packages: Let’s say for some reason I would have to parse some java source codes from a python script. Since I am not willing to spend time on writing a Java Parse now I have a look at pypi and find this. I want to give it a try and pull in the package using pip:

$ pip install javalang

However since pip tries to install the package below /usr/lib/ the installation initially fails since I am not root… So I try again this time looking up in the pip manual that there is an option for installation into my home directory:

$ pip install --user javalang

and this time it works! Now I can start trying out the example code from the javalang github page

#!/usr/bin/python3
 
import javalang
 
tree = javalang.parse.parse("package javalang.brewtab.com; class Test {}")
print(tree)
$ ./javalang-test.py3
CompulationUnit

Which looks like the process of downloading the package and making it available to the python interpreter has worked as expected! Looking up information about the installed package reveals following information:

$ pip show javalang
---
Metadata-Version: 1.1
Name: javalang
Version: 0.10.1
Summary: Pure Python Java parser and tools
Home-page: http://github.com/c2nes/javalang
Author: Chris Thunes
Author-email: cthunes@brewtab.com
License: UNKNOWN
Location: /home/user/pythonbase/lib/python3.4/site-packages
Requires: six

Manual Installation

The process is documented here.

Sometimes when I write a little python script – with only a few lines of code – it starts growing and I keep adding functionality. When a certain threshold is reached I start thinking that there should be a logger used (instead of print) and maybe a configuration file (instead of all these commandline switches). Since python coding is not something I do on a daily basis python code is not something I come up with just like that. So I either copy & paste code from other scripts or even have to look up how exactly the logger is set up, how a configuration file is loaded or how signal handler is registered. It’s always the same story and as a keen Emacs user I finally made it an end 🙂
Using emacs auto-insert-mode. In my configuration auto-insert-mode inserts the file „template.py“ from my „~/.emacs.d/templates/“ directory and inserts into the buffer whenever I open a yet non-existing file with the .py extension:

(setq auto-insert-directory "~/.emacs.d/templates/"
      auto-insert-query nil)
(add-to-list 'auto-insert-alist '(".*\\.py[3]?$" . [ "template.py" ]))

This works pretty well and of course template.py hosts my nitty gritty python boiler plate that I want to see in my scripts (no matter how small they are 🙂 ):

#!/usr/bin/python3
 
import logging
import logging.handlers
import signal
import sys
import os
import time
import argparse
 
# Author     : Matthias
# Description: Python script template
 
class Application:
 
    name               = ''
    version            = ''
    log                = None
    properties         = None
    parser             = None
    args               = None
 
    def __init__(self):
        signal.signal(signal.SIGINT, Application.signal_int_handler)        
        parser = argparse.ArgumentParser(description="", epilog="")
        parser.add_argument("-v", "--verbose", help="Be more verbose when logging", action="store_true")
        parser.add_argument("-P", "--properties", help="A properties file for use by the application", type=str)
        parser.add_argument("-l", "--loghost", help="Name of host to receive log messages", default="127.0.0.1")
        parser.add_argument("-p", "--logport", help="Port of service to receive log messages", type=int, default=logging.handlers.DEFAULT_TCP_LOGGING_PORT)
        parser.add_argument("-d", "--logdomain", help="Domain for logging", default="this-script")
        parser.add_argument("-r", "--remotelog", help="Enable remote logging with default host and port", action="store_true")
        self.args = parser.parse_args()
        self.parser = parser
        self.setup_logging()
        self.read_properties(self.args.properties)
 
    def setup_logging(self):
        self.log = logging.getLogger(self.args.logdomain)
        rootlogger = logging.getLogger()
        formatstring='%(asctime)s %(levelname)-15s %(name)s # %(message)s'
        formatter = logging.Formatter(fmt=formatstring, datefmt='%d.%m.%y %I:%M:%S')
        handler = None
        if self.args.remotelog:
            handler = logging.handlers.SocketHandler(self.loghost_name, self.loghost_port)
        else:
            handler = logging.StreamHandler(sys.stderr)
        handler.setFormatter(formatter)
        rootlogger.addHandler(handler)
        level = logging.INFO
        if self.args.verbose:
            level = logging.DEBUG
        self.log.setLevel(level)
        rootlogger.setLevel(level)
        self.log.propagate=1
 
    def read_properties(self, filename):
        """ Read the file passed as parameter as a properties file. """
        if filename:
            properties = {}
            comment_char = "#"
            seperator = ":"
            with open(filename, "rt") as f:
                for line in f:
                    l = line.strip()
                    if l and not l.startswith(comment_char):
                        key_value = l.split(seperator)
                        key = key_value[0].strip()
                        value = seperator.join(key_value[1:]).strip().strip('"') 
                        properties[key] = value 
            self.properties = properties
 
    @staticmethod
    def signal_int_handler(signal, frame):
        interrupt_msg = '\r\n\r\n{} {} terminated by keyboard interrupt'.format(Application.name, Application.version)
        print(interrupt_msg)
        exit(0)
 
    def run(self):
        pass
 
def main():
    app = Application()
    app.log.info('{} {} is starting'.format(app.name, app.version))
    app.run()
    app.log.info('{} {} is done'.format(app.name, app.version))
 
if __name__ == '__main__':
    main()
 
#
# Done
#
# # # end of script