Copying files between dired buffers

There is a couple of packages for emacs that really have application character. Like for example:

  • https://www.emacswiki.org/emacs/Magit
  • https://www.emacswiki.org/emacs/EmacSpeak
  • https://www.emacswiki.org/emacs/GnusTutorial
  • https://github.com/skeeto/elfeed
  • https://emacswiki.org/emacs/Sunrise_Commander

And there are propably some more. Me personally – I did not adopt well to the „App in Emacs“ approach as I use none of the mentioned packages. It’s not a categorical decision, I just did not get used to it. My writings in Emacs Lisp have a much lesser extend: Recently I had need to copy files from one directory to the other and I just love to have this done with two directories showing up side by side as in well known „file commanders“. That said Emacs Lisp comes to the rescue!

(defun mp-dired-2pane-copy-over ()
  (interactive)
    (when (eq 2 (length (window-list)))
      (let ((other-directory nil)
            (file-to-copy (dired-get-filename)))
        (progn
          (other-window 1)
          (setq other-directory (dired-current-directory))
          (other-window 1)
          (copy-file file-to-copy other-directory)
          (other-window 1)
          (revert-buffer)
          (other-window 1)))))

Show long filenames in ibuffer

I have customized my ibuffer-format to have the name column width set to 36. This is fine in 99% of the filenames showing up there. However I also have to access a couple of files that have a common prefix that is longer then 36 characters. This way the files cannot be distinguished in ibuffer… Since I am not willing to have the width set to even higher value I have added this little defun to my init.el to deal with the issue:

 
(defun mp/ibuffer-show-filename ()
  (interactive)
  (let ((buf (ibuffer-current-buffer))
        (lsoutput nil))
    (when (file-exists-p (buffer-file-name buf))
      (with-temp-buffer
        (let* ((filename (buffer-file-name buf))
               (default-directory (file-name-directory filename))
               (just-filename (file-name-nondirectory filename)))
          (call-process "/usr/bin/ls" nil t nil "-l" just-filename)
          (setq lsoutput (buffer-substring-no-properties (point-min) (- (point-max) 1))))))
    (message lsoutput)))
 
(define-key ibuffer-mode-map (kbd "f") 'mp/ibuffer-show-filename)

This way I can stick to my column width of 36 and whenever I have need to see the a longer filename I can just press f on the entry and see the name at the bottom of the screen.

Pulling foreign code into your python project

Introduction

Software creeping slowly into every day devices and everybody carrying around a device that easily outperforms a couple of 90s supercomputers is a clear indicator for the increased presence of software. The Number of projects hosted on a well known „developer-platform“ also indicates that the amount of software has increaseda lot in the last years. Long story made short: As a software developer there is no need to re-invent the wheel every now and then. Normally there is a library available for one or the other task that one has to solve. Question is how to pull foreign projects into your own python project? There is a couple of options!

pip

pip allows for installation of packages from the python package index which is an extensive repository of python packages. I cannot say what pip stands for, but it is an extremly usefull tool. Using opensuse I had to install it first. The package is called python3-pip for me. Then I can start pulling in packages: Let’s say for some reason I would have to parse some java source codes from a python script. Since I am not willing to spend time on writing a Java Parse now I have a look at pypi and find this. I want to give it a try and pull in the package using pip:

$ pip install javalang

However since pip tries to install the package below /usr/lib/ the installation initially fails since I am not root… So I try again this time looking up in the pip manual that there is an option for installation into my home directory:

$ pip install --user javalang

and this time it works! Now I can start trying out the example code from the javalang github page

#!/usr/bin/python3
 
import javalang
 
tree = javalang.parse.parse("package javalang.brewtab.com; class Test {}")
print(tree)
$ ./javalang-test.py3
CompulationUnit

Which looks like the process of downloading the package and making it available to the python interpreter has worked as expected! Looking up information about the installed package reveals following information:

$ pip show javalang
---
Metadata-Version: 1.1
Name: javalang
Version: 0.10.1
Summary: Pure Python Java parser and tools
Home-page: http://github.com/c2nes/javalang
Author: Chris Thunes
Author-email: cthunes@brewtab.com
License: UNKNOWN
Location: /home/user/pythonbase/lib/python3.4/site-packages
Requires: six

Manual Installation

The process is documented here.

Building a tree-view for xml data

Even though I am a long time Emacs user I only recently realized that I am really missing treeviews. There are numerous applications like getting an overview of a directory hierarchy  or structure of documents in xml format. It’s a nice thing to have and of course Emacs does not stand back and has something to offer:

To name a couple that immediatly show up. For the dyi people emacs brings an xml parser and a thing called „Tree-widget“ that allows for building of tree-views.  I gave these a try. And since the web does not seem to overflow with information about using tree widget I dump it here – hoping that other beginners might profit from it:

 

 
(setq xml "<post time=\"20050716234509\" id=\"010101\"><login><id>123</id></login><msg>A message.</msg><info>Information!</info></post>")
(setq xml-2 "<post><login><id>123</id></login><msg>Here is the message</msg><info>Information!</info></post>")
(setq xml-3 "<id>123</id>")
 
(setq root (with-temp-buffer
             (insert xml)
             (xml-parse-region (point-min) (point-max))))
 
(setq root-2 (with-temp-buffer
               (insert xml-2)
               (xml-parse-region (point-min) (point-max))))
 
(setq root-3 (with-temp-buffer
               (insert xml-3)
               (xml-parse-region (point-min) (point-max))))
 
(setq root-4 (with-current-buffer "some-more-complex.xml"
               (xml-parse-region (point-min) (point-max))))
 
(xml-node-children (car root-3))
 
(cdr (car (car (nthcdr 1 (car root)))))
 
(let* (
       (post (car root))
       (attrs (xml-node-attributes post)))
  attrs)
 
(require 'tree-widget)
 
(defun xml-to-tree-widget (xml)
  (interactive)
  (cond 
   ((stringp xml)
    (widget-convert 'item :tag xml))
   ((listp xml)
    (let ((attributes (xml-node-attributes xml))
          (attrib-widgets nil)
          (children (xml-node-children xml))
          (current-node))
      (progn
        (when attributes
          (setq attrib-widgets (mapcar (lambda (prop)
                                         (widget-convert 'item
                                                         :tag (format "%s=%s" (symbol-to-string (car prop)) (cdr prop))))
                                       attributes)))
        (setq current-node (widget-convert 'tree-widget 
                                           :tag (symbol-to-string (car xml))
                                           :args (append (if children 
                                                             (mapcar (lambda (node)
                                                                       (xml-to-tree-widget node))
                                                                     children)
                                                           nil)
                                                         attrib-widgets)))
        current-node ) ) ) ) )
 
(xml-to-tree-widget (car root))
 
(xml-to-tree-widget (car root-3))
 
(xml-to-tree-widget "text")
 
(defun test-tree-widget ()
  (interactive)
  (with-current-buffer (get-buffer-create "*tree-widget-test*")
    (erase-buffer)
    (setq-local my-tree-widget (widget-create (xml-to-tree-widget (car root-4))))
    (switch-to-buffer (current-buffer))))
 
;; This one will show the tree-widget in an empty buffer.
(test-tree-widget)

It’s not yet a piece of code that can be (use-package)ed or even (require)d 🙂 I entered the code in emacs and then „C-x C-e“ where necessary. When the initial xml data contains spaces (e.g. between a closing and the next opening tag) the resulting „tree“ contains empty lines, too.

Emacs Gems: scroll-all-mode

An Emacs Gem is one of the many little tools/functions/modes that Emacs ships with and that turned out to be ‚extremely usefull‘ in a specific situation.

Introduction

If you ever want to compare two buffers line by line and are too lazy to fire up ediff or don’t use ediff at all you might end up with a google search for „Emacs scroll windows simultaniously“. The top results will then inform you that Emacs brings a solution for this task: scroll-all-mode.

scroll-all-mode

Scroll-all-mode does not need lots of explanation and if I am not mistaken the official Emacs Manual dedicates a single line of documentation (which in fact suffices):

Scroll All mode (M-x scroll-all-mode) is a global minor mode that causes scrolling commands and point motion commands to apply to every single window.

Every single window in the selected frame one might add – but that’s it. For a quick comparison it’s definetly a great tool and it comes with Emacs. No need for installation (it’s not a package). No need for a command-window (it’s not ediff).

Neotree for Emacs

Neotree is a package for emacs (available for example on elpa) that displays the directory tree in a themeable tree-buffer. It looks nice but I was looking for a feature that I was missing. A window configuration I frequently use looks like this:

+-------+------------------+
|A      |B                 |  A - neotree
|       |                  |  B - some file
|       |                  |
|       |                  |
|       |                  |
+-------+------------------+

Now I am expecting that when I change the buffer in window B I have neotree in window A also show the buffers file. So I rolled up my sleeves and entered following lines of Emacs Lisp into my init.el:

  (defun mp:neotree-updater ()
    "Hook run on buffer list update."
    (interactive)
    (when (eq 2 (length (window-list)))
      (let* ((wnd-0 (nth 0 (window-list)))
             (wnd-1 (nth 1 (window-list)))
             (buf-0 (window-buffer wnd-0))
             (buf-1 (window-buffer wnd-1))
             (neo-buf nil)
             (other-buf nil)
             (filename nil))
        (when (or (eq buf-0 neo-global--buffer)
                  (eq buf-1 neo-global--buffer))
          (progn
            (if (eq buf-0 neo-global--buffer)
                (setq neo-buf buf-0
                      other-buf buf-0)
              (setq neo-buf buf-1
                    other-buf buf-0))
            (setq filename (buffer-file-name other-buf))
            (when filename
            (progn
              (when (file-exists-p filename)
                (message (concat "New filename " filename))
                (setq mp:neotree-go-to-dir filename)))))))))
 
  (add-hook 'buffer-list-update-hook 'mp:neotree-updater)
 
  (defun mp:neotree ()
    (interactive)
    (if mp:neotree-go-to-dir
        (progn
          (neotree-find mp:neotree-go-to-dir)
          (setq mp:neotree-go-to-dir nil))
      (neotree)))

Not perfect, but this way I can bind mp:neotree to „C-c n“ and have neotree jump to the current file when I hit „C-c n“. Since the mp:neotree-updater function is called in buffer-list-update-hook I was running into several recursions until I realized I can set buffer-list-update-hook temporarily to nil and so came up with this function:

(defun mp:neotree-updater ()
    (when (eq 2 (length (window-list)))
      (let* ((wnd-0 (nth 0 (window-list)))
             (wnd-1 (nth 1 (window-list)))
             (buf-0 (window-buffer wnd-0))
             (buf-1 (window-buffer wnd-1))
             (neo-buf nil)
             (other-buf nil)
             (neo-wnd nil)
             (other-wnd nil)
             (filename nil)
             (neo-buffer (get-buffer " *NeoTree*")))
        (when (and neo-buffer
                   (or (eq buf-0 neo-buffer)
                       (eq buf-1 neo-buffer)))
          (progn
            (if (eq buf-0 neo-buffer)
                (setq neo-buf buf-0
                      other-buf buf-1
                      neo-wnd wnd-0
                      other-wnd wnd-1)
              (setq neo-buf buf-1
                    other-buf buf-0
                    neo-wnd wnd-1
                    other-wnd wnd-0))
            (when (not (eq wnd-0 neo-wnd))
              (progn
                (setq filename (buffer-file-name other-buf))
                (when (and filename
                           (file-exists-p filename))
                  (progn
                    (let ((buffer-list-update-hook nil))
                      (neotree-find filename)
                      (select-window other-wnd)))))))))))

I have to take care for the case when I actually want to „C-x o“ into the neotree window, otherwise point will always jump out of the window with (select-window other-wnd).

Writing Java code with Emacs

Writing Java sourcecode in Emacs is a rather hard task. Emacs does not usually support the developer with lots of context awareness (friendly put…). What does the unsatisfied developer do when he is in need for functionality? He codes his own solution! So I have just started work on „Emacs Java Coding Extension (version 0.0.1)“. The project is admittedly in a very early phase. But there is an ambitious roadmap and some lines of code already available. Fresh from the scratch buffer where I tried it out:

(defvar mp:ac-classpath-cache nil)
 
(defun mp:ac-classpath-init ()
  (setq mp:ac-classpath-cache (mp:read-classes-from-jar)))
 
(defvar ac-source-classpath
  '((prefix . "^import \\(.*\\)")
    (init . mp:ac-classpath-init)
    (candidates . mp:ac-classpath-cache)))
 
(defun mp:read-classes-from-jar ()
  (with-temp-buffer
    (call-process "/usr/bin/unzip" nil t nil "-l" "/home/map/opt/jdk1.8.0_101/jre/lib/rt.jar")
    (goto-char (point-min))
    (let ((end 0)
          (result '())
          (classname ""))
      (while (search-forward ".class" nil t nil)
        (end-of-line)
        (setq end (point))
        (beginning-of-line)
        (goto-char (+ (point) 30))
        (setq classname (substring 
                         (replace-regexp-in-string "/" "."
                                                   (buffer-substring-no-properties (point) end))
                         0 -6))
        (setq result (cons classname result))
        (forward-line 1)
        (beginning-of-line))
      result)))

Put together correctly the code supplies a auto-complete source that knows about classes from the java rt.jar file (or any other jar file, or with some modifications several jar files).

Unittesting with QUnit

I have another toy project that i use to develop in my free time (No, I do not have too much free time. I just have started a couple of minor projects in the past and have not yet written about it). It’s a javascript based graph drawing tool. It turned out that i had introduced a couple of bugs and only then wanted to have Unittests. So I looked around for unittesting frameworks for Javascript and found QUnit.

QUnit

QUnit is a powerful, easy-to-use JavaScript unit testing framework.“ says the official homepage (link). It actually is easy to get used to and offers what one can expect from a unit testing framework (which I consider to be a nice test runner and a couple of asserts). The framework is quickly set up and first tests are written soon. The question I asked myself was „If I do this again tomorrow, the day after tomorrow and every day for the next 2 years – how can I simplify my usage of this powerful framework.

And this is where Emacs jumps in!

Emacs

So currently the situation is like this: I am editing some javascript file and realize „Time for Unittests“. This is what i came up with:

(defun indent-buffer ()
  (interactive)
  (indent-region (point-min) (point-max)) )
 
(defun mp/html-post-processing ()
  "This method looks for a couple of key-strings and replaces them with some meaningful values."
  (save-excursion
    (goto-char (point-min))
    (when (re-search-forward "%TITLE%" nil t)
      (replace-match (replace-regexp-in-string (regexp-quote ".html") "" (buffer-name) 'fixedcase)))
    (goto-char (point-min))
    (when (re-search-forward "%CSSFILE%" nil t)
      (replace-match (replace-regexp-in-string (regexp-quote ".html") ".css" (buffer-name) 'fixedcase) 'fixedcase))
    (when (re-search-forward "%TESTEE%" nil t)
      (replace-match (replace-regexp-in-string (regexp-quote "-test.html") ".js" (buffer-name) 'fixedcase) 'fixedcase))
    (when (re-search-forward "%UNITTESTS%" nil t)
      (replace-match (replace-regexp-in-string (regexp-quote "-test.html") "-test.js" (buffer-name) 'fixedcase) 'fixedcase))))
 
(define-auto-insert '("\\.html\\'" . "HTML5 Skeleton for QUnit-Unittests")
  [ '(nil
      "<!DOCTYPE html>\n"
      "<html>\n"
      "<head>\n"
      "<meta charset=\"UTF-8\" />\n"
      "<title>%TITLE%</title>\n"
      "<script src=\"jquery-3.1.0.js\"></script>\n"
      "<script src=\"jquery-ui-1.12.0.js\"></script>\n"
      "<script src=\"subrx.js\"></script>\n"
      "<link rel=\"stylesheet\" href=\"%CSSFILE%\" />\n"
      "</head>\n"
      "<body>\n"
      "</body>\n"
      "</html>\n" )
    indent-buffer 
    mp/html-post-processing ] )
 
;; with the nifty auto-insert defined above opening a buffer for the unit test is actually as
;; simple as a call to find-file!
(defun mp/qunit-test-for-current-buffer ()
  (interactive)
  (let ((test-file-name (replace-regexp-in-string (regexp-quote ".js") "-test.html" (buffer-name))))
      (find-file test-file-name)))

I found this very convincing and currently do it exatcly this way! I hope everybody who shares this snippet enjoys it, too!

Getting involved in Genisys development

Genisys is a Minecraft server written in PHP. It runs fine on Linux and forked from the Pocketmine codebase some time ago in the past. My attention came to Genisys after running Pocketmine for some time and constantly found myself in the situation, that after Minecraft Pocket Edition (which is my minecraft of choice) became updated Pocketmine did not allow the client to connect. Researching the topic in the Pocketmine forums revealed that there are a couple of people saying that Pocketmine is actually not further developed with enough manpower to keep on track with mcpe updates and that Genisys is more active developed. Since i did not know about Genisys and was not so happy with the disconnections I was facing I gave Genisys a try.

After installing Genisys and having ot up and running current clients could definetly connect to the server. That found out and beeing a developer on my own I want to do my part in sparing Genisys the fate of her „father“ project and at least start reading her source code so that maybe one day I can contribute to her development!

I share my experience here to easy the steps just in case someone is in the same situation like me: Here we go.

Getting Started

I just present some steps to get a private development environment.

Check out the source code

Genisys is available on Github under URL https://github.com/iTXTech/Genisys. So the first step is to clone the github repository:

$ git clone https://github.com/iTXTech/Genisys.git

The next step is get a php7 interpreter. Available here
To start the server cd into your genisys folder and execute the start script.

$ PHP_BINARY=/path/to/php7 ./start.sh

Php7 interpreters do not enjoy wide spread availability it seems. At least it is not available in the package repositories of my distribution of choice (Linux Mint Rosa). I managed to find a binary distribution but apparently lost the url… To see the server in action is quite fun since it works pretty nice. To proceed the development journey it’s a good idea to have some documentation at hand. Apparently the genisys github repository contains only a Doxygen file. This allows at least for creation of nice html formated source files.

Run doxygen

Doxygen creates a set of files that present the source code. To get it you need doxygen installed (I am on Mint Linux and had it already installed).

$ doxygen Doxyfile

Doxyfile resids in the Genisys root folder and hosts all the information necessary for doxygen to create a folder called „html“ filled with all the source-code in html format and more (like dependency trees for classes).

Start reading some source code

This is the tricky part and for me currently work in progress. Beeing a minecraft player i found it convincing to start with what i see when i play and find the classes that seem responsible/involved in the source code. So to stay i started my browsing the source-code journey with the namespaces:

  • block
  • item
  • entity

My current impression goes like this: Block involves all the stuff that is not moving (Air, Dirt, Door, Fence, Wine, Water) normally. Entity on the other hand involves moving members of the world (Monsters, Rabbits, Humans, etc). And last the item namespace involves all the stuff that can be picked up and overlaps with the block namespace (for example there is a potato in both namespaces…).

I know there is not much to take with from here but as i noted it is work in progress!

Official Sites

There is an online version of the Genisys doxygen site: https://storage.googleapis.com/itx-technologies-141911.appspot.com/docs/genisys/index.html and since Genisys sourcecode is hosted on github there is a list of issues available: https://github.com/iTXTech/Genisys/issues

Self signed X.509 certificates with openssl

I had to deal with S/MIME-certificates a lot recently. Even though i conecptionally do know about public key cryptography and certificates I rarely had to deal with openssl. So i took some time and worked through this tutorial by jamielinux.com about creating a private public key infrastructure comprising of a root ca and a intermediary ca. I accompanied my reading with the writing of a shell script that basically is the extraction of the shell script included in the tutorial. To make it run you should also download this file. It consists of the two openssl configuration files mentioned in jamie linux‘ tutorial. Fire up the script and you get a root ca and three intermediary ca’s.

#!/bin/sh
 
#
# Author: Matthias
#
 
set -x
 
BASE=$(pwd)
ROOT_DIR=${BASE}/PKI_0/ROOT
ROOT_KEY=private/ca.key.pem
ROOT_CERT=certs/ca.cert.pem
KEYSIZE=2048
ENCRYPT=
#ENCRYPT=-aes256
 
mkdir -p ${ROOT_DIR}
cd ${ROOT_DIR}
mkdir certs crl newcerts private
chmod 700 private
touch index.txt
echo 1000 > serial
cp -v ${BASE}/resources/openssl-root.cnf openssl.cnf
sed -i -e "s#%ROOT_DIR%#${ROOT_DIR}#" openssl.cnf
 
# create the root key
 
echo "#"
echo "# Creating encrypted key-pair"
echo "#"
 
openssl genrsa ${ENCRYPT} -out ${ROOT_KEY} ${KEYSIZE}
chmod 400 ${ROOT_KEY}
 
# create certificate from root key and personal data
 
echo "#"
echo "# Creating certificate"
echo "#"
 
openssl req -config openssl.cnf \
        -key ${ROOT_KEY} \
        -new -x509 -days 7300 -sha256 -extensions v3_ca \
        -out ${ROOT_CERT} \
        -subj "/C=DE/ST=NRW/L=Aachen/O=Private/OU=NA/CN=Ernie/emailAddress=ernie@sesamstreet.com"
 
#
# Verify the root certificate
#
 
echo "Do you want to verify the certificate (y/n)?"
read -n 1
 
if [[ ${REPLY} == "y" ]] ; then
    echo "#"
    echo "# Verifying certificate"
    echo "#"
    openssl x509 -noout -text -in ${ROOT_CERT}
fi
 
for INDEX in 0 1 2 ; do
 
    INTERMEDIATE_DIR=${BASE}/PKI_0/INTERMEDIARIES/${INDEX}
    INTERMEDIATE_KEY=${INTERMEDIATE_DIR}/private/intermediate.key.pem
    INTERMEDIATE_CSR=${INTERMEDIATE_DIR}/csr/intermediate.csr.pem
    INTERMEDIATE_CERT=${INTERMEDIATE_DIR}/certs/intermediate.cert.pem
 
    mkdir -p ${INTERMEDIATE_DIR}
    cd ${INTERMEDIATE_DIR}
    mkdir certs crl csr newcerts private
    chmod 700 private
    touch index.txt
    echo 1000 > serial
    cp -v ${BASE}/resources/openssl-intermediate.cnf openssl.cnf
    sed -i -e "s#%ROOT_DIR%#${INTERMEDIATE_DIR}#" openssl.cnf
    echo 1000 > ${INTERMEDIATE_DIR}/crlnumber
 
    # create the intermediate key
 
    echo "#"
    echo "# Creating encrypted key-pair"
    echo "#"
 
    cd ${ROOT_DIR}
    openssl genrsa ${ENCRYPT} -out ${INTERMEDIATE_KEY} ${KEYSIZE}
    chmod 400 ${INTERMEDIATE_KEY}
 
    # create certificate from root key and personal data
 
    echo "#"
    echo "# Creating intermediate certificate"
    echo "#"
 
    cd ${ROOT_DIR}
    openssl req -config ${INTERMEDIATE_DIR}/openssl.cnf \
            -new -sha256 \
            -key ${INTERMEDIATE_KEY} \
            -out ${INTERMEDIATE_CSR} \
            -subj "/C=DE/ST=NRW/L=Aachen/O=Private/OU=NA/CN=Dummy Intermediate ${INDEX}/emailAddress=intermediate_${INDEX}@dummy.local"
 
    cd ${ROOT_DIR}
    openssl ca -config openssl.cnf -extensions v3_intermediate_ca \
            -days 3650 -notext -md sha256 \
            -in ${INTERMEDIATE_CSR} \
            -out ${INTERMEDIATE_CERT}
 
    chmod 444 ${INTERMEDIATE_CERT}
 
    # Verify the interemediate certificate
    #
 
    echo "Do you want to verify the intermediate certificate (y/n)?"
    read -n 1
 
    if [[ ${REPLY} == "y" ]] ; then
        echo "#"
        echo "# Verifying certificate"
        echo "#"
        openssl x509 -noout -text -in ${INTERMEDIATE_CERT}
        openssl verify -CAfile ${ROOT_CERT} ${INTERMEDIATE_CERT}
    fi
 
done
 
################
#              #
# End of file  #
#              #
################

If you are still reading your are propably very interested in certificates and s/mime. I have one more thing to share – I have collected a couple of information about related rfcs and some terminology. The information is partly copied from a stackoverflow answer. Basically a list of who is who in s/mime related rfc’s. Click here to go there.

Thematisch nicht näher bestimmte Gedankenschnippsel