LibreOffice under KDE crashes on save

I kept thinking this would be fixed, I finally had to look up a workaround. When using KDE, LibreOffice simply crashes when attempting to save a new document (at least in Writer). This has to be the worst possible bug, you work on the document get it to the point you want it, then lose everything and it has been happening for quite some time.

The bug is apparently in KDE's dialog boxes and can be worked around by selecting "Use LibreOffice Dialogs" under Tools->Options->General.

Found on askubuntu

When to use attributes in XML

Its not rocket science but I guess I thought there was some general school of thought on this subject.
But I like this idea from w3schools which is to basically avoid attributes except for possibly an ID field.

rename command in Arch

I couldn't figure out why my rename command wasn't working in Arch, and it slowly dawned on me that the rename command in Arch doesn't support regular expressions... making it almost completely useless.
After some searching I found this blog entry that confirmed my suspicions and pointed me to an AUR package that basically implements rename with regular expressions.

$ packer -S rename

$ renamexm -s/\ /_/g *

Replace spaces with underscores in all files.

Comparing contents of files

Apparently the diff command can be used for comparing the contents of two folders.
This LinuxCommando post gives the details.

$ diff folder1/ folder2/

Indenting code with vim

This stackoverflow post has a wealth of information about indenting, and particularly indenting code blocks in vim.
Though the block indent stuff doesn't seem to work with shell scripts, the re-indent entire buffer command does:

gg=G

Bash snippets

I guess I will continue my current trend of having a single post be a clearing house for tips relating to a specific language.

 Math

$ x=0; let x+=3; let x-=1; echo $x
2

Generate range of numbers
$ for i in {1..5}; do echo -n $i; done
12345

Alternatively
$ for i in $(seq 1 5); do echo $i; done
12345

Iteration, etc...

Change password expiration data

$ chage -d now user

Printf

$ printf  "%02d %02d\n" 5 6
05 06

Echo list of folder names

$ echo */

home/ Desktop/ ...

Iterating through a file line by line

IN_FILE=$1
OUT_FILE=$2
<!-- Add a generated password to end of line -->
while read s; do
    echo "$s, `apg -n 1`" >> $OUT_FILE
done < $IN_FILE

Or as a one liner (even works if lines have spaces)

$ while read s; do echo "$s"; done < broke.txt

Dereference variables

$ var1='some text'
$ var2='var1'
$ echo ${!var2}
some text

String and array manipulation

From tldp.org we have two articles covering similar things named Issue18 and string-manipulation:

Creating an array and indexing into arrays:

$ arr=(hi there dude)
$ echo "Middle value: ${arr[1]}"

Middle value: there
$ echo ${arr[*]}
hi there dude

Indexing into strings: ${variable_name:start_position:length}

$ var="Hello" && echo ${var:1:3}
ell

Separate a path and its base filename.

$ FILE="/some/file/with/path.txt"
$ basename $FILE
path.txt
$ dirname $FILE

/some/file/with

Another way to do this (the first doesn't always seem to work)

$ echo "${FILE##*/}"
path.txt

 I use this in scripts within a usage() function to print the name of the script

$ usage() { echo "Usage: ${0##*/} <options>" ... }

JavaScript Snippets

A repository of nice finds for the JavaScript language.

JavaScript Collection

The top answer on this stackoverflow question has an amazing set of  JavaScript examples and explanations of various things as well as why you should use certain methods over other methods.

Objects

Create a JavaScript object, code from this blog

var date = {
    year: 2013,
    month: 3,
    day: 3,
    getDate: function() {
        return ""+ this.year + " " +  this.month + " " + this.day;
    },
};

alert(date.getDate());

Bottom

CODE

PHP Snippets

Output

This really improves the look of data arrays.

print "<pre>";
print_r($data_array);
print "</pre>";

GIT Snippets

I'm gonna just put all my handy git snippets in one place (right here) eventually

Git Configuration

Tell git to stop replacing my LFs with CRLFs

Git likes to replace Unix CRLFs with Windows CRLFs, this is really annoying, and is apparently the default now. The following fixes it, but it has to be done in every repository where you want to disable this. The global config file is in ~/.gitconfig

Make the change global (probably what you want for Linux).

$ git config --global core.autocrlf input

Per repository changes.

$ git config core.autocrlf         # check the existing value of autocrlf
true
$ git config core.autocrlf false   # change it to false

Other useful commands

$ git config <--global> --list     # Show config settings
$ git config <--global> -e         # Edit config settings

Retrieve previous revision of a file

First determine the revision you are interested in:

git rev-list -n num_results HEAD -- file

Retrieve the file associated with the revision:

git show revision:file

Example:

$ git rev-list -n 1 HEAD -- image.png299cf09892f7af2f9433782bb14a096547f6d9aa$ git show 299cf09892f7af2f9433782bb14a096547f6d9aa:./image.png > temp.png

Discard local changes
DANGER!!! I did this, and despite the option file name, it discarded ALL changes made! F#$@$@

$ git checkout HEAD <optional file name>
$ git checkout .

Show the remote URL

git remote -v

Show the remote URL with lots of additional info (requires ability to login to remote site)

git remote show origin

Change remote URL

git remote set-url origin  git@GIT_HUB_LAB_etc.com:USERNAME/REPO.git

Copy a file from other_branch to the current branch

git checkout other_branch probstmt.tex

Merge branch_x into current branch

git merge branch_x

Copy an branch over another without merging (for the most part)

git rebase branch_x

Several ways to show all files ever deleted from a project

git log --diff-filter=D --summary

git log --diff-filter=D --summary | grep delete

git log --all --pretty=format: --name-only --diff-filter=D | sort -u

To see files that have been deleted in the working copy

git ls-files --deleted

Restore deleted files (from a previous post)

Find the last commit that affected the given path. As the file isn't in the HEAD commit, this commit must have deleted it.
git rev-list -n 1 HEAD -- <file_path> 

Then checkout the version at the commit before.
git checkout <deleting_commit> -- <file_path or name> 

Or in one command, if $file is the file in question.
git checkout $(git rev-list -n 1 HEAD -- "$file")^ -- "$file"

Python snippets

Python is full of cool shortcuts that I can't seem to remember when I need to, so I'm gonna document them here.

Default dictionary

This can be used to avoid having to check for existence of keys by providing default values.          
from collections import defaultdict

    from collections import defaultdict
    counter = defaultdict(int) # Default to integer set to 0
    for akey in alist:
        counter[akey] += 1

Or one to many lists.

    from collections import defaultdict

    onetomany = defaultdict(list)
    for akey, anitem in acollection:
        onetomany[akey].append(anitem)

Output Latex Formats

print r"\begin{tabular}{|l|l|}"
print r"\hline"
print r"MAC & IP\\"
print r"\hline"
for snd,rcv in ans:
    print rcv.sprintf(r"%Ether.src% & %ARP.psrc%\\")
print r"\hline"
print r"\end{tabular}"

List Comprehensions

Convert a string to an array of ASCII characters

[ord(ch) for ch in "Hallo"]

Nested(?) access to list
in: r = [1,2]
in: [ (x, y) for x in r for y in r ]
out: [(1, 1), (1, 2), (2, 1), (2, 2)]

Create various shapes and sizes of sets of points

from random import random as rnd
_2x1 = [rnd() for _ in range(2)]
_3x2 = [[rnd() for _ in range(2)] for _ in range(3)]
_4x3x2 = [[[rnd() for _ in range(2)] for _ in range(3)] for _ in range(4)] 
ThreePoints = [[rnd(),rnd()] for _ in range(3)]

Strange Objects

Evaluate completely dynamic code

a=5
b="a*a"
eval(b)
25

Language Features

Built in list iteration

# x is a list of integers.

In : x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

In : x[::1]  # Every item

Out: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]In : x[::2]  # Get every 2nd item

Out: [0, 2, 4, 6, 8]In : x[::3]  # Every 3rd item

Out: [0, 3, 6, 9]

# Backwards

In : x[::-1]Out: [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]In : x[::-2]Out: [9, 7, 5, 3, 1]

Decorators to dynamically add functionality to class instance from StackOverflow

class Pizza(object):
    def __init__(self):
        self.toppings = []
    def __call__(self, topping):
        # Use '@instance_of_pizza' before a function def and the function gets passed onto 'topping'
        self.toppings.append(topping())
    def __repr__(self):
        return str(self.toppings)
myPizza = Pizza()
@myPizza
def cheese():
    return 'cheese'
@myPizza
def sauce():
    return 'sauce'
print myPizza
# ['cheese', 'sauce']

Iterators allow objects to work with for statements, require 'next' method

class PizzaFactory(object):
    def __init__(self):
        self.orders = {1:'ted", 2:"joe"}
    def __iter__(self):
        return OrderIterator(self)

class OrderIterator(object):

    def __init__(self, container):

        self.items = container

        self.index = 0

    def next(self):

        if self.index >= len(self.items): raise StopIteration

        ret = self.items[self.index]

        self.index += 1
        return ret

# pf = PizzaFactory()
for order in pizza: print order
1:ted
2:joe

Generators allow function to return intermediate values

def fib(n):

  a = 0

  yield a

  b = 1

  yield b

  while b < n:

    a, b = b, a + b

    yield b

# for e in fib(5): print e

0

1

1

2

3

5

Ternary operator

def factorial(n):
  return 1 if n == 0 else n * factorial(n - 1)
print factorial(5)

Generating core files from systemd-coredumpctl

Arch Linux stores core dumps in the journal, so if you want to debug a core dump you first have to dump the core using systemd-coredumpctl
Examples (you may have to be root if not in the systemd-journal group):

List core dumps

~ $ systemd-coredumpctl

Dump to file named core based on pid

~ $ systemd-coredumpctl dump 6738 -o core

Dump to file named core based on executable name

~ $ systemd-coredumpctl dump a.out -o core

Magic Computers

There was a time when computers seemed like pure magic. That is why I became so fascinated by them back in the early 1980s. The mystery surrounding how they worked was simply incredible and made me think anything was possible within their intangible boundaries.

I will never forget a day when my mother took me to a small computer shop in our small town and the proprietor sat me down in front of an Apple ][ running a game called Zork. I had no idea what it was or what I was supposed to do, but I typed something and it responded. To this day I can remember the angst I felt at having to get up and leave after what must have been half an hour of exploring.

I remember countless all nighters spent trying to get assembly language programs working on my 8088 "portable" computer. Writing data directly into the CGA video card or sending commands directly to the hard drive controller to move the read/write head made me feel as if I was exploring the moon.

I really miss those days, there is still much that I don't know, more than ever really. But sadly the combination of 1) a good understanding of how a computer works, 2) incredibly complex operating systems that have taken away the low level control we had with DOS and 3) the loss of youth have combined to robbed me of the sense of magic that computers came with in the 1980s.

Relative root issues with gitlab

StackOverflow sure seems to get a lot of good answers. This entry addresses an issue I've been struggling with for some time related to setting relative roots in ruby on rails applications.

I'm using some Turnkey Linux templates with ProxMox and one is gitlab, but changing the relative root I was still missing some icons when loading the gitlab pages.

According the the post above after updating the config files I needed to go into the gitlab home folder and run:

bundle exec rake assets:precompile RAILS_ENV=production RAILS_RELATIVE_URL_ROOT=/gitlab

This took several minutes to complete (that's how you know its working?) but fixed my problem.

Pacman and "exists in filesystem" errors

Every now and then I try to update and am rebuffed because some file already exists and would be overwritten.

To find out what package owns the file: pacman -Qo /path/to/file

To install anyway and overwrite the existing file(s): pacman -S --force $package

Each installed package has a folder under /var/lib/pacman/local/ with metadata about the package.

Some ProxMox commands.

I've figured out how to bulk clone VMs in Proxmox which is very useful.

qm clone $VM_ID $CLONEID -name "$NAME" -pool "$POOL"

There are several other things that are useful dealing with clusters at http://wmunguiam.blogspot.com/2013/02/remove-node-cluster-forever-on-proxmox.html

List nodes:

pvecm nodes

Remove cluster node:

pvecm delnode node_name

But this may fail if you don't have a "quorum", in which case you change the number of nodes voting:

pvecm expected 1

Then delete the node.

Reverse DNS Lookup Zone File and Whitespace

I have been struggling to get my reverse look up working on my DNS (bind) server for a LONG time... The problem I finally figured out:

$TTL 3D
@       IN      SOA     ns.linux.bogus. hostmaster.linux.bogus. (
                        199802151 ; Serial, todays date + todays serial
                        8H      ; Refresh
                        2H      ; Retry
                        4W      ; Expire
                        1D)     ; Minimum TTL
                NS      ns.linux.bogus.

; NO WHITESPACE!!!!
    1               PTR     gw.linux.bogus.
    2               PTR     ns.linux.bogus.
    3               PTR     donald.linux.bogus.
    4               PTR     mail.linux.bogus.
    5               PTR     ftp.linux.bogus.

See the spaces in front of the 1, 2, etc...? Those cannot be there for whatever reason.

Is Prism a reverse hoax?

I always figure things out before everyone else, but I never tell anyone about it. This time is going to be different.

Which is more likely:

1. Our government actually thinks they could keep a monstrous scam like Prism a secret from the masses?


2. Or the possibility that our government is willing to take a black eye PR wise in order to make bad guys *think* that they are being monitored therefore making their lives more difficult with relatively little effort?

I would actually be pretty impressed if the latter were to be the case. I'm way too cynical to believe it, but wouldn't that be great to find out that our government is actually doing all this to scam the terrorist?

This is similar to what I have heard said that Ronald Regan did to the Soviets with the Star Wars project. Made them think we were dumping tons of research into this project to the point they couldn't compete and they tore down that wall.

Here's hoping.

Restore previous version of a file from GIT repository

From the surprisingly helpful stackoverflow.com, regarding how to restore a file from GIT:

Find the last commit that affected the given path. As the file isn't in the HEAD commit, this commit must have deleted it.
git rev-list -n 1 HEAD -- <file_path>

Then checkout the version at the commit before.
git checkout <deleting_commit>^ -- <file_path>

Or in one command, if $file is the file in question.
git checkout $(git rev-list -n 1 HEAD -- "$file")^ -- "$file"

VLC Fine Speed Control shortcut keys

I'm always forgetting this. In VLC if you want to increase/decrease playback speed use + or -, but that only works in big chunks. For smaller increments use [ and ].

Generating entropy quickly

I wrote a script to generate passwords for users, but it runs very slow at times. The solution was to install rng-tools.

From http://www.101tech.net/2011/11/01/apg-automated-password-generator-runs-slow-on-centos/

# apt-get install rng-tools
# echo "rngd -r /dev/urandom -o /dev/random -t 3" >> /etc/rc.local
# rngd -r /dev/urandom -o /dev/random -t 3

EDIT:
After reading this posting by Theodore Ts'o maybe using this isn't such a great idea as it is basically a bridge to the possibly compromised RNG instruction in the Intel processor.

Bash one liner

I can't believe it took me this long to figure this out but an easy way to do one liners in bash for multiple files:

~$ for name in *.pdf; do pdftotext "$name"; done

Password considerations

A student forwarded me this article: http://arstechnica.com/security/2013/08/thereisnofatebutwhatwemake-turbo-charged-cracking-comes-to-long-passwords/

I have long wondered why there is so much confusion over choosing strong passwords. I mean, I don't always do it, but its not that difficult to figure out. First of all I never use ridiculous garbage passwords such as "aJKEA43%@#5289sl2" that people like to use. This just means there is a book somewhere full of passwords that has to be accessible to various people.

Passwords are made of tokens. These tokens are usually thought of as characters from the alphabet, numbers, etc... The strength of passwords is usually thought of in terms of length and number of tokens possible at each position. The length makes a lot more difference, so if your passwords consist of between 6 and 10 tokens and each token can be one of 100 characters (a-z, A-Z, 0-9, !@#$^&*()_+ etc....) then the number of possible passwords in that range is 100^10 - 100^5.

However if you use dictionary words such as "wooden" this will count as only a single token and not the 6 you intend. This is because smart cracking programs will not try everything such as "..., woodel, woodem, wooden" but will use dictionaries instead. By using dictionaries, attackers increase the token count significantly, but reduce the effective length of the password. So if your password is two words long (woodenhorse for instance), and there are 50,000 common words in the English language, it will take at most 50,000^2 guesses. That is not many for a computer.

Good passphrases

If you simply pin together words, each word becomes a token and as this practice becomes more common, its no safer than short random passwords. But this can be beefed up significantly by simply throwing in some entropy. For instance, if you have a space between two words, now your two words have to be tested both with and without a space. If you have two spaces, that has to be tested as well. Instead of spaces if you use percent signs, again things get less likely.

So basically you can make simple passphrases that are not simple to crack if you understand how password crackers think.

Instead of "magiccarpetride" (3 tokens) use intentional misspellings, channel your inner dyslexia, add extra characters, etc...

"magiccarpetride" -> "jamic. .KRPT. .ride"

Magic becomes jamic, words end with a period and begin with a period except on the ends, KRPT is carpet with no vowels and in upper case. This is still using tricks that could be tried by a computer, but there are so many possible variations of these types of tricks, that it quickly becomes untenable to crack passwords by trying all possible combinations of these types of things. For instance:

"magiccarpetride" -> "maAgi ccaArpe triIde  "

In this one, double the first vowel in a word, lower then upper, alternate spacing 2 spaces then one space and move the space from the end of the previous word to the beginning of the next.

The variations are infinite, replace the first letter of each word with xz, remove the last letter, use no spaces but add them all at the end, or the beginning, and yes even using numbers for letters or vice versa these things all add to the complexity but simply exchanging letters/numbers or putting 1 at the end is not enough.

Making Thunderbird use Chrome for opening URLs

I've had to look this up twice now, so time to post it here.

Thunderbird will use Firefox to open email links, even if Chrome is your default. The solution is to go into Preferences->Attachments and change the browser used for links, however this option may or may not be present. If not, you have to go into ~/.thunderbird/<whatever profile name>/ and replace the mimeTypes.rdf with the one from the link above.

After this start firefox, and the options should now exist in Preferences->Attachments.

Stashing code with GIT

Pretty cool, I didn't know what stash was, but "git help stash" showed this example:

You can use git stash to simplify the above, like this:

# ... hack hack hack ...
$ git stash
$ edit emergency fix
$ git commit -a -m "Fix in a hurry"
$ git stash pop
# ... continue hacking ...

(note: This example is plagiarized)

Managing SSH keys

I found a GREAT post on dealing with multiple SSH keys. This makes it much easier to securely use mulitple keys so a compromised systems doesn't compromise multiple servers.
In essence, you create a "config" file in .ssh and tell SSH what private key to use based on the name of the host you are SSHing to. You can also specify user and port. I also found you can specify multiple hosts as "Host abc or xyz"

~/.ssh/config

Host *.home.lan
  IdentityFile ~/.ssh/id_dsa.home
  User kbsingh

Host *.vpn
  IdentityFile ~/.ssh/id_rsa.work
  User karanbir
  Port 44787

Host *.d0.karan.org
  IdentityFile ~/.ssh/id_rsa.d0
  User admin
  Port 21871

I added aliases to my /etc/hosts file so I could differentiate between SSH running on mulitple ports at the same server (VM Server). Now I can SSH by the alias name from the hosts file and I don't have to specify the port number or user.

Undeleting files deleted from the shell

I was just finishing about 8 hours of python programming on a simulation, and I went to delete my output files so I could be sure I regenerated them. I typed the following command:

$ rm * save

Notice that space between the "*" and the word "save"? Yeah, that was not supposed to be there.
So I removed all my source files.

I searched my repository for undelete and found something called "extundelete" I installed it, CTRL+ALT+1'd, logged in as administrator and killed my X session. I then unmounted /home and ran:

$ extundelete /dev/sda7  --restore-directory baloney

baloney is the name of my home folder. And I'll be damned if it didn't recover all but the most recent of my modified files.
The developer is named "N E Case" on SourceForge, thanks N.

Overwriting local changes with Git

Often I want to delete a file and pull the original back from my git repository. To do this properly you have to checkout the file from the "HEAD" tag.

$ git checkout HEAD <optional file name>
$ git checkout .

Pulling cross origin data within JavaScript

It took me a lot longer to figure this out than I thought it would, and the syntax has to be just so, minor changes cause failure due to same origin policy.

Using JQuery this code will allow you to pull data from an external website.

$.ajax({
    url: 'http://domain.com/data-request.php?foo=bar',
    dataType: 'jsonp',
    jsonp: 'jsoncallback',
    timeout: 5000,
    success: function(data, status){
        //data loaded
    },
    error: function(){
        //error loading data
    }
});

Data must be formatted in JSON format. I can't remember the site I found this on, but it gives a lot of detail on what's happening and why.

Using the Linux screen command to detach processes from terminal

A nice simple example based page on how to use the screen command in Linux.

http://www.thegeekstuff.com/2010/07/screen-command-examples/

Allows you to detach and later re-attach to SSH sessions without killing your processes.

Start screen session.
$ screen ./work_on_something.sh
Press CTRL+A and then "d" to detach.

List screen sessions.
$ screen -ls
There is a screen on:
21952.pts-0.debian-7 (07/26/13 23:25:29) (Attached)
1 Socket in /var/run/screen/S-user.

If detached, attach with -r
$ screen -r 21952.pts-0.debian-7

Removing invalid proxmox VM listings

The proxmox datacenter view sometimes shows a VM after the VM has been deleted. This is appears to be a bug where the configuration file is not deleted.

To remove the listing from the control panel, on the proxmox server:

# rm /etc/pve/nodes/<server_name>/openvz/<vmid>.conf

Using your SSH connection for source control with GIT

Example of how to use SSH as a GIT repository http://git-scm.com/book/en/Git-on-the-Server-Setting-Up-the-Server very simple if you already have an SSH server set up.

Turnkey Linux Service Credentials

I have noticed that several Turnkey Linux containers say the password for logging in is "set at first boot", but have not been able to see any such thing happening.

I found a post that says to use "turnkey" as default password and I'll be darned if it didn't work.

Apache proxy to redmine server

I've been struggling with this for awhile. I have a virtual apache server, and turnkey linux container with redmine running and I want the main server to delegate requests to the /redmine/ folder to the redmine server.

I set up the proxy statements on the primary server, but all the followup requests for css, js, images whatever would be looking back in the main server's root again instead of on the redmine sever.

I had to add the following line to the end of /var/www/redmine/config/environment.rb

ActionController::Base.relative_url_root = "/redmine"

And that did the trick.... I have no idea how rails does whatever it does.

Tagging versions in GIT

As stated in the title, I use this blog to remember stuff, so here is yet another GIT tip.

Adding a tag for a given version:

$ git tag -a v1.4 -m 'my version 1.4'

Displaying tagged versions:
$ git tag
v0.1
v1.3
v1.4

Using awk to work on data files

In order to average numbers in a file we can use awk as so:

$ cat meyer-heavy.txt | awk '{ sum+=$1;count++ } END {print sum/count}'

To find the largest value:

$ cat meyer-heavy.txt | awk 'BEGIN{max=-9999} $1 > max {print $1; max=$1}' 

Improved version (only prints largest value, not intermediate ones): 

$ cat meyer-heavy.txt | awk 'BEGIN{max=-9999} $1 > max {max=$1} END{print max}'

To find the smallest value:

$ cat meyer-heavy.txt | awk 'BEGIN{min=9999} $1 < min { print $1; min=$1 }' 

Fixed version (doesn't assign blank lines as min value): 

$ cat meyer-heavy.txt | awk 'BEGIN{min=9999} /\-[0-9]+/&&$1 < min { min=$1 } END{print min}'

Where $1 is for column 1, use $2 for column 2 etc...

Grep and recursion

Grep has for a long time had a recursive option "-r" but I have never been able to get it to do what I want because if you write

$ grep -r test *.txt

The *.txt part specifies the files to look in, and since the folders I want to recurse don't end with .txt it doesn't recurse into them.
The correct way to cause grep to recurse is to specify file patterns separately.

$ grep -r test --include "*.txt" ./

Getting the correct default paper size in LibreOffice Writer

For years OpenOffice/LibreOffice has ticked me off by endeavoring to give me documents that after I walk down the hall to the printer, are asking for A4 sized paper, though I live in the US where we use "Letter" sized paper.

Even though I set the defaults for LibreOffice, and even though the printer and any printer settings are ALL set to Letter.

Well, I found this documentation of what is a really stupid way to have to make this change actually happen. Create a template, and set it as "default" and I guess this works.

From: https://help.libreoffice.org/Writer/Changing_the_Default_Template

1. Create a document and the content and formatting styles that you want.
2. Choose File - Templates - Save.
3. In the New Template box, type a name for the new template.
4. In the Categories list, select "My Templates", and then click OK.
5. Choose File - Templates - Organize.
6. In the Templates list, double-click the "My Templates" folder.
7. Right-click the template that you created, and choose Set as Default Template.
8. Click Close.

Now if I can only get Impress to stop setting my default language as Finnish...

Git tutorial

While on the subject of danielmiessler.com, I noticed his GIT tutorial which is really easy to read:
http://danielmiessler.com/study/git/

Using tcpdump

One thing I can never seem to remember is how to get tcpdump to show entire packets.

The option is -s (for snaplength). So either -s 1514 or -s 0 will cause tcpdump to capture entire packets.

More good info on using tcpdump is at danielmiessler.com/study/tcpdump/.

Apache reverse proxy to redmine

I had all kinds of issues trying to get this to work. I'm trying to proxy from my servers /redmine folder to a container VM running redmine. The problem has been that the container was running redmine in its root folder.

The solution was to tell the container to run redmine in a /redmine folder, from http://www.redmine.org/projects/redmine/wiki/HowTo_Install_Redmine_in_a_sub-URI

Simply had to add:
Redmine::Utils::relative_url_root = "/redmine"
to the end of Redmine's environment.rb and restart Apache

Recording test audio

Linux distros I have used have no built in record software which I have wanted in order to test microphone levels before creating screencasts.
In Arch, install the "alsa-utils" package, then you can record/play with the following utilities:

$ # Record 5 seconds of audio, then play it back.
$ arecord -d 5 test-mic.wav
$ aplay test-mic.wav

VMWare player in Arch Linux

I don't use vmware too often, I prefer VirtualBox, but when I went to start it, it couldn't find kernel headers, I found what looked to be a solution in the Arch wiki (https://wiki.archlinux.org/index.php/Vmware#VMware_module_patches_and_installation) but still had issues. I finally installed the package mentioned in the wiki "wmare-patch" from the AUR repositories and everything was fine.

I will simply need to remember to re run this program each time I have the issue.

# vmware-patch

Using netstat

I use

# netstat -pant4

all the time to see what is listening/connected to my computer.
I found some other useful commands I didn't know about at http://openmaniak.com/netstat.php

To display all the opened network sockets (extended informations):

#netstat -aute
-a: All
-u: UDP
-t: TCP
-e: Extended

To display the summary statistics for each protocol

#netstat -s
-s: Summary statistics for each protocol.

To display the extended interfaces statistics:

#netstat -ie
-i: Interface
-e: Extended information

Parsing columns in BASH

I sometimes need "cut" functionality that works for fields with more than a single space. For instance, I sadly have a script just to kill nepomukindexer because it constantly goes berzerk and uses all my 8gb of ram. 

So I need to parse the process IDs of all the processes that contain the word nepomukindexer (there are hundreds at times).

First try

I figured cut would do the trick.

ps -ef | grep nepomukindexer | cut -d " " -f 5 

This only works if there are exactly 4 spaces between the two fields, and that isn't always the case.

Second try

I still don't understand why sed doesn't seem to work with the "+" regular expression symbol the way I expect. The following seems like it should replace all instances of one or more spaces with a single space, but it does nothing.

ps -ef | grep nepomukindexer | sed 's/ +/ /g'

Edit: I just found a workaround for this. I changed this to

ps -ef | grep nepomukindexer | sed 's/  */ /g'
                                      ^^ two spaces

and it worked, the only difference being the matching part which says match a single space followed by 0 or more additional spaces, and replace the whole mess with a single space.

Third try

The solution I found is to use awk.

ps -ef | grep nepomuk | awk -F" " '{print $2}'

Using this command, I pipe the results through kill and all is well.

Blogilo take 2

I've had no luck at all with Blogilo over the past month or two. But I just noticed an item in the Arch AUR called "python-gdata-svn". After installing it, I got a test post to work, so this is my second try. If you see this, then this may be something that needs to be installed in Arch (64 bit anyway) in order for Blogilo to work.

Edit: Seems to work, even allowed me to update my original post. Though truthfully I can't say for sure that its because I installed python-gdata, but it did work immediately afterwards...

Non-blogilo-edit:
Apparently Blogilo is trying to upload some HTML elements that Blogger doesn't care for.
The problem returned so started Blogilo from command line and saw the following output:

blogilo(7501): Could not regexp the id out of the result: "The element type &quot;br&quot; must be terminated by the matching end-tag &quot;&lt;/br&gt;&quot;." 

So I switched to HTML view and removed the <div><br></div> but still had issues:

blogilo(7501): Could not regexp the id out of the result: "The element type &quot;br&quot; must be terminated by the matching end-tag &quot;&lt;/br&gt;&quot;."

But I didn't see any "&nbsp" so I quit and restarted and saw a hitherto invisible "&nbsp;" that I removed at which point the post worked. 

Blogging clients: Blogtk, Drivel

After the issues with Blogilo, I installed BloGTK and Drivel. I'm making this post from BloGTK. Setup was similar to that of Blogilo. The interface is pretty similar. There is a place for "tags" but it won't allow me to enter anything.

I have not been able to get "drivel" to successfully log in as of yet.

Disable wildcard expansion when calling BASH scripts

This is a rarely needed, but very useful item. I wanted to create a script that I could pass wildcards into. But bash will automatically expand wildcards before the script can get them, so running

$ myscript.sh *

will call myscript with a bunch of filenames instead of the "*" character.

The solution is to create an alias to disable globbing:

noglob_helper() {
    "$@"
    case "$shopts" in
        *noglob*) ;;
        *) set +f ;;
    esac
    unset shopts
}

alias noglob='shopts="$SHELLOPTS"; set -f; noglob_helper'

Now you can preface your script with the alias and the wildcards will not be expanded:

$ noglob myscript.sh *

This code is from Simon Tatham's home page (the author of PuTTY among other things) http://www.chiark.greenend.org.uk/~sgtatham/aliases.html

Outlook 2010 in Wine on Arch Linux

I have been trying for WAY too long to get Outlook to connect to my exchange account under wine. This is mostly because I just can't seem to find anything else that works, I've been using Thunderbird but Meeting invitations don't appear in my Inbox. My "new message count" shows I have new messages, but there is nothing in my Inbox and there seem to be no plugins that can get it to work properly, plus I don't seem to be be able to get the exchange calendar to work.

Anyway, I finally figured out why Outlook running on Wine would not connect... My smb.conf file was missing. So the solution was....

# cp /etc/samba/smb.conf.default /etc/samba/smb.conf

And everything worked perfectly. I figured this out by the way by starting Outlook from command line with a bunch of overly complex options and seeing an error message saying that /etc/samba/smb.conf was missing.

Permissions problems when using UserDir mod with Apache2

I have had trouble in the past getting Apache to work properly with the UserDir mod. I just installed a new server and had to solve the problem again. In the past I set ww-data (Apache's user) as the group owner for all user's home folders as well as public_html, which is a pain and feels wrong somehow.

Turns out the issue is that Apache needs executable access on all parent folders at the "everyone" level.

So anyway, the following command did the trick by setting the x bit at all levels on the home directories.

# chmod 711 /home/*

The downside of course is that someone can run applications in any other user's home folder if they know the name of the program and where it is stored.
This forum exchange tipped me off to the solution.

Add existing user to a group

I can never seem to remember how to do this without having to search. There are way to many programs that allow users and groups to be created/updated/modified.

Add an existing user to an existing group:

# usermod -a -G <group> <user>

Or for a list of groups:

# usermod -a -G <group1,group2,...> <user>

I found this at http://www.cyberciti.biz/faq/howto-linux-add-user-to-group/

Using pacman

From https://wiki.archlinux.org/index.php/Pacman_Tips

Finding what pacman installs

This is what I actually was looking for was how to show what pacman installs for a given package. I didn't find it on the wiki page, but at https://bbs.archlinux.org/viewtopic.php?id=89448

$ pacman -Qql packagename

Remove packages with dependencies

If a package has dependencies, pacman will not allow you to remove it saying it is required for the dependencies. Use the -Rdd option to remove it anyway and leave the dependencies. This allows upgrading versions etc...

$ pacman -Rdd packagename

Remove old cached packages with paccache (from package "pacman-contrib") 

Packages are cached in /var/cache/pacman/pkg/. By default "paccache -rv"  deletes all but the most recent 3 versions of packages, I have no idea why I would ever want to go back 3 versions of something. Adding -k n allows you to specify the number of versions. (this freed over a gigabyte for me, and I just installed arch about 2 months ago)

$ paccache -rv -k 1

Install from a package file

$ sudo pacman -U filename.pkg.tar.xz

Remove package along with unneeded dependencies and configuration files

$ sudo pacman -Rns packagename

Show detailed information on package

Installed package:

$ pacman -Qi packagename

From repository:

$ pacman -Si packagename

Remove orphans recursively

$ pacman -Rs $(pacman -Qtdq)

TinyOS, TOSSIM and the Yeti2 plugin for Eclipse

I've wrestled with this for quite some time. Everything worked except the tossim code kept showing as an error in Eclipse.

I had added the tossim folder as an include location, but it didn't seem to matter. I finally tried clicking on some of the settings and found that by left clicking you toggle through the options. So after I made the settings look as in the image, my TOSSIM errors finally went away.

NCC: Consider, default was ignore
Include: Source, default was never
Search: Recursive, default was flat

The ip command (iproute2) and virtual interfaces

I've never quite gotten the hang of using "ip" in place of ifconfig, but this blog post shows how to do several things I have wondered about.
The main thing I was looking for was how to add and delete a virtual interface which required some additional research.

The not at all obvious way to get help on this is:
# ip link add help
or
# man ip-link

Show status of interfaces

# ip ad
<... snipped loopback stuff ...>
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 5c:26:0a:2b:29:54 brd ff:ff:ff:ff:ff:ff
    inet 64.112.251.9/21 brd 64.112.255.255 scope global eno1
    inet6 fe80::5e26:aff:fe2b:2954/64 scope link
       valid_lft forever preferred_lft forever

Add a virtual interface

# ip link add type veth
# ip ad
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 5c:26:0a:2b:29:54 brd ff:ff:ff:ff:ff:ff
    inet 64.112.251.9/21 brd 64.112.255.255 scope global eno1
    inet6 fe80::5e26:aff:fe2b:2954/64 scope link
       valid_lft forever preferred_lft forever
17: veth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 9e:59:c1:cc:3a:e3 brd ff:ff:ff:ff:ff:ff
18: veth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 2e:5e:c1:b0:71:44 brd ff:ff:ff:ff:ff:ff

For some reason, it always adds two interfaces.

Set address for a virtual interface

# ip addr add 10.1.2.3/8 dev veth0
# ip ad
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 5c:26:0a:2b:29:54 brd ff:ff:ff:ff:ff:ff
    inet 64.112.251.9/21 brd 64.112.255.255 scope global eno1
    inet6 fe80::5e26:aff:fe2b:2954/64 scope link
       valid_lft forever preferred_lft forever
17: veth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 02:d1:8f:79:fe:db brd ff:ff:ff:ff:ff:ff
    inet 10.1.2.3/8 scope global veth0
18: veth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether ee:84:34:2c:ce:bc brd ff:ff:ff:ff:ff:ff
The link is still in the "down" state, and can't be used yet.

Set status of virtual interface to "up"

# ip link set up dev veth0
# ip ad
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 5c:26:0a:2b:29:54 brd ff:ff:ff:ff:ff:ff
    inet 64.112.251.9/21 brd 64.112.255.255 scope global eno1
    inet6 fe80::5e26:aff:fe2b:2954/64 scope link
       valid_lft forever preferred_lft forever
17: veth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 100
    link/ether 02:d1:8f:79:fe:db brd ff:ff:ff:ff:ff:ff
    inet 10.1.2.3/32 scope global veth0

Notice that the state is still listed as down, but ifconfig shows the same device with a state of up. Go figure.

# ifconfig veth0
veth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 10.1.2.3  netmask 255.255.255.255  broadcast 0.0.0.0
        ether 02:d1:8f:79:fe:db  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Remove virtual interface

# ip link delete dev veth0

Easy conversion from VMDK to Proxmox

I kind of stumbled on this accidentally, but it has worked in the one instance in which I have tried it so far.

Proxmox uses KVM/QEMU for virtualization and stores disk images in raw format. Converting from a VMDK to a raw format is simple:

$ qemu-img convert -f vmdk my-vmdk.vmdk -O raw my-raw-image.raw

You can check the format if you like:

$ qemu-img info my-raw-image.raw 

image: my-raw-image.raw

file format: raw

virtual size: 8.0G (8589934592 bytes)

disk size: 1.8G

However Proxmox as far as I can tell has no method for simply choosing a virtual drive to use for a VM. So you have to create a VM (not a VZ container) using the regular Proxmox web interface (choose "no media" for the installation step since we won't be installing anything), then replace the raw image created for the new VM with the one created in the step above.

$ scp my-raw-image.raw root@PROXMOX-IP:/var/lib/vz/data/images/VM_ID/vm-VM_ID-disk-1.raw

In my instance this would be basically:

$ scp my-raw-image.raw root@192.168.1.100:/var/lib/vz/data/images/104/vm-104-disk-1.raw

The particularities here can change, so its probably better to ssh to the Proxmox server yourself and make sure the file/folder names are what you expect.

Anyway after this process I booted my VM, and it came right up, although it did inform me that I needed to run fsck (disk checker).

Note: If your virtual machine image has multiple partitions, I have no idea what you need to do, but take a look here for ideas, though the actual steps didn't work for me.

Boot speed in Arch

I've noticed that my bootups seem to be getting faster and faster. I remembered installing something that was supposed to automatically adjust the boot process over time and reboots, but I couldn't remember what it was. 

* EDIT: I remembered what I did to speed things up, from the Arch wiki :

# systemctl enable systemd-readahead-collect systemd-readahead-replay

Anyway, I ran systemd-analyze and compared the results to a post from a couple weeks ago, and man I was definitely right, just the four top entries are below:

Two weeks ago:

$ systemd-analyze blame

 7977ms NetworkManager.service
 2796ms colord.service
 2788ms bluetooth.service
 2728ms systemd-logind.service

Today:

$ systemd-analyze blame

1923ms systemd-modules-load.service
 769ms systemd-binfmt.service
 589ms NetworkManager.service
 555ms systemd-udev-trigger.service 

So these four entries alone show a change from about 16 seconds to about 3.

Convert PS to PDF, A4 vs Letter

In the US here we use some paper size called "letter" and apparently elsewhere they use A4, and for some reason is very close, but not quite the same. Probably some sort of metric thing. Anyway, I got a PostScript file that was created for A4 size output and wanted to convert it to a PDF. Regular ol' ps2pdf would truncate parts of the border, same with printing to a pdf. So, even though its not in "my" man page for ps2pdf, there is an option for telling ps2pdf the type of paper to expect.

$ ps2pdf -sPAPERSIZE=a4 toconvert.ps converted.pdf

Edit:
I did a bit more messing around: this page at linuxjournal.com has a lot of additional information on these types of conversions. In Arch, the package you need to get psselect is "psutils". I ended up having to experiment a bit to get what I needed. I wanted to remove the first 3 pages which were cover pages.

The command below did what I needed, order of arguments is important.

$ psselect -p3-25 toconvert.ps | ps2pdf -sPAPERSIZE=a4 - converted.pdf

The first command select pages 3-25 of the input PS file, pipes that to ps2pdf which then uses a dash "-" in place of the input file name.