Tuesday, December 17, 2013

rename command in Arch

I couldn't figure out why my rename command wasn't working in Arch, and it slowly dawned on me that the rename command in Arch doesn't support regular expressions... making it almost completely useless.
After some searching I found this blog entry that confirmed my suspicions and pointed me to an AUR package that basically implements rename with regular expressions.

$ packer -S rename
$ renamexm -s/\ /_/g *

Replace spaces with underscores in all files.

Thursday, December 12, 2013

Comparing contents of files

Apparently the diff command can be used for comparing the contents of two folders.
This LinuxCommando post gives the details.

$ diff folder1/ folder2/

Indenting code with vim

This stackoverflow post has a wealth of information about indenting, and particularly indenting code blocks in vim.
Though the block indent stuff doesn't seem to work with shell scripts, the re-indent entire buffer command does:
gg=G

Wednesday, December 4, 2013

Bash snippets

I guess I will continue my current trend of having a single post be a clearing house for tips relating to a specific language.

 Math

$ x=0; let x+=3; let x-=1; echo $x
2

Generate range of numbers
$ for i in {1..5}; do echo -n $i; done
12345

Alternatively
$ for i in $(seq 1 5); do echo $i; done
12345

Iteration, etc...

Change password expiration data
$ chage -d now user

Printf
$ printf  "%02d %02d\n" 5 6
05 06

Echo list of folder names
$ echo */
home/ Desktop/ ...

Iterating through a file line by line
IN_FILE=$1
OUT_FILE=$2
<!-- Add a generated password to end of line -->
while read s; do
    echo "$s, `apg -n 1`" >> $OUT_FILE
done < $IN_FILE

Or as a one liner (even works if lines have spaces)
$ while read s; do echo "$s"; done < broke.txt

Dereference variables
$ var1='some text'
$ var2='var1'
$ echo ${!var2}
some text

String and array manipulation

From tldp.org we have two articles covering similar things named Issue18 and string-manipulation:

Creating an array and indexing into arrays:

$ arr=(hi there dude)
$ echo "Middle value: ${arr[1]}"
Middle value: there
$ echo ${arr[*]}
hi there dude

Indexing into strings: ${variable_name:start_position:length}


$ var="Hello" && echo ${var:1:3}
ell

Separate a path and its base filename.

$ FILE="/some/file/with/path.txt"
$ basename $FILE
path.txt
$ dirname $FILE
/some/file/with

Another way to do this (the first doesn't always seem to work)

$ echo "${FILE##*/}"
path.txt

 I use this in scripts within a usage() function to print the name of the script

$ usage() { echo "Usage: ${0##*/} <options>" ... }

Tuesday, November 26, 2013

JavaScript Snippets

A repository of nice finds for the JavaScript language.

JavaScript Collection

The top answer on this stackoverflow question has an amazing set of  JavaScript examples and explanations of various things as well as why you should use certain methods over other methods.

Objects

Create a JavaScript object, code from this blog
var date = {
    year: 2013,
    month: 3,
    day: 3,
    getDate: function() {
        return ""+ this.year + " " +  this.month + " " + this.day;
    },
};

alert(date.getDate());

Bottom
CODE

PHP Snippets

Output

This really improves the look of data arrays.
print "<pre>";
print_r($data_array);
print "</pre>";

GIT Snippets

I'm gonna just put all my handy git snippets in one place (right here) eventually

Git Configuration

Tell git to stop replacing my LFs with CRLFs

Git likes to replace Unix CRLFs with Windows CRLFs, this is really annoying, and is apparently the default now. The following fixes it, but it has to be done in every repository where you want to disable this. The global config file is in ~/.gitconfig

Make the change global (probably what you want for Linux).
$ git config --global core.autocrlf input
Per repository changes.
$ git config core.autocrlf         # check the existing value of autocrlf
true
$ git config core.autocrlf false   # change it to false
Other useful commands
$ git config <--global> --list     # Show config settings
$ git config <--global> -e         # Edit config settings

Retrieve previous revision of a file

First determine the revision you are interested in:
git rev-list -n num_results HEAD -- file
Retrieve the file associated with the revision:
git show revision:file
Example:
$ git rev-list -n 1 HEAD -- image.png299cf09892f7af2f9433782bb14a096547f6d9aa$ git show 299cf09892f7af2f9433782bb14a096547f6d9aa:./image.png > temp.png

Discard local changes
DANGER!!! I did this, and despite the option file name, it discarded ALL changes made! F#$@$@
$ git checkout HEAD <optional file name>
$ git checkout .

Show the remote URL

git remote -v

Show the remote URL with lots of additional info (requires ability to login to remote site)

git remote show origin

Change remote URL

git remote set-url origin  git@GIT_HUB_LAB_etc.com:USERNAME/REPO.git

Copy a file from other_branch to the current branch


git checkout other_branch probstmt.tex

Merge branch_x into current branch

git merge branch_x

Copy an branch over another without merging (for the most part)

git rebase branch_x

Several ways to show all files ever deleted from a project

git log --diff-filter=D --summary
git log --diff-filter=D --summary | grep delete
git log --all --pretty=format: --name-only --diff-filter=D | sort -u

To see files that have been deleted in the working copy

git ls-files --deleted

Restore deleted files (from a previous post)

Find the last commit that affected the given path. As the file isn't in the HEAD commit, this commit must have deleted it.
git rev-list -n 1 HEAD -- <file_path> 
Then checkout the version at the commit before.
git checkout <deleting_commit> -- <file_path or name> 
Or in one command, if $file is the file in question.
git checkout $(git rev-list -n 1 HEAD -- "$file")^ -- "$file"

Tuesday, November 19, 2013

Python snippets

Python is full of cool shortcuts that I can't seem to remember when I need to, so I'm gonna document them here.

Default dictionary

This can be used to avoid having to check for existence of keys by providing default values.          
from collections import defaultdict

    from collections import defaultdict
    counter = defaultdict(int) # Default to integer set to 0
    for akey in alist:
        counter[akey] += 1

Or one to many lists.
    from collections import defaultdict
    onetomany = defaultdict(list)
    for akey, anitem in acollection:
        onetomany[akey].append(anitem)

Output Latex Formats

print r"\begin{tabular}{|l|l|}"
print r"\hline"
print r"MAC & IP\\"
print r"\hline"
for snd,rcv in ans:
    print rcv.sprintf(r"%Ether.src% & %ARP.psrc%\\")
print r"\hline"
print r"\end{tabular}"

List Comprehensions

Convert a string to an array of ASCII characters
[ord(ch) for ch in "Hallo"]

Nested(?) access to list
in: r = [1,2]
in: [ (x, y) for x in r for y in r ]
out: [(1, 1), (1, 2), (2, 1), (2, 2)]

Create various shapes and sizes of sets of points
from random import random as rnd
_2x1 = [rnd() for _ in range(2)]
_3x2 = [[rnd() for _ in range(2)] for _ in range(3)]
_4x3x2 = [[[rnd() for _ in range(2)] for _ in range(3)] for _ in range(4)] 
ThreePoints = [[rnd(),rnd()] for _ in range(3)]

Strange Objects

Evaluate completely dynamic code
a=5
b="a*a"
eval(b)
25

Language Features

Built in list iteration
# x is a list of integers.
In : x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
 
In : x[::1]  # Every item
Out: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]In : x[::2]  # Get every 2nd item
Out: [0, 2, 4, 6, 8]In : x[::3]  # Every 3rd item
Out: [0, 3, 6, 9]

# Backwards
In : x[::-1]Out: [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]In : x[::-2]Out: [9, 7, 5, 3, 1]




Decorators to dynamically add functionality to class instance from StackOverflow
class Pizza(object):
    def __init__(self):
        self.toppings = []
    def __call__(self, topping):
        # Use '@instance_of_pizza' before a function def and the function gets passed onto 'topping'
        self.toppings.append(topping())
    def __repr__(self):
        return str(self.toppings)
myPizza = Pizza()
@myPizza
def cheese():
    return 'cheese'
@myPizza
def sauce():
    return 'sauce'
print myPizza
# ['cheese', 'sauce']

Iterators allow objects to work with for statements, require 'next' method
class PizzaFactory(object):
    def __init__(self):
        self.orders = {1:'ted", 2:"joe"}
    def __iter__(self):
        return OrderIterator(self)

class OrderIterator(object):
    def __init__(self, container):
        self.items = container
        self.index = 0

    def next(self):
        if self.index >= len(self.items): raise StopIteration
        ret = self.items[self.index]
        self.index += 1
        return ret

# pf = PizzaFactory()
for order in pizza: print order
1:ted
2:joe

Generators allow function to return intermediate values
def fib(n):
  a = 0
  yield a
  b = 1
  yield b
  while b < n:
    a, b = b, a + b
    yield b

# for e in fib(5): print e
0
1
1
2
3
5

Ternary operator
def factorial(n):
  return 1 if n == 0 else n * factorial(n - 1)
print factorial(5)

Generating core files from systemd-coredumpctl

Arch Linux stores core dumps in the journal, so if you want to debug a core dump you first have to dump the core using systemd-coredumpctl
Examples (you may have to be root if not in the systemd-journal group):

List core dumps


~ $ systemd-coredumpctl

Dump to file named core based on pid


~ $ systemd-coredumpctl dump 6738 -o core

Dump to file named core based on executable name


~ $ systemd-coredumpctl dump a.out -o core

Tuesday, November 12, 2013

Magic Computers

There was a time when computers seemed like pure magic. That is why I became so fascinated by them back in the early 1980s. The mystery surrounding how they worked was simply incredible and made me think anything was possible within their intangible boundaries.

I will never forget a day when my mother took me to a small computer shop in our small town and the proprietor sat me down in front of an Apple ][ running a game called Zork. I had no idea what it was or what I was supposed to do, but I typed something and it responded. To this day I can remember the angst I felt at having to get up and leave after what must have been half an hour of exploring.

I remember countless all nighters spent trying to get assembly language programs working on my 8088 "portable" computer. Writing data directly into the CGA video card or sending commands directly to the hard drive controller to move the read/write head made me feel as if I was exploring the moon.

I really miss those days, there is still much that I don't know, more than ever really. But sadly the combination of 1) a good understanding of how a computer works, 2) incredibly complex operating systems that have taken away the low level control we had with DOS and 3) the loss of youth have combined to robbed me of the sense of magic that computers came with in the 1980s.

Thursday, October 31, 2013

Relative root issues with gitlab

StackOverflow sure seems to get a lot of good answers. This entry addresses an issue I've been struggling with for some time related to setting relative roots in ruby on rails applications.

I'm using some Turnkey Linux templates with ProxMox and one is gitlab, but changing the relative root I was still missing some icons when loading the gitlab pages.

According the the post above after updating the config files I needed to go into the gitlab home folder and run:

bundle exec rake assets:precompile RAILS_ENV=production RAILS_RELATIVE_URL_ROOT=/gitlab


This took several minutes to complete (that's how you know its working?) but fixed my problem.

Friday, October 25, 2013

Pacman and "exists in filesystem" errors

Every now and then I try to update and am rebuffed because some file already exists and would be overwritten.
To find out what package owns the file: pacman -Qo /path/to/file

To install anyway and overwrite the existing file(s): pacman -S --force $package

Each installed package has a folder under /var/lib/pacman/local/ with metadata about the package.

Monday, October 14, 2013

Some ProxMox commands.

I've figured out how to bulk clone VMs in Proxmox which is very useful.
qm clone $VM_ID $CLONEID -name "$NAME" -pool "$POOL"


There are several other things that are useful dealing with clusters at http://wmunguiam.blogspot.com/2013/02/remove-node-cluster-forever-on-proxmox.html

List nodes:
pvecm nodes

Remove cluster node:
pvecm delnode node_name

But this may fail if you don't have a "quorum", in which case you change the number of nodes voting:
pvecm expected 1

Then delete the node.