Tuesday, November 26, 2013

GIT Snippets

I'm gonna just put all my handy git snippets in one place (right here) eventually

Git Configuration

Tell git to stop replacing my LFs with CRLFs

Git likes to replace Unix CRLFs with Windows CRLFs, this is really annoying, and is apparently the default now. The following fixes it, but it has to be done in every repository where you want to disable this. The global config file is in ~/.gitconfig

Make the change global (probably what you want for Linux).
$ git config --global core.autocrlf input
Per repository changes.
$ git config core.autocrlf         # check the existing value of autocrlf
true
$ git config core.autocrlf false   # change it to false
Other useful commands
$ git config <--global> --list     # Show config settings
$ git config <--global> -e         # Edit config settings

Retrieve previous revision of a file

First determine the revision you are interested in:
git rev-list -n num_results HEAD -- file
Retrieve the file associated with the revision:
git show revision:file
Example:
$ git rev-list -n 1 HEAD -- image.png299cf09892f7af2f9433782bb14a096547f6d9aa$ git show 299cf09892f7af2f9433782bb14a096547f6d9aa:./image.png > temp.png

Discard local changes
DANGER!!! I did this, and despite the option file name, it discarded ALL changes made! F#$@$@
$ git checkout HEAD <optional file name>
$ git checkout .

Show the remote URL

git remote -v

Show the remote URL with lots of additional info (requires ability to login to remote site)

git remote show origin

Change remote URL

git remote set-url origin  git@GIT_HUB_LAB_etc.com:USERNAME/REPO.git

Copy a file from other_branch to the current branch


git checkout other_branch probstmt.tex

Merge branch_x into current branch

git merge branch_x

Copy an branch over another without merging (for the most part)

git rebase branch_x

Several ways to show all files ever deleted from a project

git log --diff-filter=D --summary
git log --diff-filter=D --summary | grep delete
git log --all --pretty=format: --name-only --diff-filter=D | sort -u

To see files that have been deleted in the working copy

git ls-files --deleted

Restore deleted files (from a previous post)

Find the last commit that affected the given path. As the file isn't in the HEAD commit, this commit must have deleted it.
git rev-list -n 1 HEAD -- <file_path> 
Then checkout the version at the commit before.
git checkout <deleting_commit> -- <file_path or name> 
Or in one command, if $file is the file in question.
git checkout $(git rev-list -n 1 HEAD -- "$file")^ -- "$file"

Tuesday, November 19, 2013

Python snippets

Python is full of cool shortcuts that I can't seem to remember when I need to, so I'm gonna document them here.

Default dictionary

This can be used to avoid having to check for existence of keys by providing default values.          
from collections import defaultdict

    from collections import defaultdict
    counter = defaultdict(int) # Default to integer set to 0
    for akey in alist:
        counter[akey] += 1

Or one to many lists.
    from collections import defaultdict
    onetomany = defaultdict(list)
    for akey, anitem in acollection:
        onetomany[akey].append(anitem)

Output Latex Formats

print r"\begin{tabular}{|l|l|}"
print r"\hline"
print r"MAC & IP\\"
print r"\hline"
for snd,rcv in ans:
    print rcv.sprintf(r"%Ether.src% & %ARP.psrc%\\")
print r"\hline"
print r"\end{tabular}"

List Comprehensions

Convert a string to an array of ASCII characters
[ord(ch) for ch in "Hallo"]

Nested(?) access to list
in: r = [1,2]
in: [ (x, y) for x in r for y in r ]
out: [(1, 1), (1, 2), (2, 1), (2, 2)]

Create various shapes and sizes of sets of points
from random import random as rnd
_2x1 = [rnd() for _ in range(2)]
_3x2 = [[rnd() for _ in range(2)] for _ in range(3)]
_4x3x2 = [[[rnd() for _ in range(2)] for _ in range(3)] for _ in range(4)] 
ThreePoints = [[rnd(),rnd()] for _ in range(3)]

Strange Objects

Evaluate completely dynamic code
a=5
b="a*a"
eval(b)
25

Language Features

Built in list iteration
# x is a list of integers.
In : x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
 
In : x[::1]  # Every item
Out: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]In : x[::2]  # Get every 2nd item
Out: [0, 2, 4, 6, 8]In : x[::3]  # Every 3rd item
Out: [0, 3, 6, 9]

# Backwards
In : x[::-1]Out: [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]In : x[::-2]Out: [9, 7, 5, 3, 1]




Decorators to dynamically add functionality to class instance from StackOverflow
class Pizza(object):
    def __init__(self):
        self.toppings = []
    def __call__(self, topping):
        # Use '@instance_of_pizza' before a function def and the function gets passed onto 'topping'
        self.toppings.append(topping())
    def __repr__(self):
        return str(self.toppings)
myPizza = Pizza()
@myPizza
def cheese():
    return 'cheese'
@myPizza
def sauce():
    return 'sauce'
print myPizza
# ['cheese', 'sauce']

Iterators allow objects to work with for statements, require 'next' method
class PizzaFactory(object):
    def __init__(self):
        self.orders = {1:'ted", 2:"joe"}
    def __iter__(self):
        return OrderIterator(self)

class OrderIterator(object):
    def __init__(self, container):
        self.items = container
        self.index = 0

    def next(self):
        if self.index >= len(self.items): raise StopIteration
        ret = self.items[self.index]
        self.index += 1
        return ret

# pf = PizzaFactory()
for order in pizza: print order
1:ted
2:joe

Generators allow function to return intermediate values
def fib(n):
  a = 0
  yield a
  b = 1
  yield b
  while b < n:
    a, b = b, a + b
    yield b

# for e in fib(5): print e
0
1
1
2
3
5

Ternary operator
def factorial(n):
  return 1 if n == 0 else n * factorial(n - 1)
print factorial(5)

Generating core files from systemd-coredumpctl

Arch Linux stores core dumps in the journal, so if you want to debug a core dump you first have to dump the core using systemd-coredumpctl
Examples (you may have to be root if not in the systemd-journal group):

List core dumps


~ $ systemd-coredumpctl

Dump to file named core based on pid


~ $ systemd-coredumpctl dump 6738 -o core

Dump to file named core based on executable name


~ $ systemd-coredumpctl dump a.out -o core

Tuesday, November 12, 2013

Magic Computers

There was a time when computers seemed like pure magic. That is why I became so fascinated by them back in the early 1980s. The mystery surrounding how they worked was simply incredible and made me think anything was possible within their intangible boundaries.

I will never forget a day when my mother took me to a small computer shop in our small town and the proprietor sat me down in front of an Apple ][ running a game called Zork. I had no idea what it was or what I was supposed to do, but I typed something and it responded. To this day I can remember the angst I felt at having to get up and leave after what must have been half an hour of exploring.

I remember countless all nighters spent trying to get assembly language programs working on my 8088 "portable" computer. Writing data directly into the CGA video card or sending commands directly to the hard drive controller to move the read/write head made me feel as if I was exploring the moon.

I really miss those days, there is still much that I don't know, more than ever really. But sadly the combination of 1) a good understanding of how a computer works, 2) incredibly complex operating systems that have taken away the low level control we had with DOS and 3) the loss of youth have combined to robbed me of the sense of magic that computers came with in the 1980s.

Thursday, October 31, 2013

Relative root issues with gitlab

StackOverflow sure seems to get a lot of good answers. This entry addresses an issue I've been struggling with for some time related to setting relative roots in ruby on rails applications.

I'm using some Turnkey Linux templates with ProxMox and one is gitlab, but changing the relative root I was still missing some icons when loading the gitlab pages.

According the the post above after updating the config files I needed to go into the gitlab home folder and run:

bundle exec rake assets:precompile RAILS_ENV=production RAILS_RELATIVE_URL_ROOT=/gitlab


This took several minutes to complete (that's how you know its working?) but fixed my problem.

Friday, October 25, 2013

Pacman and "exists in filesystem" errors

Every now and then I try to update and am rebuffed because some file already exists and would be overwritten.
To find out what package owns the file: pacman -Qo /path/to/file

To install anyway and overwrite the existing file(s): pacman -S --force $package

Each installed package has a folder under /var/lib/pacman/local/ with metadata about the package.

Monday, October 14, 2013

Some ProxMox commands.

I've figured out how to bulk clone VMs in Proxmox which is very useful.
qm clone $VM_ID $CLONEID -name "$NAME" -pool "$POOL"


There are several other things that are useful dealing with clusters at http://wmunguiam.blogspot.com/2013/02/remove-node-cluster-forever-on-proxmox.html

List nodes:
pvecm nodes

Remove cluster node:
pvecm delnode node_name

But this may fail if you don't have a "quorum", in which case you change the number of nodes voting:
pvecm expected 1

Then delete the node.

Thursday, September 26, 2013

Reverse DNS Lookup Zone File and Whitespace

I have been struggling to get my reverse look up working on my DNS (bind) server for a LONG time... The problem I finally figured out:

$TTL 3D
@       IN      SOA     ns.linux.bogus. hostmaster.linux.bogus. (
                        199802151 ; Serial, todays date + todays serial
                        8H      ; Refresh
                        2H      ; Retry
                        4W      ; Expire
                        1D)     ; Minimum TTL
                NS      ns.linux.bogus.

; NO WHITESPACE!!!!
    1               PTR     gw.linux.bogus.
    2               PTR     ns.linux.bogus.
    3               PTR     donald.linux.bogus.
    4               PTR     mail.linux.bogus.
    5               PTR     ftp.linux.bogus.

See the spaces in front of the 1, 2, etc...? Those cannot be there for whatever reason.

Tuesday, September 24, 2013

Is Prism a reverse hoax?

I always figure things out before everyone else, but I never tell anyone about it. This time is going to be different.

Which is more likely:
  1. Our government actually thinks they could keep a monstrous scam like Prism a secret from the masses?

  2. Or the possibility that our government is willing to take a black eye PR wise in order to make bad guys *think* that they are being monitored therefore making their lives more difficult with relatively little effort?
I would actually be pretty impressed if the latter were to be the case. I'm way too cynical to believe it, but wouldn't that be great to find out that our government is actually doing all this to scam the terrorist?

This is similar to what I have heard said that Ronald Regan did to the Soviets with the Star Wars project. Made them think we were dumping tons of research into this project to the point they couldn't compete and they tore down that wall.

Here's hoping.

Friday, September 20, 2013

Restore previous version of a file from GIT repository

From the surprisingly helpful stackoverflow.com, regarding how to restore a file from GIT:

Find the last commit that affected the given path. As the file isn't in the HEAD commit, this commit must have deleted it.
git rev-list -n 1 HEAD -- <file_path>
Then checkout the version at the commit before.
git checkout <deleting_commit>^ -- <file_path>
Or in one command, if $file is the file in question.
git checkout $(git rev-list -n 1 HEAD -- "$file")^ -- "$file"

Thursday, September 12, 2013

VLC Fine Speed Control shortcut keys

I'm always forgetting this. In VLC if you want to increase/decrease playback speed use + or -, but that only works in big chunks. For smaller increments use [ and ].

Tuesday, September 10, 2013

Generating entropy quickly

I wrote a script to generate passwords for users, but it runs very slow at times. The solution was to install rng-tools.

From http://www.101tech.net/2011/11/01/apg-automated-password-generator-runs-slow-on-centos/

# apt-get install rng-tools
# echo "rngd -r /dev/urandom -o /dev/random -t 3" >> /etc/rc.local
# rngd -r /dev/urandom -o /dev/random -t 3

EDIT:
After reading this posting by Theodore Ts'o maybe using this isn't such a great idea as it is basically a bridge to the possibly compromised RNG instruction in the Intel processor.

Thursday, September 5, 2013

Bash one liner

I can't believe it took me this long to figure this out but an easy way to do one liners in bash for multiple files:
~$ for name in *.pdf; do pdftotext "$name"; done