Getting started with proxmark 3 (Proxmark III)

This blog post has moved here

returning by reference faster than returning by value?

Is returning by reference faster than returning by value? It's always been my assumption that returning by value can in many cases be optimized away. However to test this assumption I wrote a couple of small tests:


By value:

#include <math.h>
#include <iostream>
#include <vector>

using namespace std;

vector<size_t> test() {
  vector<size_t> t;

  for(size_t n=0;n<100;n++) {
    t.push_back(rand());
  }

  return t;
}

int main() {

  size_t sum=0;
  for(size_t n=0;n<10000000;n++){
    vector<size_t> t = test();
    for(size_t n=0;n<t.size();n++) {
      sum = sum + t[n];
    }
  }
  cout << sum << endl;

}

By reference:

#include <math.h>
#include <iostream>
#include <vector>

using namespace std;

void test(vector<size_t> &t) {
  for(size_t n=0;n<100;n++) {
    t.push_back(rand());
  }
}

int main() {

  size_t sum=0;
  for(size_t n=0;n<10000000;n++){
    vector<size_t> t;
    test(t);
    for(size_t n=0;n<t.size();n++) {
      sum = sum + t[n];
    }
  }
  cout << sum << endl;

}

Here are my timings on an Intel Core2Duo:

By value:
real 0m34.122s
user 0m33.219s
sys 0m0.436s

By reference:
real 0m34.147s
user 0m33.308s
sys 0m0.512s


So in this case the user times seem to be more or less the same. What's happening is the return value being optimized out as expected? Is there something else I should take in to account in my test?

i686-apple-darwin10-g++-4.2.1 was used.

http://cpp-next.com/archive/2009/08/want-speed-pass-by-value/

Also interesting comment on gcc mailing list:

Your subject isn't quite right.  It should be "Returning by reference faster than returning by value".                                                                                                                                      
                                                                                                                                                                                                                                            
C++ has RVO ("return value optimization").                                                                                                                                                                                                  
                                                                                                                                                                                                                                            
RVO is part of C++, just as the elide constructors is a part of C++.                                                                                                                                                                        
                                                                                                                                                                                                                                            
Also note:  you get RVO even if you are -O0.  It really is part of the C++ language, even for non-optimized compiling.                                                                                                                      
                                                                                                                                                                                                                                            
So your two tests as given should have the about the same performance, since the two tests are basically doing the same thing.  And, indeed, they do have about the same performance.  As expected.                                         
                                                                                                                                                                                                                                            
Read carefully the constraints upon when RVO can be applied.  (I don't have a URL handy.  You'll have to use your Google-fu, or read the ISO 14882 standard.)                                                                               
                                                                                                                                                                                                                                            
In situations where RVO cannot be applied, then for non-trivial data types, an output reference parameter will (likely) be faster.                                                                                                          
                                                                                                                                                                                                                                            
A lot of STL and Boost rely upon RVO for their template magic to be highly performant.

Reverting an svn commit

To revert an svn commit do something like:

svn merge -r 4552:4551 svn+ssh://svn/path_to_svn_location
svn commit

Where 4552 is the last commit and 4551 is the one before.

Simple quicksort implementation

A simple C++ quicksort implementation. This version is not in place and therefore requires O(n) extra space...

#include "math.h"
#include <vector>
#include <iostream>

using namespace std;

template<class value_type>
void quick_sort(vector<value_type> &values,size_t start,size_t end,size_t pivot) {

  vector<value_type> less;
  vector<value_type> greater;

  for(size_t n=start;n<end;n++) {
    if(n != pivot)
    if(values[n] < values[pivot]) {
      less.push_back(values[n]);
    } else {
      greater.push_back(values[n]);
    }
  }

  value_type pivot_temp = values[pivot];
  for(size_t n=0;n<less.size();n++) {
    values[n+start] = less[n];
  }

  values[start+less.size()] = pivot_temp;

  for(size_t n=0;n<greater.size();n++) {
    values[n+start+less.size()+1] = greater[n];
  }

  if(less.size()    > 1) { quick_sort(values,start      ,start+less.size(),start+(less.size()/2)); }
  if(greater.size() > 1) { quick_sort(values,less.size(),end              ,less.size()+(end/2)  ); }
}

int main() {

  vector<size_t> values;

  for(size_t n=0;n<10;n++) {
    values.push_back(rand());
  }

  cout << "start values..." << endl;
  for(size_t n=0;n<values.size();n++) {
    cout << "val[" << n << "]: " << values[n] << endl;
  }

  quick_sort(values,0,values.size(),5);
  cout << "sorted values..." << endl;
  for(size_t n=0;n<values.size();n++) {
    cout << values[n] << endl;
  }
}

Templates v subclasses v.2

A slightly more complicated test similar to the previous post. In this case the templated version doesn't have such a huge advantage.

Subclassed version:

#include <iostream>

using namespace std;

class mybaseclass {
  public:

    virtual int get_i() = 0;
};

class myclass : public mybaseclass {

  public:

    myclass(int in) {
      i=in;
    }

    int get_i() {
      i++;
      return i;
    }

  int i;
};

class otherclass : public mybaseclass {

  public:

    otherclass(int in) {
      i = in;
    }

    int get_i() {
      return i*i;
    }

  int i;
};

int main(void) {
  myclass *c = new myclass(2);

  double sum=0;
  for(size_t n=0;n<2000000000;n++) {

    mybaseclass *cc = c;

    sum += (static_cast<double>(cc->get_i())/2.2);

  }
  cout << sum << endl;
  delete c;
}

Templated version:

#include <iostream>

using namespace std;

template<class t>
class myclass {

  public:

    myclass(int in) {
      i=in;
    }

    inline int get_i() {
      i++;
      return i;
    }

  int i;
};

template<class t>
class otherclass {

  public:

    otherclass(int in) {
      i = in;
    }

    inline int get_i() {
      return i*i;
    }

  int i;
};

int main(void) {
  myclass<int> *c = new myclass<int>(2);

  double sum=0;
  for(size_t n=0;n<2000000000;n++) {

    sum += (static_cast<double>(c->get_i())/2.2);

  }
  cout << sum << endl;
  delete c;
}

$ g++ small_classed.cpp -O3;time ./a.out
9.09091e+17

real 0m21.151s
user 0m20.909s
sys 0m0.072s


$ g++ small_templateclassed.cpp -O3;time ./a.out
9.09091e+17

real 0m20.736s
user 0m20.527s
sys 0m0.064s

Are templates faster than subclasses?

Are template classes faster than using a derived class? Intuition would tell me that the template class should be faster. However to test if this is the case I wrote the following small programs:

Subclassed version:

#include <iostream>

using namespace std;

class mybaseclass {
  public:

    virtual int get_i() = 0;
};

class myclass : public mybaseclass {

  public:

    myclass(int in) {
      i=in;
    }

    int get_i() {
      return i;
    }

  int i;
};

class otherclass : public mybaseclass {

  public:

    otherclass(int in) {
      i = in;
    }

    int get_i() {
      return i*i;
    }

  int i;
};

int main(void) {
  myclass *c = new myclass(2);
  long int sum=0;
  for(size_t n=0;n<2000000000;n++) {
    mybaseclass *cc = c;

    sum += cc->get_i();

  }
  cout << sum << endl;
  delete c;
}

Templated version:

#include <iostream>

using namespace std;

template<class t>
class myclass {

  public:

    myclass(int in) {
      i=in;
    }

    int get_i() {
      return i;
    }

  int i;
};

template<class t>
class otherclass {

  public:

    otherclass(int in) {
      i = in;
    }

    int get_i() {
      return i*i;
    }

  int i;
};

int main(void) {
  myclass<int> *c = new myclass<int>(2);

  long int sum=0;
  for(size_t n=0;n<2000000000;n++) {

    sum += c->get_i();

  }
  cout << sum << endl;
  delete c;
}

$g++ small_templateclassed.cpp -O3;time ./a.out
4000000000

real 0m0.008s
user 0m0.000s
sys 0m0.001s


$ g++ small_classed.cpp -O3;time ./a.out
4000000000

real 0m5.908s
user 0m5.853s
sys 0m0.018s

My first effort was IO bound, but in this case it appears that templatized code is substantially faster.

Nabble thread is here.

Renicing synology nfsd so it has priority over other services

We needed nfs to have priority over other services on a synology DS509+. To do this we reniced the nfsd process. Synologies are basically just linux boxes. So enable ssh on the synology and ssh to it (ssh synology -lroot) use your normal admin password for the password. Then edit /usr/syno/etc/rc.d/S83nfsd.sh. Where it currently reads:

/sbin/portmap
/usr/sbin/nfsd
/usr/sbin/mountd -p 892
;;
stop)

change it to read:

/usr/sbin/nfsd
/usr/sbin/mountd -p 892
renice -10 `pidof nfsd`

;;
stop)

Commands used to install golang

Commands I used to install Go language (golang) on Ubuntu amd64:

sudo apt-get install mercurial bison gcc libc6-dev ed make
cd ~
export GOROOT=$HOME/go
export GOARCH=amd64
export GOOS=linux
mkdir $HOME/gobin
export GOBIN=$HOME/gobin
export PATH=$PATH:$GOBIN
hg clone -r release https://go.googlecode.com/hg/ $GOROOT
cd $GOROOT/src
./all.bash

To compile something make a test program e.g.:

package main

import "fmt"

func main() {
       fmt.Printf("hello, world\n")
}

and compile with:

8g test.go ;8l test.8 ;./8.out

Version

Hi,

Probably did this before - but administered lost and found computers has made me need this info again.

Version of linux.

cat /proc/version

Gives you:

Linux version 2.6.28-15-generic (buildd@rothera) (gcc version 4.3.3 (Ubuntu 4.3.3-5ubuntu4) ) #52-Ubuntu SMP Wed Sep 9 10:49:34 UTC 2009

And then you need the release info for the distribution.

lsb_release -a

Gives you:

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 9.04
Release:        9.04
Codename:       jaunty

And of course uname -a. Which is a start. Would love to hear other tips on determining the state of a box that has previously been abandoned.

Update: Naming of the actual machine and shit as well.

Running hostname should give you the name of the actual machine. Not entirely sure how this works when you have multiple "machines" kinda running off the same physical machine.

user@machine:~$ hostname
machine-name

You can then check this the other way round:

user@machine:~$ host name.domain.tld
ip address .in-addr.arpa domain name pointer name.domain.tld

list installed packages on debian and uninstall those not correctly removed.

dpkg --get-selections

Will give you a list of packages currently installed on a debian system (possibly works for Ubuntu too?). You may see a few which are listed as "deinstall". They've not uninstalled correctly. To uninstall them use this command I found on the debian mailing lists to purge them:

dpkg --get-selections | grep deinstall | cut -f1 | xargs dpkg -P

Turning a Ubuntu install into a Window terminal server dumb terminal

I've written a quick script to term a clean Ubuntu (I'm using 9.04 netbook remix) into a dumb terminal to connect to a windows terminal server.

You can use it like this:

wget http://sgenomics.org/~new/term.tar
tar xvf term.tar
sudo ./termsvr MYTERMINALSERVERNAME MYDOMAINNAME


Under the hood, the script looks like this:

if [ $# -ne 2 ]
then
echo "Usage: `basename $0` "
exit 1
fi

apt-get -y --force-yes remove gnome*
apt-get update
apt-get install rdesktop
cp ./rc.local /etc/rc.local
sed "s/\*SERVERNAME\*/$1/" ./xinitrc > ./xinitrc.fixed0
sed "s/\*DOMAINNAME\*/$2/" ./xinitrc.fixed0 > ./xinitrc.fixed1
cp ./xinitrc.fixed1 /etc/X11/xinit/xinitrc

where xinitrc contains:

#!/bin/bash
# $Xorg: xinitrc.cpp,v 1.3 2000/08/17 19:54:30 cpqbld Exp $

# /etc/X11/xinit/xinitrc
#
# global xinitrc file, used by all X sessions started by xinit (startx)

while [ true ]
do
rdesktop -f *SERVERNAME* -d *DOMAINNAME* -u "" -k en-gb
done

# invoke global X session script
. /etc/X11/Xsession

and rc.local contains:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

startx

exit 0

Upgrading eclipse on Ubuntu

After totally borking my eclipse install I seriously suggest just removing every mention of the damned thing before installing the new version. By removing I mean move to a new dir in /tmp/ of course.

Things to look out for:

/home/user/.eclipse
/home/user/.mozilla/eclipse
/foo/bar/eclipse

Seriously this should be a lot easier. Anyway - the other thing to attempt is exporting your old software update sites list and then importing this into the new install. This should save an ungodly amount of time reinstalling plugins - which vastly outweighs the amount of time installing takes.

strings in the n900 firmware

The n900 isn't out yet, but it's firmware is! I ran strings over it and looked at the longer, more interesting strings as follows:

strings RX-51_2009SE_1.2009.42-11_PR_F5_MR0_ARM.bin | awk '{if(length($0) > 50) print $0}' | more

Some interesting finds:

Uses glibc, good to know... GLIBC_2.4

"Major hack here, this needs to be spawned from a proper class"

Always good to know. :)

Skype:

usr/share/locale/fr_FR/LC_MESSAGES/skype-ui.mo

Lots of these:

TMS320C6x COFF Linker Unix v6.0.7 Copyright (c) 1996-2006 Texas Instrument

''C:/users/Prasad/Nokia/CHK_100_V_H264AVC_D_BP_OMAP3430_1_Q

5320C6x COFF Linker Unix v6.0.7 Copyright (c) 1996-2006 Texas Instrumentl

Looks like they are making extensive use of the DSP on the OMAP3430. I guess for accelerating graphics etc. This is good news in terms of functionality, should make the whole experience smoother. But bad news for open source zealots as there's no open source TMS320C6x compiler.

Some familiar apps:

usr/share/doc/osso-fm-transmitter-l10n-cscz/changelog.gz

/usr/share/applications/hildon/chess_startup.desktop
/usr/share/applications/hildon/mahjong_startup.desktop
/usr/share/applications/hildon/osso_lmarbles.desktop
/usr/share/applications/hildon/osso_pdfviewer.desktop
/usr/share/applications/hildon/osso_calculator.desktop
/usr/share/applications/hildon/osso_rss_feed_reader.desktop

What does it use prolog for?!?

usr/share/doc/swi-prolog/changeD

No IPv6?

8Removing IPv6 stuff because it is no longer found in fremantle BUT only ifR

echo "Welcome to your upstart powered internet tablet."

Upstart powered!!

Custom bootloader:

INokia OMAP Loader v1.4.13 (Oct 5 2009) running on %s %s (%s

Wonder if this is the same bootloader they use on the n97...

%s: Checksum mismatch (stored 0x%04x, calc. 0x%04x). Who cares, right?

Yea! Who cares!

Yup, there it is. Stand by for some Linux pleasure.

That's right bitches! Bring on the pleasure.

Make directory if it doesn't exist return true so you can use it in a make file

[ -d /opt/foo/bin ] || mkdir -p /opt/foo/bin

Routertech AR7RD ppp interface not coming up

The ppp0 interface on my RouterTech router wasn't coming up on reboot, I have no idea why. To get it to come up I telneted in to the router (login as root, with normal admin password). Then executed the following:

/etc/ppp/ip-up

vector bounds checking in GCC

Robbed from somewhere on the internet. I can't try this is my application because my libraries are not compiled with GLIBCXX_DEBUG but I believe this option should enable bounds checking on stl vectors etc. Here as a note for future use:

Pedantic debug mode checks not only for errors, but also for usages that rely on GNU extensions and that, while correct, may not be portable to other compilers. In pedantic mode, for example, constructing an std::string with a null pointer will result in an abort, not an exception. Pedantic debug mode is not the default. To enable pedantic debug mode, compile your program with both -D_GLIBCXX_DEBUG and -D_GLIBCXX_DEBUG_PEDANTIC.

Robbed from here: http://lists.apple.com/archives/xcode-users/2006/Feb/msg00209.html

Submit a bunch of jobs to SGE using a file as a template

This script takes a filename from the third whitespace separated field of each file of the file "work-to-do" it uses this as the argument of a command (at /home/user/stuff/bin/stuff). The second argument of this command is the directory of the first argument. The third is a name generated from the current date. These commands are generated and then submitted to an SGE queue using qsub (the queue is called myqueueq). As always probably useless to anyone but me.

awk '{print "cd /home/user/stuff;./bin/stuff "$3" `dirname "$3"` auto-`date +%Y-%m-%dT$H-%M-%S`"}' work-to-do > sgecommands

awk '{print "echo \"" $0 "\" > /tmp/regen_run;qsub -q myqueueq /tmp/regen_run"}' sgecommands

Windows 95/98/Me registry kludging

There is a neat trick that puts Windows 9x back into setup. This lets you resetup all your drivers and can be useful in recovering a dead system (for example one that will not boot into Windows normally). It also keeps all your application settings.
Basicly what you do is merge a registry file called "SYSTEM.1ST" which is stored in the root of the C: drive ontop of your registry. To do this do the following:

Reset the computer, when it says "Starting Windows..." press F5.
You should be presented with a C:\> prompt type:

attrib -h -r -s C:\SYSTEM.1ST
regedit /L:C:\SYSTEM.1ST /E C:\1ST.REG
regedit C:\1ST.REG

next time you boot Windows you will be presented with the setup screen. Your serial number sould already be displayed and you should just be able to click through most of the options.

If your registry is completely corrupt then you may want to just copy C:\SYSTEM.1ST over C:\WINDOWS\SYSTEM.DAT

Misc RM Connect 2.x fixes

Is anybody still stupid enough to use RM Connect? I hope not but in the event that they are here are a random collection of bug fixes:

No packages install

Symptoms

No packages install on any workstations, even clean builds. No error is displayed at all.

Solution

Last time I encountered this it was due to the Nimda virus messing up Share permissions on our network. Check that the "Propagate" share on your PDC has the correct permissions. The share should be located at D:\RMVA\Manage\pkgctrl and permissions should be (as for all RM based shares) Everyone/Full Control. File permissions should be "Propagator/Full Control".

If this problems has been caused by Nimda trashing you file permissions you will probably find that many other shares have been distroyed as well. I've written some scripts that will reset the share permissions to there correct values, please contact at: help@asknav.com if you would like more information.

Users exceed quota due to large Internet History folders

Symptoms

Users (on a 200/300 station network) come to you on an almost daily basis saying that their file area is full and they can't logon, but when they look in their file area they can't find anything to delete. When you you examine their file area it's AppData is huge, and cookies/History information is using up their quota.

Solution

I've written a small program, which you run from the login script of all users, to solve this problem. Basicly it will delete everything in the Internet folder of a user as they login, this should keep the filearea down to a reasonable size.

The fix is available from the link below:

delHist2.exe

I remove NetBEUI from a client and it continues to work, but I get an error from licence manager when I bootup

You want to get rid of NetBEUI, good idea! NetBEUI produces far too much network traffic, and is not recommended for use on networks larger than 30/40 computers.

The problem you are getting is caused by the fact that RM licence manager is configured by default to use NetBEUI to check for licences. The good news is however that RM licence manager can be configured to use TCP/IP to do this. I've created a package to do this and it can be downloaded from the link below. To install the package on your network simply unzip the folder and copy it to D:\RMVA\manage\packages\32bitWin or the share \\PDCNAME\manage\packages\32bitWin. Then allocate it to all your workstations using application wizard.

remNetB.zip

Licence Manager doesn't work even on a clean build

Well, it sounds like licence manager is broken on the server. The simplest thing to do is reset it, and add some new licences. Licence manager is really a tool for you to keep track of licencing, so if you are having problems you can add as many licences as you want.

Step 1: Stop the "RM Licence Manager" service on all of your servers.
Step 2: Delete the file called "C:\WINNT\rmlicman.lic" or "C:\WINNT\System32\rmlicman.lic".
Step 3: Start the "RM Licence Manager" service on all of yout servers.
Step 4: Run the "RM Licence Manager" program from Admin Tools on the start menu, it may take some time to startup. Add as many licences as you want to each of your servers.

I'm running Sophos with file Protector, but everytime a user logs in it reinstalls itself, but I've got the correct exculsions setup

Solution

(Quick and Dirty tobe updated soon)

Your problem is that File Protector recovers the registry from a backup (taken when a package is allocated or when you "Backup key files") when the system boots up.

The easiest way to solve this is to stop File Protector from replacing the registry at boot. The downside is that any changes that a user makes could stick (which includes, if they know how to do it, disabling File protector).

OK what you need to do is edit a batch file in the "FileProt" package. This is located at: D:\RMVA\Manage\packages\32BitWin\FileProt on your PDC. Or mapped to L:\Packages\32BitWin\FileProt on Admin accounts. You need to edit the file in the STATION\HDS directory called recover.bat load it into notepad. Find the lines under "rem System.Dat" and alter them so they read:

rem System.Dat
rem c:\windows\command\attrib -r -h -s c:\Windows\System.DAT
rem c:\windows\command\attrib -r -h c:\BackUp\System.da0
rem copy c:\BackUp\System.da0 c:\Windows\System.DAT >> %logfile%
rem c:\windows\command\attrib +r +h +s c:\Windows\System.DAT
rem c:\windows\command\attrib +r +h c:\BackUp\System.da0

rem User.Dat
rem c:\windows\command\attrib -r -h -s c:\Windows\User.DAT
rem c:\windows\command\attrib -r -h c:\BackUp\User.da0
rem copy c:\BackUp\User.da0 c:\Windows\User.DAT >> %logfile%
rem c:\windows\command\attrib +r +h +s c:\Windows\User.DAT
rem c:\windows\command\attrib +r +h c:\BackUp\User.da0

Instead of:

rem System.Dat
c:\windows\command\attrib -r -h -s c:\Windows\System.DAT
c:\windows\command\attrib -r -h c:\BackUp\System.da0
copy c:\BackUp\System.da0 c:\Windows\System.DAT >> %logfile%
c:\windows\command\attrib +r +h +s c:\Windows\System.DAT
c:\windows\command\attrib +r +h c:\BackUp\System.da0

rem User.Dat
c:\windows\command\attrib -r -h -s c:\Windows\User.DAT
c:\windows\command\attrib -r -h c:\BackUp\User.da0
copy c:\BackUp\User.da0 c:\Windows\User.DAT >> %logfile%
c:\windows\command\attrib +r +h +s c:\Windows\User.DAT
c:\windows\command\attrib +r +h c:\BackUp\User.da0


The run Application wizard and allocate "FileProt", choose to re-allocate to workstations already setup.

Registry changes should new stick and Sophos should install itself only once.

If you still have problems make sure the follow exclusions are set in file protector:

C:\InterChk.chk
C:\Program Files\Sophos Sweep

Windows NT/2000 password recovery

I've found two options for cracking Windows 2000 passwords (this probably still works for XP and maybe even Vista).

The first is to buy LockSmith and NTRecover from Wininternal, these cost a hell of a lot (about 200quid I think). They will however almost definitely work as the people who write them really know their stuff.

The second (cheaper option) is to use a free program that's available on the web (http://home.eunet.no/~pnordahl/ntpasswd/). I've saved many systems using this but comes with no guarantees.

The required files are available on the above website, with local copies below. Download them into a directory on your computer and then from the commandline do the following:

C:
CD C:\
rawrite2

When prompted for the disk image source file name enter "bd010114.bin" the target drive will of course be A:. Insert a disc and the program should create you a bootable floppy which is able to reset the Administrator password.

When you boot from this disc on the workstation you are cracking you should be able to accept the defaults for most of the options. You probably won't need to probe for SCSI devices, and you need to answer yes when prompted to write the SAM.
bd010114.bin
rawrite2.exe

Internet Explorer and Acrobat print incorrectly. But Word and notepad do.

Symptoms

You can print from Microsoft Word and notepad correctly, but when you try printing from Internet explorer you get an error message similar to:

Line: 539
Char: 2
Error: Invalid argument.
Code: 0 URL: res://C:\WINDOWS\SYSTEM\SHDOCLC.DLL/preview.dlg

You may also (or possibly only) find that documents in Adobe Acrobat don't print correctly (words run into each other etc). The printer you're trying to print to is an HP printer.

Solution
These errors most commonly occur with HP printers, usually they are down to the PCL 6 driver that is supplied with the printer. Your best option is to download a PCL 5e, or PS driver from the HP website.

Appariently the Internet Explorer problem can also be caused by a faultly DLL (although I haven't come across this), details are available at: http://support.microsoft.com/support/kb/articles/Q293/1/76.ASP

Network configuration for older brother (1260/1660) printers/Emulex configuration

I just rediscovered the internet archive backup of a website I had back in the 2000 odd. Most of the stuff from there is old and no longer relevant but may just be of some use to someone.

Network configuration for older brother (1260/1660) printers/Emulex configuration

Step 0 resets the card, you can probably skip this for a new card.

0. Remove the Emulex NetJet card from the printer.
0.1 Set Jumper JX1 to the 1-2 position (nearest the connector).
0.2 Put the card back in the printer and switch it on for 30 seconds.
0.3 Switch the printer off and remove the card.
0.4 Set JX1 to 2-3 and put the card back in the printer.
0.5 Switch the printer on.
1. Make a note of the MAC address of the printer (writen on the back of the NetJet card).
2. At a cmd prompt on the workstation type:
"arp -s ipaddress mac" for example: "arp -s 192.168.3.50 00-00-c9-09-22-67". Telnet to the printer
4. Enter the default password which is "access"
5. Type "su", when prompted for a password type "system"
6. You should now have a "Server>>" prompt.
7. Type "define subnet yoursubnet" to set the subnet.
8. Type "define node ip your_ip" to set the ip address. for example "define node ip 192.168.3.50"
9. Type "logout".
10. You should now be able to ping the printer from any workstation on the network.
11. You'll need to setup an LPR port on the server to connect to the printer, under Windows NT4/2000 use generic LPR.
12. The queue name will be "PASSTHRU", and protocol should be LPR not RAW.

A short script to move files out of the way retaining full path

Find all files named *.fast4 under the current directory. Create replicate the path under ./obsolete and copy the file to the associated location. This script creates the required commands in a file called "moveobsolete" which you can then check over and execute.

find . -name "*.fast4" > obsoletefiles
awk '{print "mkdir -p ./obsolete/`dirname "$1"`; mv " $0 " ./obsolete/" $0}' obsoletefiles > moveobsolete
chmod 777 moveobsolete

Rename files matching given spec to incrementing filenames

This one liner renames all files starting with shred_ to shred_0.tif, shred_1.tif etc.

ls shred_* | awk 'BEGIN{n=0;}{print "mv " $0 " shred_" n ".tif"; n++;}' | sh

My .mailcap

Used with mutt to view gifs and jpegs as ascii graphics!!!! requires aview package on ubuntu/debian:

image/gif; asciiview '%s'
image/png; asciiview '%s'
image/jpg; asciiview '%s'
image/jpeg; asciiview '%s'

Robbed off some random place on the internets.

Simple pysam example

This python code uses pysam to parse a bam file and print out all the alignment contained therein.

import pysam

# Open our sam file
samfile = pysam.Samfile()
samfile.open( "sim.bam", "rb" )

# Grab an alignment from the sam file

def m_fetch_callback( alignment ):
  print str(alignment)

num_targets = samfile.getNumTargets()

for n in xrange(num_targets):
  samfile.fetch(samfile.getTarget(n),m_fetch_callback)

Simple zlib file reading example

This C++ example can read from both compressed and compressed files. It will read from testfile.gz (which can either be compressed or uncompressed) and write the contents to the standard output (screen).

#include "zlib.h"

#if defined(MSDOS) || defined(OS2) || defined(WIN32) || defined(__CYGWIN__)
#  include <fcntl.h>
#  include <io.h>
#  define SET_BINARY_MODE(file) setmode(fileno(file), O_BINARY)
#else
#  define SET_BINARY_MODE(file)
#endif

#include <iostream>

using namespace std;

int main(void) {

  gzFile f = gzopen("testfile.gz","rb");

  char buffer[1001];

  for(;!gzeof(f);) {

    int e = gzread(f,buffer,1000);
    buffer[1000] = 0;
    cout << buffer << endl;
  }

  gzclose(f);

}

Compile with g++ testprog.cpp -lz -o testprog

Count files matching a given type/find spec

Count file size in bytes of all files matching *fast4 modified in the last 90 days.

x=`find . -atime 90 -name \*fast4 -ls | awk '{print $7 " +"}' | xargs echo`; echo "$x 0" | bc

examine irssi scrollback

I always find this a pain to lookup. But to scrollback in irssi: esc then p, forward esc then n.

iWorks '09 Keynote hangs loading a ppt file potentially losing all your work

I had a problem. I'd been working all day on a presentation which I hadn't saved (yea I know). Then I tried to load a powerpoint file and Keynote just hung trying to load the file. If I couldn't get it to unhang I would have lost a days work as Keynote doesn't autosave. The solution is to attach gdb to the Keynote and get it to jump out of whichever loop it's stuck in. Like so:

news-macbook:MacOS new$ ps -ef | grep Keynote
 501 13290    82   0   1:22.74 ??        46:51.60 /Applications/iWork '09/Keynote.app/Contents/MacOS/Keynote -psn_0_1552763
news-macbook:MacOS new$ grep -p 13290
grep: invalid option -- p
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
news-macbook:MacOS new$ gdb -p 13290
GNU gdb 6.3.50-20050815 (Apple version gdb-1344) (Fri Jul  3 01:19:56 UTC 2009)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin".
/Applications/iWork '09/Keynote.app/Contents/MacOS/13290: No such file or directory
Attaching to process 13290.
Reading symbols for shared libraries . done
Reading symbols for shared libraries ................................................................................................................................................................................................................................ done
0x942ed26d in ___CFBasicHashFindBucket1 ()
(gdb) bt
#0  0x942ed26d in ___CFBasicHashFindBucket1 ()
#1  0x942edd5b in CFBasicHashFindBucket ()
#2  0x942edcd4 in -[NSClassicMapTable objectForKey:] ()
#3  0x942edca8 in NSMapGet ()
#4  0x008dcf77 in -[ADReference reference] ()
#5  0x008db594 in -[ADNode firstChildWithName:xmlNamespace:objcClass:resolveReference:] ()
#6  0x008db3d2 in -[ADNode firstChildWith:resolveReference:] ()
#7  0x008db369 in -[ADNode firstChildWith:] ()
#8  0x009036b6 in -[ADSStyleSheet(ADInternal) parentSheet] ()
#9  0x00903f22 in -[ADSStyleSheet(Private) recursiveStyleWithIdentifier:] ()
#10 0x00903613 in -[ADSStyleSheet ensureOwnStyleWithIdentifierExists:] ()
#11 0x005f50a7 in +[PIOGraphicStyleMapper mapMasterStylesFromPptSlide:toStyleSheet:withState:] ()
#12 0x005fbe13 in -[PIOMasterSlideMapper mapMasterStyles] ()
#13 0x005fbf9d in -[PIOMasterSlideMapper map] ()
#14 0x00610ea9 in -[PIOThemeMapper mapMaster:includingGraphics:] ()
#15 0x00610f94 in -[PIOThemeMapper mapMaster:] ()
#16 0x006111b1 in -[PIOThemeMapper mapMasters] ()
#17 0x00611285 in -[PIOThemeMapper map] ()
#18 0x005f28c1 in -[PIODocMapper(Private) mapThemes] ()
#19 0x005f238b in -[PIODocMapper map] ()
#20 0x005f9467 in -[PIOImporter readFromFile:binaryDirectory:templatePath:localizer:] ()
#21 0x0022e0e2 in ?? ()
#22 0x0022e2d0 in ?? ()
#23 0x0022f789 in ?? ()
#24 0x0022ecea in ?? ()
#25 0x000a8c5f in ?? ()
#26 0x000a875f in ?? ()
#27 0x96c08769 in -[NSDocument initWithContentsOfURL:ofType:error:] ()
#28 0x96c0830d in -[NSDocumentController makeDocumentWithContentsOfURL:ofType:error:] ()
#29 0x000163ed in ?? ()
#30 0x00096fb4 in ?? ()
#31 0x02ddac2e in -[SFAppApplicationDelegate(Private) pOpenDocumentWithContentsOfFile:] ()
#32 0x02dd822d in -[SFAppApplicationDelegate application:openFiles:] ()
#33 0x001f7f55 in ?? ()
#34 0x96c0637d in -[NSApplication(NSAppleEventHandling) _handleAEOpenDocumentsForURLs:] ()
#35 0x96b44104 in -[NSApplication(NSAppleEventHandling) _handleCoreEvent:withReplyEvent:] ()
#36 0x9432346c in -[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon:] ()
#37 0x94323230 in _NSAppleEventManagerGenericHandler ()
#38 0x90905de6 in aeDispatchAppleEvent ()
#39 0x90905ce5 in dispatchEventAndSendReply ()
#40 0x90905bf2 in aeProcessAppleEvent ()
#41 0x9061a381 in AEProcessAppleEvent ()
#42 0x969bddd6 in _DPSNextEvent ()
#43 0x969bd40e in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()
#44 0x9697f5fb in -[NSApplication run] ()
#45 0x0000f13f in ?? ()
#46 0x0000f0b8 in ?? ()
#47 0x0000ee50 in ?? ()
#48 0x0005cc79 in ?? ()
(gdb) ret
Make selected stack frame return now? (y or n) y

Keep 'ret'ing until you reach 'readFromFile'. At this point type 'c'. Keynote will then tell you it couldn't open the file and you will be able to save your existing document.

Postfix won't start

Just noticed I wasn't receiving system mail for the last few weeks. Coming back after holiday. Anyway, one of the few services not to restart after the computer had been switched off in my absence was postfix.

Couldn't for the life of me work out the problem, it was complaining about not finding the host. Therefore I had a look in /etc/hosts and commented out the line below:

#
# hosts         This file describes a number of hostname-to-address
#               mappings for the TCP/IP subsystem.  It is mostly
#               used at boot time, when no name servers are running.
#               On small systems, this file can be used instead of a
#               "named" name server.
# Syntax:
#
# IP-Address  Full-Qualified-Hostname  Short-Hostname
#

127.0.0.1       localhost


# special IPv6 addresses
#::1             localhost ipv6-localhost ipv6-loopback

fe00::0         ipv6-localnet

ff00::0         ipv6-mcastprefix
ff02::1         ipv6-allnodes
ff02::2         ipv6-allrouters
ff02::3         ipv6-allhosts

The one about IPv6. Dunno what that was about - but it works now.

Program for calculating E

This post is now here

Remove any linefeed from line which did not end in capital or special character (vim)

Remove any linefeed from line which did not end in capital or one of ./?<>:'

Like so:

%s/\([^./?<>":'A-Z]\)\n/\1/g

Twitter botnet/remote access

Be the envy of your friends! Remote control your PC via twitter! Add the following code to cron to send all tweets with CMD in them to the shell.

curl twitter.com/new299 | html2text | grep "CMD" | awk '{$1="";$0=substr($0,2)}1' | sh

Simulate dataset with samtools, align with Bowtie, BWA, MAQ and pileup

This post has moved here

Reset author to filename on a bunch of pdf files

This script will reset the author to metadata on all files in the current directory to be the same as the filename of the file. It will write new files to a sub directory (which should already exist) called fixed.

Quick hacky script to so that papers are index more sensibility on my Sony Reader.

for i in *.pdf
do
        echo $i
        echo "InfoKey: Author" > data.txt
        echo "InfoValue: " $i >> data.txt
        cat data.txt
        pdftk ./$i update_info data.txt output ./fixed/$i
done

ö (O with umlaut) on Mac OS X

This post has moved here

Running samtools pileup on some aligned reads

samtools needs a sorted file for pileup.. doesn't give you an error if it isn't sorted so watch out!

./samtools faidx phi.fa
./samtools import ./phi.fa.fai /scratch/nava/lane7.phi.sam /scratch/nava/lane7.phi.bam
./samtools sort /scratch/nava/lane7.phi.bam /scratch/nava/lane7.phi.bam.sort
./samtools pileup -f ./phi.fa /scratch/nava/lane7.phi.bam.sort.bam > pile

vim/vi inverted replace

Replace any character that isn't the ^ character with nothing:

%s/[^^]//g

n810 g++ installation (os 2008.1 diablo)

Install the following repositories by editing /etc/apt/sources.list.d/hildon-application-manager.list

add the following:

deb http://repository.maemo.org/ diablo sdk/free sdk/non-free tools/free tools/non-free
deb http://repository.maemo.org/extras/ diablo free non-free
deb http://repository.maemo.org/extras-devel/ diablo free non-free

apt-get update
apt-get install g++

done

Openmoko DNS settings

After several promising starts, I seem to have networking working reasonably reliably now. This is all available in the wiki on the site - so this is really just to remind me what the settings are for the resolv.conf after my office network moved to openDNS recently.

Home (wireless config)

echo nameserver 208.67.222.222
echo nameserver 208.67.220.220

Work (usb to computer)

search my-work.domain
nameserver 10.1.1.11
nameserver 10.1.1.12
nameserver 10.1.1.11
nameserver 10.1.1.12

No idea as yet how to automagically set these. Though this should get better. The network setup at work isn't helping though.

Quick bash script (check zero sized file) for reference

This is a quick bash script I wrote. Pasting it here for reference. It reads a bunch of ipar intensity files processes them with swift, and calibrates them with the swift calibrator. It checks if the file is zero sized (that's because I was updating a set of files some of which had failed). The:

   if [ ! -s /scratch/user/"$lane"_"$tile".fastq.pf.calibrated ]

That checks that a file /is/ zero sized. And only runs swift if it is.

sourcedir=/ADIRECTORY

for lane in {1..8}
do

for tile in {1..100}
do
 if [ ! -s /scratch/user/"$lane"_"$tile".fastq.pf.calibrated ]
   then
   nice cp "$sourcedir"/s_"$lane"_*0"$tile"_int.txt.p.gz /scratch/user/input.gz
   nice gzip -d /scratch/user/input.gz
   nice ./ipar2gapipelineint /scratch/user/input "$sourcedir"/s_"$lane"_*0"$tile"_pos.txt /scratch/user/gaint
   nice ./swift --purity_threshold 0.56 --intfile /scratch/user/gaint --ref ./phi.fa --optical_duplicates_distance 6 --optical_duplicates_mismatches 8 --fastq /scratch/user/"$lane"_"$tile".fastq --phasing_window 10 --phasing_iterations 3 --align_every 50
   nice ./QualityCalibration/quality_remap --in /scratch/user/"$lane"_"$tile".fastq.pf --mapfile ./QualityCalibration/firecrest_1_3_2_fixedtable --out /scratch/nava/"$lane"_"$tile".fastq.pf.calibrated
   rm /scratch/user/input
   rm /scratch/user/gaint
   rm /scratch/user/*.fastq.pf
   rm /scratch/user/*.fastq.nonpf
 fi
done
done

Palm Pre uses glibc and has busybox

This post has moved here

Making a C++ object printable to standard output

You want to make a C++ object printable to the standard output. For example you have an object call ScoredSequence and you want do be able to do: ScoredSequence s("BLAHBLAH"); cout << s; You need to add a << operator to ScoredSequence, for example:

class ScoredSequence {
public:

  ScoredSequence(string s) : sequence(s) {
  }


  string get_sequence() {
    return s;
  }

  /// \brief Extractor which prints the whole sequence.
  inline friend std::ostream& operator<<(std::ostream& out, const ScoredSequence& s) {
    out << s.get_sequence();
    return out;
  }

private:
  string sequence;
};

delete all linefeeds from a file

like this:

tr -d "\n" < gaps2 > nogaps

Connect to Freerunner Android from Opensuse

Plug in the freerunner, and yast2 should prompt you to configure a new network device of type usb. Follow the instructions, and configure using ifup (Not network manager) and use a static IP address for the freerunner. Assign it to 192.168.0.200.

The adb program is available here: http://people.openmoko.org/sean_mcneil/adb

Download it and put it somewhere and make it executable.

Then save this set of commands as a bash script with a name like connect or whatever.

#!/bin/bash

$/usr/local/share/adb kill-server
$ADBHOST=192.168.0.202 /usr/local/share/adb devices
$/usr/local/share/adb shell

Make this executable and then run it. It should drop you automatically into the shell interface on your freerunner.

Multi column table in latex

You want a table the spans across pages and raps in 2 columns. You need supertabular and to stick everything in a \twocolumn. Do it:

\pagebreak
\twocolumn
\begin{supertabular}{ |l|c|l| }
\hline
stuff1 & stuff2 & stuff3 \\
1 & 2 & 3 \\
1 & 2 & 3 \\
1 & 2 & 3 \\
1 & 2 & 3 \\
1 & 2 & 3 \\
1 & 2 & 3 \\
1 & 2 & 3 \\
1 & 2 & 3 \\
1 & 2 & 3 \\
1 & 2 & 3 \\
1 & 2 & 3 \\
1 & 2 & 3 \\
\hline
\end{supertabular}
\onecolumn
\pagebreak

latex "File ignored" not found on nfs ( isilon )

You are unable to compile latex files (i.e. latex myreport.tex) on an nfs drive but they compile correctly from the local disk. You receive errors such as "File ignored" or "no found". In my case this occurred when using an Isilon. The solution is to switch file ID size from 64bit to 32bit on the isilon configuration. "File Sharing"->NFS->"Edit Settings" on the Isilon web configuration.

Growing a VM

I started a VM with the default 8Gb for a linux install. I have successfully added additional HDD to the VM but eventually I hit a problem that required editing the actual size of the main disk. I used the vmware-vdiskmanager (-x 12Gb Kubuntu.vmdk) to successfully resize the VM disk. Then I booted the VM into a gparted image using the ISO as a CD (option in the VMware server). This worked fine. I then edited the partition. First increase the size of the swap. Then shift it to the right and then make it smaller again, before resizing the main partition. Click apply changes and then wait. Or in my case head up to the terrace and enjoy beer and "Schnittchen". The KDE session does appear borked after all this. It complains about some DCOP setting not running. I tried changing permissions on the tmp dir and following general advice on the internet - however moving .kde to .kde_old solved the problem for me. The partition is now showing up as 12 gigs and all the files seem to be okay.

Aluminium Mac Keyboard disassembly

This post has been moved here

avr-gcc read I/O on single port with AT90USB162

I want to repeatedly read from and write to a single port on a AT90USB162. This is to implement a key matrix scanner and pins will be repeated reconfigured as input or output. I ran in to trouble, reads seems to overlap as if they were being cached.



For example, I have pins 1 and 2 shorted.

pin 1 is set as an output. pin 2 an input.

pin 2 is set to pullup.

I output logic 0 on pin 1.

I read the value from pin 2 and store the result in a variable VAL1.



Now I set pin 1 as a input (and pullup).

I set another pin, say pin 3 to logic 0 (not shorted to anything).

I output logic 0 on pin 3.

I read the value from pin 2 in to a variable VAL2.





Now logically VAL1 should be 0 and VAL2 1. However that doesn't seem to be the case. If I do this I often read 1 for both VAL1 and VAL2. Removing the second read however makes VAL1 revert to it's correct value, EVEN THOUGH BOTH READS SHOULD BE INDEPENDENT!! I've not seen this with the AT90USBKey so it's a bit odd.


The solution I've found is to place a delay loop after the port configuration but before the read, in my code:

inline unsigned char get_scan(unsigned char pin) {

  unsigned char p = 1 << pin;

  DDR = 0x00 ^ p;
  PORTD = 0xFF ^ p;

  for(volatile int i=0;i<100;i++);

  return ~p & PIND;
}

The volatile is important, if you omit that it doesn't work. I guess the compiler optimizes it out. (volatile tells the compiler some other external code could change this variable so explicitly check it where ever it's referenced).

jvterm: text based cut and based

I've been trying to rid myself of the mouse. The final thing holding me back is cut and paste for terminal applications. Turns out that most Linux terminal programs don't have any support for keyboard based cut and paste. And in fact the only mainstream terminal app I've seen it in the The OS X Tiger Terminal.app (it seems to have been removed from later versions).



Anyway, after some hunting I found a Linux terminal program that does text based cut and paste. It's called jvterm. The downside is that though it's an X11 application it's not really designed to run in concert with X. It only works full screen and you can't alt-tab between it and X applications. Anyway, I couldn't find the documentation for text based selection so here's typically how you might use it:


Press Shift twice to enter selection mode

Use the cursor keys to move to the start of the selection region

Press Shift again

Move to the end of the selection region (text will be highlighted)

Press space


The text will now be on the clipboard. To paste it press Shift Twice and then Enter.


This is a really neat feature, and I hope it finds its way in to other terminal applications.



(protip: Read the man page. I found this out from guessing, but it's actually in the man page).





AHHH! You can do something similar with screen but how do I get it to share the X clipboard?!?




There's a urxvt (rxvt-unicode) perl script for this!

I found it in google cache and have copied it here:

HERE

To use it press alt-y to activate. Move using vi style keystrokes (hjkl) then press space to start selection and space to end. The text appears on you x buffer via xclip! Neato!

To install the script copy it to ~/.urxvt The copy the following in to you ~/.Xdefaults:

URxvt.keysym.M-y: perl:mark-and-yank:activate_mark_mode
URxvt.keysym.M-u: perl:mark-and-yank:activate_mark_url_mode
URxvt.perl-lib: /home/new/.urxvt/
URxvt.perl-ext-common: tabbed,mark-and-yank
URxvt.urlLauncher: firefox

Gaussian and Binomial distributions

I'm most probably being entirely stupid here...

I've been trying to understand the Gaussian distribution and it's relation to the Binomial distribution. I've kind of decided that the Gaussian distribution is a continuous representation of the Binomial distribution. Is this correct?

Anyway, here is some C++ which creates a binomial distribution by adding n random values (-1,0 or 1) to a fixed initial value. I think for this as a kind of 2D random walk. Here's the code:

#include <vector>
#include <iostream>
#include <math.h>

using namespace std;

int main(void) {

  vector<size_t> space(100,0);

  size_t tries=1000000;
  size_t offset=50;
  for(size_t n=0;n<tries;n++) {

    size_t moves=1000;

    int final=50;
    for(size_t t=0;t<moves;t++) {
      int val = rand()%3;
      val -= 1;

      final +=val;
    }
    space[final]++;
  }

  for(int n=0;n<space.size();n++) {
    cout << n << " " << space[n] << endl;
  }

  return 0;
}

Here's the gnuplot generated from the output:



If you have any pointer on the link between Binomial and Gaussian distributions explained in a way a computer scientist can understand please add them below.

Simple XY plot with gnuplot

This blog post has moved here

LaCie NAS Drive

I recently purchased a NAS ethernet drive from LaCie. They are pretty cheap these days - about a hundred quid. I'm sick of finding out that I downloaded a particular file to my laptop rather than my desktop or whatever and this seemed like a good solution. It works flawlessly on the MacBook through the LaCie provided software. Less so, with the openSuSe.

I ended up installing everything in yast2 that even tangentically mentioned samba or netbios. I then opened up my firewall to the fullest extent possible, sacrificed two virgins, restarted netbios and samba a couple of times and lo and behold it worked. I then switched the firewall back on with samba and netbios selected as allowed services for the external zone and it seems happy enough with that. No real lesson learned here, just black magic. This is really just a note for other to let them know that it could in theory work.

Mirroring a mediawiki to another server

There are lots of good guides on the interwebs on how to do this but I wanted to get a few notes down so I remember my preferred/installed method. These instructions are for Debian/Ubuntu and assume the target server doesn't have an existing mediawiki or mysql installation. All this needs to be done as root.

1. Install mediawiki on the target server, install the same version.

2. Delete /var/lib/mediawiki1.7 on the target server.

3. Dump the mysql permissions database on the source server:

mysqldump -u root -pYOURPASSWORD --opt mysql > ~/mysql.sql

4. Copy that file over to the target file. Import it using the following command.

mysql -pYOURPASSWORD -h 127.0.0.1 mysql < ~/mysql.sql

5. Setup a cron job on the source host to dump the mediawiki sql database every night. Type crontab -e to edit the crontab and add the following line:

0 0 * * * mysqldump -u root -pYOURPASSWORD --opt wikidb > ~/wikidb.sql

6. On the target host, add cron jobs to copy across the database dump, the mediawiki config/uploaded files and to import the mediawiki database. Again do crontab -e and add the following:

0 5 * * *  rsync -oglpvtr root@SOURCEHOST:/var/lib/mediawiki1.7 /var/lib > ~/backups/SOURCEHOST_mediawiki.log
0 5 * * *  rsync -vtr root@SOURCEHOST:/root/wikidb.sql ~/wikidb.sql
0 6 * * *  mysql -pYOURPASSWORD -h 127.0.0.1 wikidb < ~/wikidb.sql

Protip: you'll need your ssh keys setup so you can do passwordless authentication.

The Viterbi algorithm

For your viewing pleasure I present an implementation of the Viterbi algorithm. I implemented this to help me understand the algorithm as presented in "Biological Sequence Analysis" by Durbin et al. The code solves the "occasionally dishonest casino, part 2". The example displays the output as copied from the book as well as it's own calculation so you can verify the results.


Code here

tkz-graph example (A latex graph drawing package)

This blog post has moved here

PGF and tkz-graph installation in Ubuntu

Ubuntu likes to keep it's texlive packages at least a year out of date, so if you want to install tkz-graph you need to install it and pgf manually. Create a directory called texmf in your home directory (mkdir ~/texmf). Create a subdirectory called tex (mkdir ~/texmf/tex). Create subdirectories of this called generic and latex. From the pgf tarball copy pgf-2.00/generic into the generic subdirectory. Copy pgf-2.00/latex to the latex subdirectory (you now have ~/texmf/generic/pgf and ~/texmf/latex/pgf. Download the tkz-graph style (tkz-graph.sty) and copy it to ~/texmf/latex. I copied it to ~/texmf/latex/prof I don't think that matters... Next "sudo texhash". That should do it.

Download a bunch of papers from a journal

Say you want to download a bunch of pdf files from a webpage. You want to do this remotely over ssh so those cool firefox extensions are our of the question. Use wget. For example:

tsocks wget -r -l1 -t1 -N -A.pdf -erobots=off http://www.springerlink.com/content/x49g43t85772/ 

Does the job.

Openmoko freerunner with vodafone DE sim card

Just a note to mention for anyone curious that I managed to get a 3G german vodafone sim working with OM 2008.12 by flashing the gsm firmware to moko10. Whilst the wiki advises using shr I found it didn't make any difference. Still have to test this with 3 cards from the UK but hopefully some of them will work now too.

Silc client won~t accept the / (backslash) key

I use silc as a replacement for things like irc. Most of the time I think its great. It provides secure and simple encryption and allows me to waste time instead of actually working. Well that last bit is a tad unfair as I learn a lot of things on there.

Anyways, for the last while, it has been impossible for me to enter the backslash character in a session. This is regardless of the keyboard layout on the client computer, the OS and was a seemingly random error. Today I erased the .silc/silc.conf file and restarted the silc client. Success - I am now able to connect to the server once again. The most infuriating aspect of this is that the error occured during a running session. When I copied urls into that session none of the slash characters would be displayed on the screen. Which was funny at first but very annoying once I restarted silc only to discover that I couldn't now type /connect. Which was funny for about 30 seconds. So there you go. If you are ever in the situation where you can't enter the slash key in a silc client just remove the configuration file.

Replacing a string in vim to fix an dxf file

I have an DXF file that has entries that look like this:

LCD.1
70
64
62
150
6
CONTINUOUS
0
LAYER
2
LCDMOUNT.1
70
64
62
252
6

Autocad doesn't like this file... I have no idea what created it. After much screwing around I've discovered that it doesn't like the "."s in the entity (or whatever they're called) names. Load the file in vim and use the following regexp to get rid of the dots.

%s/\([A-Z]\)\.\([0-9]\+\)/\1\2/g

Fixed file looks like this:

LCD1
70
64
62
150
6
CONTINUOUS
0
LAYER
2
LCDMOUNT1
70
64
62
252
6

FYI, the \( \) are delimiters in the search sequence. \1 and \2 reference back to these.

Addition: The file in question was produced by a program called "Camera 3D" or "3D Camera" or something... Just in case anyone else has the same problem with this code.

Merging in Subversion

Merging in subversion doesn't work like you'd think it'd work. You don't say "take this branch and apply all the changes to trunk". No... you say "calculate the differences between this branch and and older revision then apply these changes to trunk". That's very different. Because of this you need to note the revision at which you're branch was made and use this when merging. Here's a quick example:

Make a branch:

svn copy trunk ./branches/nw3

NOTE THE REVISION NUMBER (basenum) AND LEAVE A USEFUL LOG COMMENT. OK, now make edits commit them etc on the branch. At some point you want to merge the changes in to trunk (run this while in the trunk directory):

svn merge -r basenum:currentrev https://swiftng.svn.sourceforge.net/svnroot/swiftng/trunk

Where basenum is the revision number at which the branch was made and currentrev is the current revision.

Beagleboard GPIO Input (driverless)

This blog post has moved HERE

Disk space usage

My VM keeps getting filled up, and then crashing cos the programs that write logs are not being correctly handled by the underlying FS. So I am interested in generating useful infos on the disk to find out where all the disk space is at. The command du is a big help here. But I frequently just want to know in which directories the most space is being used.

du -h --max-depth=1

That command will give you a listing of which directories under the current dir are using all the space. So in the home dir or at the usr dir run this and find out where all the unneccessary crap is.

Giving me in the /usr dir the following output:

1.1G    ./share
40K     ./games
1.4G    ./local
7.1M    ./sbin
125M    ./src
36K     ./X11R6
136M    ./bin
14M     ./include
1.6G    ./lib
4.3G    .

And this tells me that I ought to tidy up the /usr/local dir. If you ran the above command without the max-depth option then the resulting output will probably overrun your cache/history in your bash shell and besides the output would be impossible to make sense of.