# This blog post has moved here

## Friday, December 4, 2009

### returning by reference faster than returning by value?

Is returning by reference faster than returning by value? It's always been my assumption that returning by value can in many cases be optimized away. However to test this assumption I wrote a couple of small tests:

By value:
#include <math.h>#include <iostream>#include <vector>using namespace std;vector<size_t> test() {  vector<size_t> t;  for(size_t n=0;n<100;n++) {    t.push_back(rand());  }  return t;}int main() {  size_t sum=0;  for(size_t n=0;n<10000000;n++){    vector<size_t> t = test();    for(size_t n=0;n<t.size();n++) {      sum = sum + t[n];    }  }  cout << sum << endl;}

By reference:

#include <math.h>#include <iostream>#include <vector>using namespace std;void test(vector<size_t> &t) {  for(size_t n=0;n<100;n++) {    t.push_back(rand());  }}int main() {  size_t sum=0;  for(size_t n=0;n<10000000;n++){    vector<size_t> t;    test(t);    for(size_t n=0;n<t.size();n++) {      sum = sum + t[n];    }  }  cout << sum << endl;}

Here are my timings on an Intel Core2Duo:

By value:
real 0m34.122s
user 0m33.219s
sys 0m0.436s

By reference:
real 0m34.147s
user 0m33.308s
sys 0m0.512s

So in this case the user times seem to be more or less the same. What's happening is the return value being optimized out as expected? Is there something else I should take in to account in my test?

i686-apple-darwin10-g++-4.2.1 was used.

http://cpp-next.com/archive/2009/08/want-speed-pass-by-value/

Also interesting comment on gcc mailing list:

Your subject isn't quite right.  It should be "Returning by reference faster than returning by value".                                                                                                                                                                                                                                                                                                                                                                                  C++ has RVO ("return value optimization").                                                                                                                                                                                                                                                                                                                                                                                                                                              RVO is part of C++, just as the elide constructors is a part of C++.                                                                                                                                                                                                                                                                                                                                                                                                                    Also note:  you get RVO even if you are -O0.  It really is part of the C++ language, even for non-optimized compiling.                                                                                                                                                                                                                                                                                                                                                                  So your two tests as given should have the about the same performance, since the two tests are basically doing the same thing.  And, indeed, they do have about the same performance.  As expected.                                                                                                                                                                                                                                                                                     Read carefully the constraints upon when RVO can be applied.  (I don't have a URL handy.  You'll have to use your Google-fu, or read the ISO 14882 standard.)                                                                                                                                                                                                                                                                                                                           In situations where RVO cannot be applied, then for non-trivial data types, an output reference parameter will (likely) be faster.                                                                                                                                                                                                                                                                                                                                                      A lot of STL and Boost rely upon RVO for their template magic to be highly performant.

## Wednesday, December 2, 2009

### Reverting an svn commit

To revert an svn commit do something like:

svn merge -r 4552:4551 svn+ssh://svn/path_to_svn_location
svn commit

Where 4552 is the last commit and 4551 is the one before.

## Tuesday, December 1, 2009

### Simple quicksort implementation

A simple C++ quicksort implementation. This version is not in place and therefore requires O(n) extra space...

#include "math.h"#include <vector>#include <iostream>using namespace std;template<class value_type>void quick_sort(vector<value_type> &values,size_t start,size_t end,size_t pivot) {  vector<value_type> less;  vector<value_type> greater;  for(size_t n=start;n<end;n++) {    if(n != pivot)    if(values[n] < values[pivot]) {      less.push_back(values[n]);    } else {      greater.push_back(values[n]);    }  }  value_type pivot_temp = values[pivot];  for(size_t n=0;n<less.size();n++) {    values[n+start] = less[n];  }  values[start+less.size()] = pivot_temp;  for(size_t n=0;n<greater.size();n++) {    values[n+start+less.size()+1] = greater[n];  }  if(less.size()    > 1) { quick_sort(values,start      ,start+less.size(),start+(less.size()/2)); }  if(greater.size() > 1) { quick_sort(values,less.size(),end              ,less.size()+(end/2)  ); }}int main() {  vector<size_t> values;  for(size_t n=0;n<10;n++) {    values.push_back(rand());  }  cout << "start values..." << endl;  for(size_t n=0;n<values.size();n++) {    cout << "val[" << n << "]: " << values[n] << endl;  }  quick_sort(values,0,values.size(),5);  cout << "sorted values..." << endl;  for(size_t n=0;n<values.size();n++) {    cout << values[n] << endl;  }}

## Tuesday, November 24, 2009

### Templates v subclasses v.2

A slightly more complicated test similar to the previous post. In this case the templated version doesn't have such a huge advantage.

Subclassed version:

#include <iostream>using namespace std;class mybaseclass {  public:    virtual int get_i() = 0;};class myclass : public mybaseclass {  public:    myclass(int in) {      i=in;    }    int get_i() {      i++;      return i;    }  int i;};class otherclass : public mybaseclass {  public:    otherclass(int in) {      i = in;    }    int get_i() {      return i*i;    }  int i;};int main(void) {  myclass *c = new myclass(2);  double sum=0;  for(size_t n=0;n<2000000000;n++) {    mybaseclass *cc = c;    sum += (static_cast<double>(cc->get_i())/2.2);  }  cout << sum << endl;  delete c;}

Templated version:
#include <iostream>using namespace std;template<class t>class myclass {  public:    myclass(int in) {      i=in;    }    inline int get_i() {      i++;      return i;    }  int i;};template<class t>class otherclass {  public:    otherclass(int in) {      i = in;    }    inline int get_i() {      return i*i;    }  int i;};int main(void) {  myclass<int> *c = new myclass<int>(2);  double sum=0;  for(size_t n=0;n<2000000000;n++) {    sum += (static_cast<double>(c->get_i())/2.2);  }  cout << sum << endl;  delete c;}

$g++ small_classed.cpp -O3;time ./a.out 9.09091e+17 real 0m21.151s user 0m20.909s sys 0m0.072s$ g++ small_templateclassed.cpp -O3;time ./a.out
9.09091e+17

real 0m20.736s
user 0m20.527s
sys 0m0.064s

### Are templates faster than subclasses?

Are template classes faster than using a derived class? Intuition would tell me that the template class should be faster. However to test if this is the case I wrote the following small programs:

Subclassed version:

#include <iostream>using namespace std;class mybaseclass {  public:    virtual int get_i() = 0;};class myclass : public mybaseclass {  public:    myclass(int in) {      i=in;    }    int get_i() {      return i;    }  int i;};class otherclass : public mybaseclass {  public:    otherclass(int in) {      i = in;    }    int get_i() {      return i*i;    }  int i;};int main(void) {  myclass *c = new myclass(2);  long int sum=0;  for(size_t n=0;n<2000000000;n++) {    mybaseclass *cc = c;    sum += cc->get_i();  }  cout << sum << endl;  delete c;}

Templated version:
#include <iostream>using namespace std;template<class t>class myclass {  public:    myclass(int in) {      i=in;    }    int get_i() {      return i;    }  int i;};template<class t>class otherclass {  public:    otherclass(int in) {      i = in;    }    int get_i() {      return i*i;    }  int i;};int main(void) {  myclass<int> *c = new myclass<int>(2);  long int sum=0;  for(size_t n=0;n<2000000000;n++) {    sum += c->get_i();  }  cout << sum << endl;  delete c;}

$g++ small_templateclassed.cpp -O3;time ./a.out 4000000000 real 0m0.008s user 0m0.000s sys 0m0.001s$ g++ small_classed.cpp -O3;time ./a.out
4000000000

real 0m5.908s
user 0m5.853s
sys 0m0.018s

My first effort was IO bound, but in this case it appears that templatized code is substantially faster.

## Friday, November 20, 2009

### Renicing synology nfsd so it has priority over other services

We needed nfs to have priority over other services on a synology DS509+. To do this we reniced the nfsd process. Synologies are basically just linux boxes. So enable ssh on the synology and ssh to it (ssh synology -lroot) use your normal admin password for the password. Then edit /usr/syno/etc/rc.d/S83nfsd.sh. Where it currently reads:

/sbin/portmap
/usr/sbin/nfsd
/usr/sbin/mountd -p 892
;;
stop)

/usr/sbin/nfsd
/usr/sbin/mountd -p 892
renice -10 pidof nfsd

;;
stop)

## Monday, November 16, 2009

Commands I used to install Go language (golang) on Ubuntu amd64:sudo apt-get install mercurial bison gcc libc6-dev ed makecd ~export GOROOT=$HOME/goexport GOARCH=amd64export GOOS=linuxmkdir$HOME/gobinexport GOBIN=$HOME/gobinexport PATH=$PATH:$GOBINhg clone -r release https://go.googlecode.com/hg/$GOROOTcd $GOROOT/src./all.bash To compile something make a test program e.g.: package mainimport "fmt"func main() { fmt.Printf("hello, world\n")} and compile with: 8g test.go ;8l test.8 ;./8.out ### Version Hi, Probably did this before - but administered lost and found computers has made me need this info again. Version of linux. cat /proc/version Gives you: Linux version 2.6.28-15-generic (buildd@rothera) (gcc version 4.3.3 (Ubuntu 4.3.3-5ubuntu4) ) #52-Ubuntu SMP Wed Sep 9 10:49:34 UTC 2009 And then you need the release info for the distribution. lsb_release -a Gives you: No LSB modules are available.Distributor ID: UbuntuDescription: Ubuntu 9.04Release: 9.04Codename: jaunty And of course uname -a. Which is a start. Would love to hear other tips on determining the state of a box that has previously been abandoned. Update: Naming of the actual machine and shit as well. Running hostname should give you the name of the actual machine. Not entirely sure how this works when you have multiple "machines" kinda running off the same physical machine. user@machine:~$ hostnamemachine-name

You can then check this the other way round:
user@machine:~$host name.domain.tldip address .in-addr.arpa domain name pointer name.domain.tld ## Sunday, November 15, 2009 ### list installed packages on debian and uninstall those not correctly removed. dpkg --get-selections Will give you a list of packages currently installed on a debian system (possibly works for Ubuntu too?). You may see a few which are listed as "deinstall". They've not uninstalled correctly. To uninstall them use this command I found on the debian mailing lists to purge them: dpkg --get-selections | grep deinstall | cut -f1 | xargs dpkg -P ## Friday, November 13, 2009 ### Turning a Ubuntu install into a Window terminal server dumb terminal I've written a quick script to term a clean Ubuntu (I'm using 9.04 netbook remix) into a dumb terminal to connect to a windows terminal server. You can use it like this: wget http://sgenomics.org/~new/term.tar tar xvf term.tar sudo ./termsvr MYTERMINALSERVERNAME MYDOMAINNAME Under the hood, the script looks like this: if [$# -ne 2 ]
then
echo "Usage: basename $0 " exit 1 fi apt-get -y --force-yes remove gnome* apt-get update apt-get install rdesktop cp ./rc.local /etc/rc.local sed "s/\*SERVERNAME\*/$1/" ./xinitrc > ./xinitrc.fixed0

## Sunday, September 27, 2009

### Rename files matching given spec to incrementing filenames

This one liner renames all files starting with shred_ to shred_0.tif, shred_1.tif etc.

ls shred_* | awk 'BEGIN{n=0;}{print "mv " 0 " shred_" n ".tif"; n++;}' | sh ## Wednesday, September 23, 2009 ### My .mailcap Used with mutt to view gifs and jpegs as ascii graphics!!!! requires aview package on ubuntu/debian: image/gif; asciiview '%s'image/png; asciiview '%s'image/jpg; asciiview '%s'image/jpeg; asciiview '%s' Robbed off some random place on the internets. ## Tuesday, September 22, 2009 ### Simple pysam example This python code uses pysam to parse a bam file and print out all the alignment contained therein. import pysam# Open our sam filesamfile = pysam.Samfile()samfile.open( "sim.bam", "rb" )# Grab an alignment from the sam filedef m_fetch_callback( alignment ): print str(alignment)num_targets = samfile.getNumTargets()for n in xrange(num_targets): samfile.fetch(samfile.getTarget(n),m_fetch_callback) ## Friday, September 18, 2009 ### Simple zlib file reading example This C++ example can read from both compressed and compressed files. It will read from testfile.gz (which can either be compressed or uncompressed) and write the contents to the standard output (screen). #include "zlib.h"#if defined(MSDOS) || defined(OS2) || defined(WIN32) || defined(__CYGWIN__)# include <fcntl.h># include <io.h># define SET_BINARY_MODE(file) setmode(fileno(file), O_BINARY)#else# define SET_BINARY_MODE(file)#endif#include <iostream>using namespace std;int main(void) { gzFile f = gzopen("testfile.gz","rb"); char buffer[1001]; for(;!gzeof(f);) { int e = gzread(f,buffer,1000); buffer[1000] = 0; cout << buffer << endl; } gzclose(f);} Compile with g++ testprog.cpp -lz -o testprog ### Count files matching a given type/find spec Count file size in bytes of all files matching *fast4 modified in the last 90 days. x=find . -atime 90 -name \*fast4 -ls | awk '{print7 " +"}' | xargs echo; echo "$x 0" | bc ### examine irssi scrollback I always find this a pain to lookup. But to scrollback in irssi: esc then p, forward esc then n. ## Wednesday, September 9, 2009 ### iWorks '09 Keynote hangs loading a ppt file potentially losing all your work I had a problem. I'd been working all day on a presentation which I hadn't saved (yea I know). Then I tried to load a powerpoint file and Keynote just hung trying to load the file. If I couldn't get it to unhang I would have lost a days work as Keynote doesn't autosave. The solution is to attach gdb to the Keynote and get it to jump out of whichever loop it's stuck in. Like so: news-macbook:MacOS new$ ps -ef | grep Keynote 501 13290    82   0   1:22.74 ??        46:51.60 /Applications/iWork '09/Keynote.app/Contents/MacOS/Keynote -psn_0_1552763news-macbook:MacOS new$grep -p 13290grep: invalid option -- pUsage: grep [OPTION]... PATTERN [FILE]...Try grep --help' for more information.news-macbook:MacOS new$ gdb -p 13290GNU gdb 6.3.50-20050815 (Apple version gdb-1344) (Fri Jul  3 01:19:56 UTC 2009)Copyright 2004 Free Software Foundation, Inc.GDB is free software, covered by the GNU General Public License, and you arewelcome to change it and/or distribute copies of it under certain conditions.Type "show copying" to see the conditions.There is absolutely no warranty for GDB.  Type "show warranty" for details.This GDB was configured as "x86_64-apple-darwin"./Applications/iWork '09/Keynote.app/Contents/MacOS/13290: No such file or directoryAttaching to process 13290.Reading symbols for shared libraries . doneReading symbols for shared libraries ................................................................................................................................................................................................................................ done0x942ed26d in ___CFBasicHashFindBucket1 ()(gdb) bt#0  0x942ed26d in ___CFBasicHashFindBucket1 ()#1  0x942edd5b in CFBasicHashFindBucket ()#2  0x942edcd4 in -[NSClassicMapTable objectForKey:] ()#3  0x942edca8 in NSMapGet ()#4  0x008dcf77 in -[ADReference reference] ()#5  0x008db594 in -[ADNode firstChildWithName:xmlNamespace:objcClass:resolveReference:] ()#6  0x008db3d2 in -[ADNode firstChildWith:resolveReference:] ()#7  0x008db369 in -[ADNode firstChildWith:] ()#8  0x009036b6 in -[ADSStyleSheet(ADInternal) parentSheet] ()#9  0x00903f22 in -[ADSStyleSheet(Private) recursiveStyleWithIdentifier:] ()#10 0x00903613 in -[ADSStyleSheet ensureOwnStyleWithIdentifierExists:] ()#11 0x005f50a7 in +[PIOGraphicStyleMapper mapMasterStylesFromPptSlide:toStyleSheet:withState:] ()#12 0x005fbe13 in -[PIOMasterSlideMapper mapMasterStyles] ()#13 0x005fbf9d in -[PIOMasterSlideMapper map] ()#14 0x00610ea9 in -[PIOThemeMapper mapMaster:includingGraphics:] ()#15 0x00610f94 in -[PIOThemeMapper mapMaster:] ()#16 0x006111b1 in -[PIOThemeMapper mapMasters] ()#17 0x00611285 in -[PIOThemeMapper map] ()#18 0x005f28c1 in -[PIODocMapper(Private) mapThemes] ()#19 0x005f238b in -[PIODocMapper map] ()#20 0x005f9467 in -[PIOImporter readFromFile:binaryDirectory:templatePath:localizer:] ()#21 0x0022e0e2 in ?? ()#22 0x0022e2d0 in ?? ()#23 0x0022f789 in ?? ()#24 0x0022ecea in ?? ()#25 0x000a8c5f in ?? ()#26 0x000a875f in ?? ()#27 0x96c08769 in -[NSDocument initWithContentsOfURL:ofType:error:] ()#28 0x96c0830d in -[NSDocumentController makeDocumentWithContentsOfURL:ofType:error:] ()#29 0x000163ed in ?? ()#30 0x00096fb4 in ?? ()#31 0x02ddac2e in -[SFAppApplicationDelegate(Private) pOpenDocumentWithContentsOfFile:] ()#32 0x02dd822d in -[SFAppApplicationDelegate application:openFiles:] ()#33 0x001f7f55 in ?? ()#34 0x96c0637d in -[NSApplication(NSAppleEventHandling) _handleAEOpenDocumentsForURLs:] ()#35 0x96b44104 in -[NSApplication(NSAppleEventHandling) _handleCoreEvent:withReplyEvent:] ()#36 0x9432346c in -[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon:] ()#37 0x94323230 in _NSAppleEventManagerGenericHandler ()#38 0x90905de6 in aeDispatchAppleEvent ()#39 0x90905ce5 in dispatchEventAndSendReply ()#40 0x90905bf2 in aeProcessAppleEvent ()#41 0x9061a381 in AEProcessAppleEvent ()#42 0x969bddd6 in _DPSNextEvent ()#43 0x969bd40e in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()#44 0x9697f5fb in -[NSApplication run] ()#45 0x0000f13f in ?? ()#46 0x0000f0b8 in ?? ()#47 0x0000ee50 in ?? ()#48 0x0005cc79 in ?? ()(gdb) retMake selected stack frame return now? (y or n) y

Keep 'ret'ing until you reach 'readFromFile'. At this point type 'c'. Keynote will then tell you it couldn't open the file and you will be able to save your existing document.

## Thursday, September 3, 2009

### Postfix won't start

Just noticed I wasn't receiving system mail for the last few weeks. Coming back after holiday. Anyway, one of the few services not to restart after the computer had been switched off in my absence was postfix.

Couldn't for the life of me work out the problem, it was complaining about not finding the host. Therefore I had a look in /etc/hosts and commented out the line below:

## hosts         This file describes a number of hostname-to-address#               mappings for the TCP/IP subsystem.  It is mostly#               used at boot time, when no name servers are running.#               On small systems, this file can be used instead of a#               "named" name server.# Syntax:## IP-Address  Full-Qualified-Hostname  Short-Hostname#127.0.0.1       localhost# special IPv6 addresses#::1             localhost ipv6-localhost ipv6-loopbackfe00::0         ipv6-localnetff00::0         ipv6-mcastprefixff02::1         ipv6-allnodesff02::2         ipv6-allroutersff02::3         ipv6-allhosts

The one about IPv6. Dunno what that was about - but it works now.

# This post is now here

## Monday, August 24, 2009

### Remove any linefeed from line which did not end in capital or special character (vim)

Remove any linefeed from line which did not end in capital or one of ./?<>:'

Like so:

%s/$$[^./?<>":'A-Z]$$\n/\1/g

## Wednesday, August 19, 2009

Be the envy of your friends! Remote control your PC via twitter! Add the following code to cron to send all tweets with CMD in them to the shell.

curl twitter.com/new299 | html2text | grep "CMD" | awk '{$1="";$0=substr(0,2)}1' | sh ### Simulate dataset with samtools, align with Bowtie, BWA, MAQ and pileup # This post has moved here ## Wednesday, August 12, 2009 ### Reset author to filename on a bunch of pdf files This script will reset the author to metadata on all files in the current directory to be the same as the filename of the file. It will write new files to a sub directory (which should already exist) called fixed. Quick hacky script to so that papers are index more sensibility on my Sony Reader. for i in *.pdfdo echoi        echo "InfoKey: Author" > data.txt        echo "InfoValue: " $i >> data.txt cat data.txt pdftk ./$i update_info data.txt output ./fixed/idone ## Thursday, August 6, 2009 ### ö (O with umlaut) on Mac OS X # This post has moved here ## Wednesday, August 5, 2009 ### Running samtools pileup on some aligned reads samtools needs a sorted file for pileup.. doesn't give you an error if it isn't sorted so watch out! ./samtools faidx phi.fa./samtools import ./phi.fa.fai /scratch/nava/lane7.phi.sam /scratch/nava/lane7.phi.bam./samtools sort /scratch/nava/lane7.phi.bam /scratch/nava/lane7.phi.bam.sort./samtools pileup -f ./phi.fa /scratch/nava/lane7.phi.bam.sort.bam > pile ### vim/vi inverted replace Replace any character that isn't the ^ character with nothing: %s/[^^]//g ## Saturday, August 1, 2009 ### n810 g++ installation (os 2008.1 diablo) Install the following repositories by editing /etc/apt/sources.list.d/hildon-application-manager.list add the following: deb http://repository.maemo.org/ diablo sdk/free sdk/non-free tools/free tools/non-free deb http://repository.maemo.org/extras/ diablo free non-free deb http://repository.maemo.org/extras-devel/ diablo free non-free apt-get update apt-get install g++ done ## Tuesday, July 28, 2009 ### Openmoko DNS settings After several promising starts, I seem to have networking working reasonably reliably now. This is all available in the wiki on the site - so this is really just to remind me what the settings are for the resolv.conf after my office network moved to openDNS recently. Home (wireless config) echo nameserver 208.67.222.222echo nameserver 208.67.220.220 Work (usb to computer) search my-work.domainnameserver 10.1.1.11nameserver 10.1.1.12nameserver 10.1.1.11nameserver 10.1.1.12 No idea as yet how to automagically set these. Though this should get better. The network setup at work isn't helping though. ## Friday, July 17, 2009 ### Quick bash script (check zero sized file) for reference This is a quick bash script I wrote. Pasting it here for reference. It reads a bunch of ipar intensity files processes them with swift, and calibrates them with the swift calibrator. It checks if the file is zero sized (that's because I was updating a set of files some of which had failed). The:  if [ ! -s /scratch/user/"lane"_"$tile".fastq.pf.calibrated ] That checks that a file /is/ zero sized. And only runs swift if it is. sourcedir=/ADIRECTORYfor lane in {1..8}dofor tile in {1..100}do if [ ! -s /scratch/user/"$lane"_"$tile".fastq.pf.calibrated ] then nice cp "$sourcedir"/s_"$lane"_*0"$tile"_int.txt.p.gz /scratch/user/input.gz   nice gzip -d /scratch/user/input.gz   nice ./ipar2gapipelineint /scratch/user/input "$sourcedir"/s_"$lane"_*0"$tile"_pos.txt /scratch/user/gaint nice ./swift --purity_threshold 0.56 --intfile /scratch/user/gaint --ref ./phi.fa --optical_duplicates_distance 6 --optical_duplicates_mismatches 8 --fastq /scratch/user/"$lane"_"tile".fastq --phasing_window 10 --phasing_iterations 3 --align_every 50 nice ./QualityCalibration/quality_remap --in /scratch/user/"lane"_"$tile".fastq.pf --mapfile ./QualityCalibration/firecrest_1_3_2_fixedtable --out /scratch/nava/"$lane"_"$tile".fastq.pf.calibrated rm /scratch/user/input rm /scratch/user/gaint rm /scratch/user/*.fastq.pf rm /scratch/user/*.fastq.nonpf fidonedone ## Wednesday, June 10, 2009 ### Palm Pre uses glibc and has busybox # This post has moved here ## Sunday, May 17, 2009 ### Making a C++ object printable to standard output You want to make a C++ object printable to the standard output. For example you have an object call ScoredSequence and you want do be able to do: ScoredSequence s("BLAHBLAH"); cout << s; You need to add a << operator to ScoredSequence, for example: class ScoredSequence {public: ScoredSequence(string s) : sequence(s) { } string get_sequence() { return s; } /// \brief Extractor which prints the whole sequence. inline friend std::ostream& operator<<(std::ostream& out, const ScoredSequence& s) { out << s.get_sequence(); return out; }private: string sequence;}; ## Tuesday, May 12, 2009 ### delete all linefeeds from a file like this: tr -d "\n" < gaps2 > nogaps ## Thursday, May 7, 2009 ### Connect to Freerunner Android from Opensuse Plug in the freerunner, and yast2 should prompt you to configure a new network device of type usb. Follow the instructions, and configure using ifup (Not network manager) and use a static IP address for the freerunner. Assign it to 192.168.0.200. The adb program is available here: http://people.openmoko.org/sean_mcneil/adb Download it and put it somewhere and make it executable. Then save this set of commands as a bash script with a name like connect or whatever. #!/bin/bash$/usr/local/share/adb kill-server$ADBHOST=192.168.0.202 /usr/local/share/adb devices$/usr/local/share/adb shell

Make this executable and then run it. It should drop you automatically into the shell interface on your freerunner.

## Thursday, April 30, 2009

### Multi column table in latex

You want a table the spans across pages and raps in 2 columns. You need supertabular and to stick everything in a \twocolumn. Do it:

\pagebreak\twocolumn\begin{supertabular}{ |l|c|l| }\hlinestuff1 & stuff2 & stuff3 \\1 & 2 & 3 \\1 & 2 & 3 \\1 & 2 & 3 \\1 & 2 & 3 \\1 & 2 & 3 \\1 & 2 & 3 \\1 & 2 & 3 \\1 & 2 & 3 \\1 & 2 & 3 \\1 & 2 & 3 \\1 & 2 & 3 \\1 & 2 & 3 \\\hline\end{supertabular}\onecolumn\pagebreak

## Friday, April 17, 2009

You are unable to compile latex files (i.e. latex myreport.tex) on an nfs drive but they compile correctly from the local disk. You receive errors such as "File ignored" or "no found". In my case this occurred when using an Isilon. The solution is to switch file ID size from 64bit to 32bit on the isilon configuration. "File Sharing"->NFS->"Edit Settings" on the Isilon web configuration.

## Friday, April 3, 2009

### Growing a VM

I started a VM with the default 8Gb for a linux install. I have successfully added additional HDD to the VM but eventually I hit a problem that required editing the actual size of the main disk. I used the vmware-vdiskmanager (-x 12Gb Kubuntu.vmdk) to successfully resize the VM disk. Then I booted the VM into a gparted image using the ISO as a CD (option in the VMware server). This worked fine. I then edited the partition. First increase the size of the swap. Then shift it to the right and then make it smaller again, before resizing the main partition. Click apply changes and then wait. Or in my case head up to the terrace and enjoy beer and "Schnittchen". The KDE session does appear borked after all this. It complains about some DCOP setting not running. I tried changing permissions on the tmp dir and following general advice on the internet - however moving .kde to .kde_old solved the problem for me. The partition is now showing up as 12 gigs and all the files seem to be okay.

## Saturday, March 7, 2009

### Aluminium Mac Keyboard disassembly

This post has been moved here

## Friday, March 6, 2009

### avr-gcc read I/O on single port with AT90USB162

I want to repeatedly read from and write to a single port on a AT90USB162. This is to implement a key matrix scanner and pins will be repeated reconfigured as input or output. I ran in to trouble, reads seems to overlap as if they were being cached.

For example, I have pins 1 and 2 shorted.

pin 1 is set as an output. pin 2 an input.

pin 2 is set to pullup.

I output logic 0 on pin 1.

I read the value from pin 2 and store the result in a variable VAL1.

Now I set pin 1 as a input (and pullup).

I set another pin, say pin 3 to logic 0 (not shorted to anything).

I output logic 0 on pin 3.

I read the value from pin 2 in to a variable VAL2.

Now logically VAL1 should be 0 and VAL2 1. However that doesn't seem to be the case. If I do this I often read 1 for both VAL1 and VAL2. Removing the second read however makes VAL1 revert to it's correct value, EVEN THOUGH BOTH READS SHOULD BE INDEPENDENT!! I've not seen this with the AT90USBKey so it's a bit odd.

The solution I've found is to place a delay loop after the port configuration but before the read, in my code:

inline unsigned char get_scan(unsigned char pin) {  unsigned char p = 1 << pin;  DDR = 0x00 ^ p;  PORTD = 0xFF ^ p;  for(volatile int i=0;i<100;i++);  return ~p & PIND;}

The volatile is important, if you omit that it doesn't work. I guess the compiler optimizes it out. (volatile tells the compiler some other external code could change this variable so explicitly check it where ever it's referenced).

## Wednesday, March 4, 2009

### jvterm: text based cut and based

I've been trying to rid myself of the mouse. The final thing holding me back is cut and paste for terminal applications. Turns out that most Linux terminal programs don't have any support for keyboard based cut and paste. And in fact the only mainstream terminal app I've seen it in the The OS X Tiger Terminal.app (it seems to have been removed from later versions).

Anyway, after some hunting I found a Linux terminal program that does text based cut and paste. It's called jvterm. The downside is that though it's an X11 application it's not really designed to run in concert with X. It only works full screen and you can't alt-tab between it and X applications. Anyway, I couldn't find the documentation for text based selection so here's typically how you might use it:

Press Shift twice to enter selection mode

Use the cursor keys to move to the start of the selection region

Press Shift again

Move to the end of the selection region (text will be highlighted)

Press space

The text will now be on the clipboard. To paste it press Shift Twice and then Enter.

This is a really neat feature, and I hope it finds its way in to other terminal applications.

(protip: Read the man page. I found this out from guessing, but it's actually in the man page).

AHHH! You can do something similar with screen but how do I get it to share the X clipboard?!?

There's a urxvt (rxvt-unicode) perl script for this!

I found it in google cache and have copied it here:

HERE

To use it press alt-y to activate. Move using vi style keystrokes (hjkl) then press space to start selection and space to end. The text appears on you x buffer via xclip! Neato!

To install the script copy it to ~/.urxvt The copy the following in to you ~/.Xdefaults:

URxvt.keysym.M-y: perl:mark-and-yank:activate_mark_modeURxvt.keysym.M-u: perl:mark-and-yank:activate_mark_url_modeURxvt.perl-lib: /home/new/.urxvt/URxvt.perl-ext-common: tabbed,mark-and-yankURxvt.urlLauncher: firefox

## Sunday, March 1, 2009

### Gaussian and Binomial distributions

I'm most probably being entirely stupid here...

I've been trying to understand the Gaussian distribution and it's relation to the Binomial distribution. I've kind of decided that the Gaussian distribution is a continuous representation of the Binomial distribution. Is this correct?

Anyway, here is some C++ which creates a binomial distribution by adding n random values (-1,0 or 1) to a fixed initial value. I think for this as a kind of 2D random walk. Here's the code:

#include <vector>#include <iostream>#include <math.h>using namespace std;int main(void) {  vector<size_t> space(100,0);  size_t tries=1000000;  size_t offset=50;  for(size_t n=0;n<tries;n++) {    size_t moves=1000;    int final=50;    for(size_t t=0;t<moves;t++) {      int val = rand()%3;      val -= 1;      final +=val;    }    space[final]++;  }  for(int n=0;n<space.size();n++) {    cout << n << " " << space[n] << endl;  }  return 0;}

Here's the gnuplot generated from the output:

If you have any pointer on the link between Binomial and Gaussian distributions explained in a way a computer scientist can understand please add them below.

# This blog post has moved here

## Sunday, February 22, 2009

### LaCie NAS Drive

I recently purchased a NAS ethernet drive from LaCie. They are pretty cheap these days - about a hundred quid. I'm sick of finding out that I downloaded a particular file to my laptop rather than my desktop or whatever and this seemed like a good solution. It works flawlessly on the MacBook through the LaCie provided software. Less so, with the openSuSe.

I ended up installing everything in yast2 that even tangentically mentioned samba or netbios. I then opened up my firewall to the fullest extent possible, sacrificed two virgins, restarted netbios and samba a couple of times and lo and behold it worked. I then switched the firewall back on with samba and netbios selected as allowed services for the external zone and it seems happy enough with that. No real lesson learned here, just black magic. This is really just a note for other to let them know that it could in theory work.

## Saturday, February 14, 2009

### Mirroring a mediawiki to another server

There are lots of good guides on the interwebs on how to do this but I wanted to get a few notes down so I remember my preferred/installed method. These instructions are for Debian/Ubuntu and assume the target server doesn't have an existing mediawiki or mysql installation. All this needs to be done as root.

1. Install mediawiki on the target server, install the same version.

2. Delete /var/lib/mediawiki1.7 on the target server.

3. Dump the mysql permissions database on the source server:

mysqldump -u root -pYOURPASSWORD --opt mysql > ~/mysql.sql

4. Copy that file over to the target file. Import it using the following command.

mysql -pYOURPASSWORD -h 127.0.0.1 mysql < ~/mysql.sql

5. Setup a cron job on the source host to dump the mediawiki sql database every night. Type crontab -e to edit the crontab and add the following line:

0 0 * * * mysqldump -u root -pYOURPASSWORD --opt wikidb > ~/wikidb.sql

6. On the target host, add cron jobs to copy across the database dump, the mediawiki config/uploaded files and to import the mediawiki database. Again do crontab -e and add the following:

0 5 * * *  rsync -oglpvtr root@SOURCEHOST:/var/lib/mediawiki1.7 /var/lib > ~/backups/SOURCEHOST_mediawiki.log0 5 * * *  rsync -vtr root@SOURCEHOST:/root/wikidb.sql ~/wikidb.sql0 6 * * *  mysql -pYOURPASSWORD -h 127.0.0.1 wikidb < ~/wikidb.sql

Protip: you'll need your ssh keys setup so you can do passwordless authentication.

### The Viterbi algorithm

For your viewing pleasure I present an implementation of the Viterbi algorithm. I implemented this to help me understand the algorithm as presented in "Biological Sequence Analysis" by Durbin et al. The code solves the "occasionally dishonest casino, part 2". The example displays the output as copied from the book as well as it's own calculation so you can verify the results.

Code here

# This blog post has moved here

### PGF and tkz-graph installation in Ubuntu

Ubuntu likes to keep it's texlive packages at least a year out of date, so if you want to install tkz-graph you need to install it and pgf manually. Create a directory called texmf in your home directory (mkdir ~/texmf). Create a subdirectory called tex (mkdir ~/texmf/tex). Create subdirectories of this called generic and latex. From the pgf tarball copy pgf-2.00/generic into the generic subdirectory. Copy pgf-2.00/latex to the latex subdirectory (you now have ~/texmf/generic/pgf and ~/texmf/latex/pgf. Download the tkz-graph style (tkz-graph.sty) and copy it to ~/texmf/latex. I copied it to ~/texmf/latex/prof I don't think that matters... Next "sudo texhash". That should do it.

## Tuesday, February 3, 2009

Say you want to download a bunch of pdf files from a webpage. You want to do this remotely over ssh so those cool firefox extensions are our of the question. Use wget. For example:

tsocks wget -r -l1 -t1 -N -A.pdf -erobots=off http://www.springerlink.com/content/x49g43t85772/ 

Does the job.

### Openmoko freerunner with vodafone DE sim card

Just a note to mention for anyone curious that I managed to get a 3G german vodafone sim working with OM 2008.12 by flashing the gsm firmware to moko10. Whilst the wiki advises using shr I found it didn't make any difference. Still have to test this with 3 cards from the UK but hopefully some of them will work now too.

### Silc client won~t accept the / (backslash) key

I use silc as a replacement for things like irc. Most of the time I think its great. It provides secure and simple encryption and allows me to waste time instead of actually working. Well that last bit is a tad unfair as I learn a lot of things on there.

Anyways, for the last while, it has been impossible for me to enter the backslash character in a session. This is regardless of the keyboard layout on the client computer, the OS and was a seemingly random error. Today I erased the .silc/silc.conf file and restarted the silc client. Success - I am now able to connect to the server once again. The most infuriating aspect of this is that the error occured during a running session. When I copied urls into that session none of the slash characters would be displayed on the screen. Which was funny at first but very annoying once I restarted silc only to discover that I couldn't now type /connect. Which was funny for about 30 seconds. So there you go. If you are ever in the situation where you can't enter the slash key in a silc client just remove the configuration file.

### Replacing a string in vim to fix an dxf file

I have an DXF file that has entries that look like this:

LCD.17064621506CONTINUOUS0LAYER2LCDMOUNT.17064622526

Autocad doesn't like this file... I have no idea what created it. After much screwing around I've discovered that it doesn't like the "."s in the entity (or whatever they're called) names. Load the file in vim and use the following regexp to get rid of the dots.

%s/$$[A-Z]$$\.$$[0-9]\+$$/\1\2/g

Fixed file looks like this:

LCD17064621506CONTINUOUS0LAYER2LCDMOUNT17064622526

FYI, the  are delimiters in the search sequence. \1 and \2 reference back to these.

Addition: The file in question was produced by a program called "Camera 3D" or "3D Camera" or something... Just in case anyone else has the same problem with this code.

## Monday, January 26, 2009

### Merging in Subversion

Merging in subversion doesn't work like you'd think it'd work. You don't say "take this branch and apply all the changes to trunk". No... you say "calculate the differences between this branch and and older revision then apply these changes to trunk". That's very different. Because of this you need to note the revision at which you're branch was made and use this when merging. Here's a quick example:

Make a branch:
svn copy trunk ./branches/nw3
NOTE THE REVISION NUMBER (basenum) AND LEAVE A USEFUL LOG COMMENT. OK, now make edits commit them etc on the branch. At some point you want to merge the changes in to trunk (run this while in the trunk directory):
svn merge -r basenum:currentrev https://swiftng.svn.sourceforge.net/svnroot/swiftng/trunk
Where basenum is the revision number at which the branch was made and currentrev is the current revision.

# This blog post has moved HERE

## Tuesday, January 13, 2009

### Disk space usage

My VM keeps getting filled up, and then crashing cos the programs that write logs are not being correctly handled by the underlying FS. So I am interested in generating useful infos on the disk to find out where all the disk space is at. The command du is a big help here. But I frequently just want to know in which directories the most space is being used.

du -h --max-depth=1

That command will give you a listing of which directories under the current dir are using all the space. So in the home dir or at the usr dir run this and find out where all the unneccessary crap is.

Giving me in the /usr dir the following output:
1.1G    ./share40K     ./games1.4G    ./local7.1M    ./sbin125M    ./src36K     ./X11R6136M    ./bin14M     ./include1.6G    ./lib4.3G    .`

And this tells me that I ought to tidy up the /usr/local dir. If you ran the above command without the max-depth option then the resulting output will probably overrun your cache/history in your bash shell and besides the output would be impossible to make sense of.