Skype is awesome. We use it a ton here at Automattic, and it’s my main communication service for staying in touch with friends and family who are spread out all over the world. It’s also still the best text IM client out there. And now with the iPhone app, I can make int’l calls on mobile without thinking about it.

The one issue though I’ve had lately, is that I rely on Skype so much, that when I use a second machine I find myself looking through Skype chat history for a link or reference, only to realize that it’s stored locally on another machine.

In my quest to find an ideal backup solution, I’ve been testing Dropbox which allows you to sync files between multiple machines. So I took a crack at trying to get my Skype chat history to sync. A quick google search brought up this post which I followed:

Step 1) Quit the Skype application on all machines
Step 2) Move the "main" Skype chat history files to the Dropbox directory:
"mv ~/Library/Application\ Support/Skype ~/Dropbox/"
Step 3) Create a symlink from the original folder to the Dropbox folder:
"ln -s ~/Dropbox/Skype/ ~/Library/Application\ Support/Skype"
Step 4) On your secondary machines, remove the /Library/Application\ Support/Skype folder, and just insert the symlink:
"ln -s ~/Dropbox/Skype/ ~/Library/Application\ Support/Skype"

And it works ! One issue to be aware of: If you are logged-in and running Skype on multiple machines at once, you can get file conflicts and Dropbox will create secondary files – which kills the whole sync idea. So for this to work, you need to quit Skype before you plan on using it on another machine — which isn’t a bad idea anyway since leaving it open just means missed messages.

And lastly, I posted some of this to the Dropbox forum and one member there recommended a slightly different method where: “The original stays where it is. Without moving it, one creates a symlink. Its symlink goes into the Drobox folder.” I tried that method early on, and at least with Skype, the chat history kept getting written locally and never updating the Dropbox folder.

So for now this is working nicely for me. I do wonder if in the future Skype would ever offer up a secure way to store chat history with them. I also did see mention of a new service that tackles this issue for IM history in general called im-hisotry.com — looks interesting but haven’t tested so far as it’s Windows and Linux only.

For anyone thinking of trying the symlink method, as with anything like this, please backup your files before attempting this, as YMMV.

Um mesmo aplicativo pode se comportar de forma diferente dependendo da forma como é invocado. No exemplo a seguir, o mesmo programa é usado para calcular a raiz quadrada e potência de dois de um dado argumento.

#include <libgen.h>
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char **argv)
{
    double res;
    char *program = basename(argv[0]);

    if (argc != 2)
        return 1;

    if (!strcmp(program, "sqrt")) {
        res = sqrt(atof(argv[1]));
        printf("%f\n", res);
        return 0;
    }

    if (!strcmp(program, "pow2")) {
        res = pow(atof(argv[1]), 2);
        printf("%f\n", res);
        return 0;
    }

    printf("Invalid program: %s\n", program);

    return 1;
}

O truque consiste em analisar o primeiro argumento que corresponde ao caminho do binário do programa. No exemplo, decidimos qual ação tomar comparando o nome do programa com “sqrt” e “pow2″. Para não fazer múltiplas cópias do binário gerado, fazemos links simbólicos.

$ gcc -Wall -lm -o test test.c
$ ln -s test sqrt
$ ln -s test pow2

$ ./test 5
Invalid program: test

$ ./pow2 5
25.000000

$ ./sqrt 5
2.236068

A técnica é extensamente utilizada no BusyBox e pode facilmente ser implementada em qualquer outra linguagem de programação.

I store all my music on an external hard disk which is connected to Windows Server 2003 in a cupboard hidden away. This means my music doesn’t clutter up the hard disk on my MBP and it can be shared to other devices like my Xbox, without having my laptop on. Accessing the server from Windows XP or Vista was fairly straightforward – I’d just map a drive and iTunes was happy. Unfortunately, persistent mapped drives on Mac OS X 10.5.6 (or earlier) isn’t really a feature, so I’d just plug the external hard disk into my Mac via USB.

I needed a way to get my Windows iTunes database onto my Mac, and a way of mounting a persistent network share for iTunes, and a convenient way to add new music to the networked folder and iTunes. (Oh, and I kinda like things being in the right place – you know, documents in the Documents folder, music in the Music folder – so we’ll be creating a symbolic link to make our music appear as though it’s in ~/Music/iTunes/iTunes Music. Clever, eh?) So, without further ado, here’s what we’ll be doing -

1. Setting up a shared folder for your music
2. Create a symbolic link (& mounted drive) to the network share
3. Creating a crontab entry to keep the network share mounted (alive)
4. Modifying the location of your music files in the iTunes database

1. Setting up a shared folder for your music

If you’ve got a NAS you should go into its admin area and create a file share, call it Music or something useful. Depending on what NAS you have, you’ll be able to specify a username and password for the shared folder, or for the entire NAS. Either way, note the username and password to access the share – we’ll need them later.

If you’re using an hard disk (internal or external) on a server, go into Computer Manager | Local Users and Groups – and create a new user. For security reasons, this user will only have access to your music. I created the user “music”, but you can use whatever. Note down the username and password. Still in the Computer Management window, go to Shared Folders | Shares in the tree, then Action menu New Share. Create a new shared folder for your music folder, and add the user you just created to the permissions – give them Full Access (read, write, modify).

It’s probably a good idea to test accessing the share from your Mac. In Finder, go to the Go menu, and click Connect To Server. In the server address box, type smb://192.168.0.2/Music (change the IP address to your NAS or server, and the folder from Music to whatever share name you used). Click Connect, and you should be prompted for a username and password – use the details of the user you just created.

2. Create a symbolic link (& mounted drive) to the network share

Quit iTunes if it’s running.

Here’s the command we’ll be using to create the symbolic link – this will place a drive icon on the desktop for the mounted network share, and when you go to your ~/Music/iTunes/iTunes Music folder, it’ll actually be the networked folder.

/sbin/mount -o nodev,nosuid -t smbfs '//username:password@192.168.0.2/Music' '/Users/YourName/Music/iTunes/iTunes Music'

And a explanation -

• /sbin/mount is the utility which mounts drives and shares
• -o nodev,nosuid parameters are necessary to create a symbolic link (symlink)
• -t smbfs specifies we’re connecting to a Samba (SMB) network share
• username is the name of the user you created on your NAS or server
• password is the password you specified for that user
• 192.168.0.2 is the IP address of your NAS or server
• /Music is the shared folder name
• /Users/YourName/Music/iTunes/iTunes Music is the folder we’re going to make as a symbolic link. You should change YourName to your Mac username

We need this command to work properly before we move on, so it would be a good idea to open a Terminal window and try it out. If you already have a /Users/YourName/Music/iTunes/iTunes Music folder, you should move the contents out of it for now and delete the folder. Alternatively, if you want to use a different folder, go ahead – just make sure it’s empty before you execute the command.

Ok, so if you’ve got a Music disk mounted on the desktop, proceed to the next step!

It’s a bit wasteful to blindly execute this command every 10 minutes, so we’ll put it in a script with a couple of checks. So, open your text editor and copy and paste this code snippet (you’ll need to change the parts in bold):

if [ -d '/Users/YourName/Music/iTunes/iTunes Music/ChangeThis' ]; then
    exit
else
    /sbin/ping -c 1 -t 5 192.168.0.2 &>/dev/null
    if [ $? -eq 0 ]
    then
        /sbin/mount -o nodev,nosuid -t smbfs '//username:password@192.168.0.2/Music' '/Users/YourName/Music/iTunes/iTunes Music'
    fi
fi

What does this do? Well, firstly it checks to see if the drive is already mounted by looking for the folder /Users/YourName/Music/iTunes/iTunes Music/ChangeThis – and if so, exists. Note: ChangeThis needs to be a folder on your mounted drive.  If you change the script to check for the existance of /Users/YourName/Music/iTunes/iTunes Music/, the system will always find it even when disconnected.

Next it’ll ping the IP address of your server and if it gets a response, it’ll try to mount the share. If it doesn’t get a reply – probably because you’re on a different network or not connected – it’ll exit. This prevents needlessly sending login attempts across your network.

Ok, save your script in ~/Library/Scripts – call it MountMusic.command or something you can remember.

3. Creating a crontab entry to keep the network share mounted

In a nutshell, a crontab is a set of commands which can be run at set intervals – perfect for refreshing our mounted network share, and especially handy for mounting the drive shortly after you’ve logged in or woken your Mac from sleep. Editing the crontab isn’t difficult but it can be fiddly. I use a great little utility called CronniX which I’ll be using for this tutorial.

Now the thing is with cron is that when it does something, it likes to send you an email about it. And left unchecked, it can create a whopping great big mail file. And we don’t want that, do we! In CronniX go the Edit menu, and click Env. Variables. – add a new environment variable “MAILTO” (sans quotes). For the MAILTO value, type double quotes (“”).

I run the mount command (above) every 10 minutes. If the network share has already been mounted, that’s fine, otherwise it’ll mount it again. If there’s no network share to mount – maybe you’re at work or school – it’ll fail silently. Using CronniX, create the following entries -

Min  Hour  Mday  Month  Wday  Command
0    *     *     *      *     ~/Library/Scripts/MountMusic.command
10   *     *     *      * ~/Library/Scripts/MountMusic.command
20   *     *     *      * ~/Library/Scripts/MountMusic.command
30   *     *     *      * ~/Library/Scripts/MountMusic.command
40   *     *     *      * ~/Library/Scripts/MountMusic.command
50   *     *     *      * ~/Library/Scripts/MountMusic.command

So you’ve created 6 entries, set to run at 0, 10, 20, 30, 40 and 50 minutes past the hour, and it’ll be running your mount command. Once you’ve done that, click Save in the main CronniX window.

Chances are you still have the Music disk mounted on your desktop from before – right click on it, and Eject it (or drag it onto the recycle bin). If you wait until your clock shows 0, 10, 20, 30, 40 or 50 minutes (i.e. 10:40), a few seconds later the Music disk should reappear on your desktop – if it does, move on…

4. Modifying the location of your music files in the iTunes database

I have a fairly substantial music collection, all nicely tagged and with genres, ratings and play counts – which I don’t want to lose. If you don’t care about this extra info, you can pretty much skip most of this step.

I’ve searched high and low for an AppleScript or app to go through my iTunes database and mass change the location of my music files, but let me tell you – there is NOTHING that does it without serious effort. There are a few alternative techniques, but they don’t preserve your folder structure or meta data. So, we’ll be using the “corrupt iTunes database” method…

Go to your iTunes folder (~/Music/iTunes/) and in there you should find two files – iTunes Library (or iTunes Library.itl from the PC) and iTunes Music Library.xml. Rename the iTunes Library file to something else and create a file with the same name in TextEdit. Next, open iTunes Music Library.xml in TextEdit (or whatever text editor your prefer) and not too far down,  you’ll see something like this -

file://localhost/Volumes/MyDisk/Music/Albums/Band/Track.mp3

You need to search and replace the /Volumes/MyDisk part so you end up with something like -

file://localhost/Users/YourName/Music/Albums/Band/Track.mp3

Save the file and start iTunes. You should see iTunes recreating your library from the XML file – this may take some time. Fingers crossed, iTunes will eventually open and your music library should be intact!

Done

I ran into an interesting snag today: one of my favourite games (Widelands) is installed on my second hard drive but has a save game folder in my home folder.
I didn’t like this (my home folder’s big enough as it is) so I tried to create a folder in Widelands’ folder and then make a (sym)link to it from my home folder.

Operation not Permitted.

No explanation why (that needs to be changed), but a quick Google search found the problem:

My Widelands install (and therefore the temp directory I was going to use for the saved games) is on a FAT formatted drive.

FAT (in whatever incarnation you have – NTFS, FAT32, whatever) doesn’t do (sym)links.

So… short of making the partition the game was on a different format (and wiping it in the meantime)… I moved my Widelands install to my ext3 partition.

Linked and off we go! Easy, once you know the problem.

helping to destroy ratio P2P peering one step at a time

How do I create multiple large files that consume no extra diskspace?

Create multiple Hardlinks to one or more files.  Change the names of the Hardlinks to suit.

.

[ Himself ]

Oggi voglio parlarvi di uno strumento che ci permette di rendere il file system e la gestione di file e directory molto più flessibile,sto parlando dei Collegamenti.

Continua sul Nuovo Blog…

Schon zum zweiten Mal habe ich mir heute den Kopf darüber zerbrochen was ich falsch mache beim Symlink anlegen. In der Theorie geht es so:

ln -s original_datei symlink_quelle

Doch ich bekomme ständig leere Symlinks und damit Fehler wie:

chmod: kann nicht auf der freihängenden symbolischen Verknüpfung “Testdatei” arbeiten

Die Lösung:

Die Pfade müssen absolut nicht relativ angegeben werden (besonders wenn man das bei der Original Datei nicht tut gibt das Probleme), das kann man dann zum Beispiel mit ‘pwd‘ erreichen (pwd gibt den aktuellen  Pfad zurück).

ln -s ‘pwd’/OriginalDatei /home/user/files/SymLink

Die Anführungsstriche müssen übrigens Backticks sein.

Using SSH is somewhat not easy for a newbie. To copy files over SSH, you need long commands. So, why not just mount the remote filesystem on your local computer!

Fuse is a part of the latest kernel, so you don’t need to install fuse-utils (if you’re not sure if you have fuse, run lsmod | grep fuse.) So – install sshfs:

sudo aptitude install sshfs

You need permission to use the fuse function – so add yourself to the groups:

sudo usermod -a -G fuse username

Now – create a directory where you want the remote filesystem (make sure you are the owner of the directory). I choose /media/directory

cd /media
sudo mkdir ext_sys
sudo chown myusername:myusergroup ext_sys

Now – you mount the system (I use /media/ext_sys as mount point)

sshfs remote_username@remote_ip: /media/ext_sys

To unmount, run

fusermount -u /media/ext_sys

Got to love Linux! I had a problem with my usb-stick – but ssh popped into my head – and voila! You learn something everyday

Updated:

You can allow the option to follow symlinks by adding ‘ -o follow-symlinks’

sshfs -o follow-symlinks user@server: /mnt/point

This post applies to running Steel Bank Common Lisp version 1.0.16 – 1.0.17.14 on Windows Vista. It may apply to later or earlier versions of SBCL, but I cannot directly attest to that. It may apply to certain functionality under Windows XP, but I cannot attest to that either.

[Update 07 June 2008]  I have made a few modifications to the attached code in order to:
- place (D. Lichteblau’s) support for SHORTCUT files in a separate function PARSE-WINDOWS-SHORTCUT (to ensure stream-read-invariance wrt TRUENAME);
- add a redefinition for FILE-WRITE-DATE to mimic Unix behavior under Windows;
- add behavior so that TRUENAME of a recursive or dangling link returns the pathname of the link, and does not signal an error;
- add support for symbolic links containing relative pathnames;
- add support for symbolic links to different disks/volumes.

With reference to:

http://robert.zubek.net/blog/2008/04/09/sbcl-emacs-windows-vista/
http://sbcl-internals.cliki.net/Build%20on%20Windows
http://www.sbcl.org/
http://brainrack.wordpress.com/2008/05/29/running-sbcl-on-windows/
http://brainrack.wordpress.com/2008/05/28/broken-and-ill-documented-api-for-windows-mount-points/

Broken #’truename and annoying ASDF under Windows using SBCL

The CLtL2 specification requires #’truename to return the “canonical” pathname of a file.  Most lisps interpret this to mean that when a pathname refers to a link, the link should be followed in order to return the pathname under which the file actually resides.  Indeed, this is the way the UNIX implementation of SBCL functions with regard to #’truename.  However, the Windows implementation of SBCL does not function the same as the UNIX implementation: (truename pathspec) will not follow links under Windows.

Complaining about this would be a petty bit of specs-manship were it not for the fact that the standard system definition facility ASDF relies on following links in order to function as designed.  I find the intended behavior of ASDF to be elegant and convenient and so I set out to restore the ANSI functionality of #’truename under SBCL Windows.

[This issue has a history: see Stephan Lang's http://marc.info/?l=sbcl-devel&m=117357216708358&w=2; David Lichteblau's http://www.lichteblau.com/blubba/shortcut/asdf.lisp; and a discussion initiated by Nikodemus Siivola at http://article.gmane.org/gmane.lisp.cclan.general/740.]

Fixing the problem required me to delve deeply into some very dark recesses of the Microsoft Windows API, as well as SBCL’s FFI.  Along the way I encountered all the usual abominations one must deal with when trying to program in Microsoftland: incorrect documentation, broken functions, vague and underspecified behaviors, inconsistent functionality between related functions, bizarre designs, Hungarian variable notation, and generally rococo APIs seemingly still implemented in a pre-1980 version of ‘C’ and modified from a cribbed copy of CPM.  Some details are here.  I feel most highly-privileged to have encountered the rare and wonderful ‘dammit-the-documentation-changed-while-I-was-reading-it!’ experience at http://msdn.microsoft.com/en-us/library/ms791514.aspx.

My plaints should not be taken to slight the abominations in SBCL’s FFI, however, for they are truly ghastly and profound.  (Try to figure out exactly how c-strings and (* char)’s are handled sometime!)

Nonetheless, I have a working version of

(LINKED-TRUENAME filespec) — if the last part of the path of /filespec/ points to a link, return a pathname in which the link is resolved, else return the usual truename.

I have tested this function to verify that it works on file symlinks, directory symlinks, and mount points (aka junctions) as produced by mklink, mklink /D, mklink /J, mountvol, and the Disk Manager.

I don’t know if the fix will ever make it into the SBCL distribution.  I’m not sure I want to jump through all the hoops necessary.  In the meantime, it works for me, and hopefully for you.

Here’s the source:

;;;
;;;
;;; Code for following symlinks under Windows Vista
;;; Redefined truename so truename returns pathname of linked file
;;; This functionality is necessary for running asdf as it is designed
;;; - Neil Haven, June 2008
;;;
;;; For Microsoft reparse-points handling see:
;;; http://blog.kalmbach-software.de/2008/02/28/howto-correctly-read-reparse-data-in-vista/
;;;
;;; For Windows .lnk file handling see:
;;; "Jesse Hager: The Windows Shortcut File Format."
;;; http://www.wotsit.org/list.asp?fc=13
;;; Some code excerpted from David Lichteblau's modifications to asdf.lisp
;;; http://www.lichteblau.com/blubba/shortcut/asdf.lisp
;;;

(defpackage #:winlinks
    (:nicknames #:wl)
  (:documentation "Package for redefining truename so it follows links.")
  (:use #:cl #:sb-alien #:sb-ext)
  (:shadow #:truename #:file-write-date)
  (:export #:override-truename #:override-file-write-date
           #:parse-windows-shortcut))

(in-package #:wl)

#|
To create a mount point under Vista use mklink /J ...
To create a symbolic file link use mklink ...
To create a directory link use mklink ...
|#

#|
typedef struct _REPARSE_DATA_BUFFER {
  ULONG  ReparseTag;
  USHORT  ReparseDataLength;
  USHORT  Reserved;
  union {
    struct {
      USHORT  SubstituteNameOffset;
      USHORT  SubstituteNameLength;
      USHORT  PrintNameOffset;
      USHORT  PrintNameLength;
      ULONG   Flags; // it seems that the docu is missing this entry (at least 2008-03-07)
      WCHAR  PathBuffer[1];
      } SymbolicLinkReparseBuffer;
    struct {
      USHORT  SubstituteNameOffset;
      USHORT  SubstituteNameLength;
      USHORT  PrintNameOffset;
      USHORT  PrintNameLength;
      WCHAR  PathBuffer[1];
      } MountPointReparseBuffer;
    struct {
      UCHAR  DataBuffer[1];
    } GenericReparseBuffer;
  };
} REPARSE_DATA_BUFFER, *PREPARSE_DATA_BUFFER;

#define MAXIMUM_REPARSE_DATA_BUFFER_SIZE  ( 16 * 1024 )

//
// The reparse tags are a DWORD. The 32 bits are laid out as follows:
//
//   3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1
//   1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0
//  +-+-+-+-+-----------------------+-------------------------------+
//  |M|R|N|R|     Reserved bits     |       Reparse Tag Value       |
//  +-+-+-+-+-----------------------+-------------------------------+
//
// M is the Microsoft bit. When set to 1, it denotes a tag owned by Microsoft.
//   All ISVs must use a tag with a 0 in this position.
//   Note: If a Microsoft tag is used by non-Microsoft software, the
//   behavior is not defined.
//
// R is reserved.  Must be zero for non-Microsoft tags.
//
// N is name surrogate. When set to 1, the file represents another named
//   entity in the system.
//

|#

(defun override-truename (&optional (package *package*))
  "replace definition of truename in /package/ "
  (shadowing-import 'wl::truename package))

(defun override-file-write-date (&optional (package *package*))
  "replace definition of file-write-date in /package/ "
  (shadowing-import 'wl::file-write-date package))

(defun wchar-string->lisp-string (wstring &optional (length -1))
  "
  Arguments: /wstring/ is a pointer to a zero-terminated
    array of wchars (unsigned shorts)
    /length/, if supplied, truncates the translation after
    /length/ number of characters if a zero-termination is not
    encountered before that
  Semantics: translates /wstring/ to a lisp string.  If
    /wstring/ is a null pointer, returns nil
  Returns: a lisp string or nil.
  "
  (declare (type (alien (* unsigned-short)) wstring)
           (type integer length))
  (when (null-alien wstring)
    (return-from wchar-string->lisp-string nil))
  ;; calculate the length
  (when (minusp length)
    (do ((index 0 (incf index)))
        ((= (deref wstring index) 0) (setq length index))))
  (do* ((index 0 (incf index))
        (wchar (deref wstring index) (deref wstring index))
        (lisp-string (make-array length :element-type 'character)))
       ((= index length) lisp-string)
    (declare (type integer index)
             (type (unsigned-byte 16) wchar)
             (type string lisp-string))
    (setf (aref lisp-string index) (code-char wchar))))

(defun lisp-string->wchar-string (string)
  "
  Semantics: converts the lisp /string/ into an alien pointer to
    a zero-terminated array of wide (16-bit) chars.  Any
    characters in /string/ with a unicode code point
    > #xFFFF are given a representation as #xFFFD -- the
    standard unicode Replacement Character
  Returns: Two values: an alien pointer to a zero-terminated array of
    16-bit chars suitable for passing to a Windows API;
    and the length of the string
  "
  (let* ((size (length string))
         (wchar-string (make-alien (unsigned 16) (1+ size)))
         (unicode 0)
         (wchar 0))
    (declare (type integer size unicode)
             (type (unsigned-byte 16) wchar))
    (dotimes (index size)
      (setq unicode (char-code (elt string index)))
      (when (> unicode #xFFFF) (setq unicode #xFFFD)) ; coerce to 16-bits
      (setq wchar (coerce unicode '(unsigned-byte 16)))
      (setf (deref wchar-string index) wchar))
    (setf (deref wchar-string size) 0)
    (return-from lisp-string->wchar-string
      (values wchar-string size))))

(defun lisp-string->char-string (string)
  "
  Semantics: converts the lisp /string/ into an alien pointer to
    a zero-terminated array of 8-bit chars.  Any
    characters in /string/ with a unicode code point
    > #xFF are given a representation as #x1A -- the
    standard unicode Substitute control
  Returns: Two values: an alien pointer to a zero-terminated array of
    8-bit chars suitable for passing to a Windows API;
    and the length of the string
  "
  (let* ((size (length string))
         (char-string (make-alien (unsigned  (1+ size)))
         (unicode 0)
         (char 0))
    (declare (type integer size unicode)
             (type (unsigned-byte  char))
    (dotimes (index size)
      (setq unicode (char-code (elt string index)))
      (when (> unicode #xFF) (setq unicode #x1A)) ; coerce to 8-bits
      (setq char (coerce unicode '(unsigned-byte 8)))
      (setf (deref char-string index) char))
    (setf (deref char-string size) 0)
    (return-from lisp-string->char-string
      (values char-string size))))

(define-alien-type nil
    (struct SymbolicLinkStructure
            (substitute-name-offset unsigned-short) ; 2 bytes
            (substitute-name-length unsigned-short) ; 2 bytes
            (print-name-offset unsigned-short) ; 2 bytes
            (print-name-length unsigned-short) ; 2 bytes
            (flags unsigned-long)       ; 4 bytes
            (path-buffer (array unsigned-short 246)))) ; 492 bytes

(define-alien-type nil
    (struct MountPointStructure
            (substitute-name-offset unsigned-short) ; 2 bytes
            (substitute-name-length unsigned-short) ; 2 bytes
            (print-name-offset unsigned-short) ; 2 bytes
            (print-name-length unsigned-short) ; 2 bytes
            (path-buffer (array unsigned-short 248)))) ; 496 bytes

(define-alien-type nil
    (struct GenericReparseStructure
            (data-buffer (array char 504)))) ; 504 bytes

(define-alien-type nil
    (union ReparseUnion
           (symbolic-link-structure (struct SymbolicLinkStructure))
           (mount-point-structure (struct MountPointStructure))
           (generic-reparse-structure (struct GenericReparseStructure))))

(define-alien-type nil
    (struct ReparseStructure
            (reparse-tag unsigned-long) ; 4 bytes
            (length unsigned-short)     ; 2 bytes
            (reserved unsigned-short)   ; 2 bytes
            (reparse-union (union ReparseUnion)))) ; 504 bytes

(defmacro reparse-tag-p (reparse-tag)
  "Is /reparse-tag/ a legal Microsoft reparse tag?"
  ;(declare (type (unsigned-byte 32) reparse-tag))
  `(= #x80000000 (logand ,reparse-tag #x80000000)))

(defmacro name-surrogate-p (reparse-tag)
  "Is the /reparse-tag/ from a surrogate for another file?"
  ;(declare (type (unsigned-byte 32) reparse-tag))
  `(= #x20000000 (logand ,reparse-tag #x20000000)))

(defmacro mount-point-p (reparse-tag)
  "Is the /reparse-tag/ a Microsoft mount point?"
  ;(declare (type (unsigned-byte 32)) reparse-tag)
  `(= #xA0000003 (logand ,reparse-tag #xA0000003)))

(defmacro symlink-p (reparse-tag)
  "Is the /reparse-tag/ a Microsoft symlink?"
  ;(declare (type (unsigned-byte 32)) reparse-tag)
  `(= #xA000000C (logand ,reparse-tag #xA000000C)))

#|
HANDLE WINAPI CreateFile(
  __in      LPCTSTR lpFileName,
  __in      DWORD dwDesiredAccess,
  __in      DWORD dwShareMode,
  __in_opt  LPSECURITY_ATTRIBUTES lpSecurityAttributes,
  __in      DWORD dwCreationDisposition,
  __in      DWORD dwFlagsAndAttributes,
  __in_opt  HANDLE hTemplateFile
);

BOOL WINAPI DeviceIoControl(
  __in         HANDLE hDevice,
  __in         DWORD dwIoControlCode,
  __in_opt     LPVOID lpInBuffer,
  __in         DWORD nInBufferSize,
  __out_opt    LPVOID lpOutBuffer,
  __in         DWORD nOutBufferSize,
  __out_opt    LPDWORD lpBytesReturned,
  __inout_opt  LPOVERLAPPED lpOverlapped
);
|#

(load-shared-object "Kernel32")

(declaim (inline CreateFileA))

(declaim (inline CloseHandle))

(declaim (inline DeviceIoControl))

(declaim (inline GetVolumeNameForVolumeMountPointA))

(define-alien-routine ("GetVolumeNameForVolumeMountPointA" GetVolumeNameForVolumeMountPointA) int
  (mount-point (* unsigned-char))
  (volume-name (* unsigned-char))
  (buffer-length unsigned-int))

(define-alien-routine ("CreateFileA" CreateFileA) int
  (filename (* unsigned-char))
  (access unsigned-int)
  (sharemode unsigned-int)
  (security unsigned-int)
  (create unsigned-int)
  (flags unsigned-int)
  (template unsigned-int))

(define-alien-routine ("CloseHandle" CloseHandle) int
  (device int))

(define-alien-routine ("DeviceIoControl" DeviceIoControl) int
  (device unsigned-int)
  (control unsigned-int)
  (inBuffer (* t))
  (inSize unsigned-int)
  (outBuffer (* (struct ReparseStructure)))
  (outSize unsigned-int)
  (bytesReturned (* unsigned-int))
  (overlapped (* t)))

(defun get-link-handle (filespec)
  "returns -1 on failure, otherwise returns handle for /filespec/"
  (let ((filename (lisp-string->char-string (namestring filespec))))
    ;; for the interpretation of the magic numbers, see the kalmbach reference
    (CreateFileA filename #x8 #x7 #x0 #x3 #x02200000 #x0)))

(defun get-volume-name (mount-point)
  "
  Semantics: returns lisp string for underlying volume name of mount point
    If there is no underlying volume name, return /mount-point/
  Arguments: /mount-point/ is a string pathname designator
  Returns: lisp string in form \\\\?\\Volume{GUID}\\ if /mount-point/ exists
    else returns original /mount-point/
  "
  ;; GetVolumePathNamesForVolumeName, does not seem to work for me.
  ;; So this is a tedious bit of kludgy sequential search.
  (let ((volume-name (make-alien unsigned-char 64)) ; Microsoft doco says the max length is 50
        (result 0))
    (declare (type integer result))
    ;; read the low-level volume name given a normal path
    (setq result
          (GetVolumeNameForVolumeMountPointA (lisp-string->char-string mount-point) volume-name 64))
    (when (zerop result)
      ;; if the call fails, the path isn't a mount point
      (return-from get-volume-name mount-point))
    ;; now create a lisp string from the character array
    (let ((volume (make-string 64)))
      (declare (type string volume))
      (dotimes (i 64)
        (let ((c (deref volume-name i)))
          (when (zerop c)
            ;; hit end of array, so truncate the string
            (setq volume (subseq volume 0 i))
            (return))
          (setf (char volume i) (code-char c))))
      volume)))

(defun volume-name->disk-name (volume-name)
  "
  Semantics: returns disk-name for low-level volume name
  Arguments: /volume-name/ is a low-level volume name
    of the form \\??\\Volume{GUID}\\
  Returns: lisp string in form X:\\ where X is the disk name.
    or /volume-name/ if there is no associated disk
  "
  (dotimes (disk 26 volume-name)
    (let* ((disk-name (concatenate 'string (string (code-char (+ disk (char-code #\A)))) ":\\"))
           (disk-volume-name (get-volume-name disk-name)))
      (when (and (>= (length volume-name) (length disk-volume-name) 4)
                 (equalp (subseq disk-volume-name 4) (subseq volume-name 4)))
        (return-from volume-name->disk-name disk-name)))))

(defun mount-point->disk-name (pathspec)
  "
  Semantics: Converts the mount point /pathspec/ into the name of the disk
    it points to.
  Arguments: /pathspec/ is the pathspec of a mount point
  Returns: the namestring of the disk pointed to by /pathspec/
  Error Conditions: signals
  "
  (let ((volume-name (get-volume-name (namestring (cl::truename pathspec)))))
    (volume-name->disk-name volume-name)))

(defun extract-path-from-mount-point (reparseStructure)
  "internal function for getting a path from a mount point"
  (declare (type (alien (struct ReparseStructure)) reparseStructure))
  ;; get the union fron the reparse structure
  (with-alien
      ((reparseUnion (union ReparseUnion) (slot reparseStructure 'reparse-union)))
    ;; get the mount-point from the union
    (with-alien
        ((mount-point (struct MountPointStructure) (slot reparseUnion 'mount-point-structure)))
      ;; get the string buffer from the mount-point
      (with-alien ((buffer (array unsigned-short 248) (slot mount-point 'path-buffer))
                   (sname-offset unsigned-short (slot mount-point 'substitute-name-offset))
                   (sname-length unsigned-short (slot mount-point 'substitute-name-length)))
        ;; adjust for size of wchar
        (setq sname-offset (/ sname-offset 2)) ; offset is returned in bytes
        (setq sname-length (/ sname-length 2)) ; length is returned in bytes
        ;; get the lisp equivalent
        (let ((sname-string
               (wchar-string->lisp-string (addr (deref buffer sname-offset)) sname-length)))
          ;; sname-string has form \??\Volume{GUID}\
          ;; OR sname-string has form \??\X:\ -- it all depends...
          (if (> (length sname-string) (length "\\??\\X:\\"))
              ;; we have to do this the hard way
              (volume-name->disk-name sname-string)
              ;; otherwise don't forget to lop off the \??\ part...
              (subseq sname-string 4)))))))

(defun extract-path-from-symlink (reparseStructure)
  "internal function for getting a path from a symlink"
  ;; get the union from the structure
  (with-alien
      ((reparseUnion (union ReparseUnion) (slot reparseStructure 'reparse-union)))
    ;; get the symlink from the union
    (with-alien
        ((symlink (struct SymbolicLinkStructure) (slot reparseUnion 'symbolic-link-structure)))
      ;; get the string buffer from the symlink
      (with-alien ((buffer (array unsigned-short 246) (slot symlink 'path-buffer)))
        (let ((sname-offset (slot symlink 'substitute-name-offset))
              (pname-offset (slot symlink 'print-name-offset))
              (sname-length (slot symlink 'substitute-name-length))
              (pname-length (slot symlink 'print-name-length)))
          (declare (type integer sname-offset pname-offset sname-length pname-length))
          ;; adjust for size of wchar
          (setq sname-offset (/ sname-offset 2)) ; offset is returned in bytes
          (setq pname-offset (/ pname-offset 2))
          (setq sname-length (/ sname-length 2)) ; length is returned in bytes
          (setq pname-length (/ pname-length 2))
          ;; convert the alien pointer to a lisp string
          (with-alien
              ((sname (* unsigned-short) (addr (deref buffer sname-offset))))
              ;; (pname (* short) (addr (deref buffer pname-offset))))
            (let ((slink (wchar-string->lisp-string sname sname-length))
                  (indicator-length (length "\\??\\")))
              ;; if linking another device, we need to strip a \\??\\ from the front...
              (if (and (>= (length slink) indicator-length)
                       (equal "\\??\\" (subseq slink 0 indicator-length)))
                  (subseq slink indicator-length)
                  slink))))))))

;;;; Windows shortcut support.  Based on:

;;; Jesse Hager: The Windows Shortcut File Format.
;;; http://www.wotsit.org/list.asp?fc=13
;;; Taken from David Lichteblau's modifications to asdf.lisp
;;; http://www.lichteblau.com/blubba/shortcut/asdf.lisp

;;; For the meaning of the magic numbers, see the Hager reference
(defparameter *link-initial-dword* 76)
(defparameter *link-guid* #(1 20 2 0 0 0 0 0 192 0 0 0 0 0 0 70))

(defun read-null-terminated-string (s)
  (with-output-to-string (out)
    (loop
        for code = (read-byte s)
        until (zerop code)
        do (write-char (code-char code) out))))

(defun read-little-endian (s &optional (bytes 4))
  (let ((result 0))
    (loop
        for i from 0 below bytes
        do
        (setf result (logior result (ash (read-byte s) (* 8 i)))))
    result))

(defun parse-file-location-info (s)
  (let ((start (file-position s))
        (total-length (read-little-endian s))
        (end-of-header (read-little-endian s))
        (fli-flags (read-little-endian s))
        (local-volume-offset (read-little-endian s))
        (local-offset (read-little-endian s))
        (network-volume-offset (read-little-endian s))
        (remaining-offset (read-little-endian s)))
    (declare (ignore total-length end-of-header local-volume-offset))
    (unless (zerop fli-flags)
      (cond
        ((logbitp 0 fli-flags)
         (file-position s (+ start local-offset)))
        ((logbitp 1 fli-flags)
         (file-position s (+ start
                             network-volume-offset
                             #x14))))
      (concatenate 'string
                   (read-null-terminated-string s)
                   (progn
                     (file-position s (+ start remaining-offset))
                     (read-null-terminated-string s))))))

(defun windows-link-p (filespec)
  "Is /filespec/ a Windows .lnk shortcut file?"
  ;; must end in .lnk
  (when (not (equalp (pathname-type filespec) "LNK"))
    (return-from windows-link-p nil))
  ;; needs to have the correct header
  (with-open-file (s filespec :element-type '(unsigned-byte 8))
    (handler-case
        (and (= (read-little-endian s) *link-initial-dword*)
             (let ((header (make-array (length *link-guid*))))
               (read-sequence header s)
               (equalp header *link-guid*)))
      (end-of-file () nil))))

(defun parse-windows-shortcut (pathname)
  (with-open-file (s pathname :element-type '(unsigned-byte 8))
    (handler-case
        (when (and (= (read-little-endian s) *link-initial-dword*)
                   (let ((header (make-array (length *link-guid*))))
                     (read-sequence header s)
                     (equalp header *link-guid*)))
          (let ((flags (read-little-endian s)))
            (file-position s 76)        ;skip rest of header
            (when (logbitp 0 flags)
              ;; skip shell item id list
              (let ((length (read-little-endian s 2)))
                (file-position s (+ length (file-position s)))))
            (cond
              ((logbitp 1 flags)
               (parse-file-location-info s))
              (t
               (when (logbitp 2 flags)
                 ;; skip description string
                 (let ((length (read-little-endian s 2)))
                   (file-position s (+ length (file-position s)))))
               (when (logbitp 3 flags)
                 ;; finally, our pathname
                 (let* ((length (read-little-endian s 2))
                        (buffer (make-array length)))
                   (read-sequence buffer s)
                   (map 'string #'code-char buffer)))))))
      (end-of-file ()
                   nil))))

(define-condition symlink-error (error) ())

(defun truename-aux (filespec &optional (depth 0))
  ;; RECURSIVE INVARIANT: /filespec/ is an absolute path, not relative
  ;; always returns an absolute pathname
  (when (> (incf depth) 16)
    ;; poor man's recursion detection...
    (error 'symlink-error))
  ;(when (windows-link-p filespec)
  ;  (return-from truename-aux (truename-aux (parse-windows-shortcut (namestring filespec)) depth)))
  (let ((handle (get-link-handle filespec)))
    (declare (type integer handle))
    ;; check for failure
    (when (eq handle -1)
      ;; generally, this means we can't find the file
      (error 'symlink-error))
    (with-alien ((reparseStructure (struct ReparseStructure))
                 (bytes-returned unsigned-int))
      ;; see kalmbach to interpret the constants here...
      (let ((result (DeviceIoControl handle #x900A8 nil #x0
                                     (addr reparseStructure)
                                     #x200 (addr bytes-returned) nil)))
        (declare (type integer result))
        ;; done with handle
        (CloseHandle handle)
        ;; check for failure, meaning device is not a link
        (when (zerop result)
          ;; we are done
          (return-from truename-aux filespec))
        (let ((tag (slot reparseStructure 'reparse-tag)))
          (cond
            ((not (reparse-tag-p tag))
             filespec)
            ((symlink-p tag)
             (let* ((sympath (extract-path-from-symlink reparseStructure))
                    (dirlist (pathname-directory sympath)))
               (case (car dirlist)
                 (:ABSOLUTE
                  ;; an :absolute directory can be used as is
                  (truename-aux sympath depth))
                 (otherwise
                  ;; a :relative directory must be merged with the base directory before continuing
                  (let ((merged-path (merge-pathnames sympath (directory-namestring filespec))))
                    (setq merged-path (sb-impl::simplify-win32-namestring (namestring merged-path)))
                    (truename-aux merged-path depth))))))
            ((mount-point-p tag)
             (pathname (extract-path-from-mount-point reparseStructure)))
            (t
             filespec)))))))

(defun truename (filespec)
  #.(concatenate 'string
                 (documentation 'cl::truename 'function)
  "

Under Windows, the TRUENAME of a symlink that does not
exist or of a recursive symlink, or a symlink more than
16 levels deep, is the name of the symlink.

  ")
  (let ((original-path (cl::truename filespec)))
    (handler-case
        (cl::truename (truename-aux original-path))
      (symlink-error () original-path)
      (file-error () original-path))))

(defun file-write-date (pathspec)
  #.(documentation 'cl::file-write-date 'function)
  (cl::file-write-date (truename pathspec)))

Today I searched for the method to create symbolic link in NTFS file system and found that there are no system built-in commands to do the job. The tools are provided in Resource Kit only.

Fortunatelly unbeatable SysInternals have handy command line utility for it. It is called Junction and available for download from here: http://www.microsoft.com/technet/sysinternals/fileanddisk/junction.mspx.

DVD Library on Extenders How-To

Somebody figured out how to use symbolic links and a playlist to make it possible to rip all your dvds and then play them back on an extender. Sweet!

DVD Library on Extenders How-To

Changing the Target of a Symbolic Link

I thought I might as well post this one, too, after posting how to read a symbolic link. This one isn’t so obvious, though. It forces the target to be changed on an existing link.

ln -s -n -f /new/target/path /existing/link

How to Find the Target of a Symbolic Link

It took me like twenty minutes to find this simple command on google:

readlink /path/to/link

A soft link, or more common, a symlink, is link a shortcut to the targeted file or directory. So when is removed the original target stays present. This is the opposite of a hard link which is a reference to the target and so, if the hard link is removed, so is the target.

A symlink can be created like:

ln -s /path/ linkname

from the ln man pages:

ln [OPTION]… [-T] TARGET LINK_NAME (1st form)

-s, –symbolic
make symbolic links instead of hard links

to remove a symlink

rm linkname

What is important here is to note that the command doesn’t have the trailing slash

$ rm linkname/
will output the error:
rm: cannot remove `linkname/': Is a directory

$ rmdir linkname/
will output:
rmdir: linkname/: Not a directory

So it’s a directory or not?? i guess different tools have different ways of looking at this. Isn’t GNU/Linux great?!