I loved this screen cast on Git and i strongly recommend this for any one. It runs for an hour and half. He goes from basics to all the way to advanced in the end. I would rate this to be one of the best screen casts. He explains it so well the internals of the git that it solidified my understanding of branching, merging and using remote repository well. Here are the links…

• Screen cast.
• Slide in pdf format.

I’ve been a somewhat faithful user of SVN since I started programming some 7 years back. It’s better than CVS (which I’ve used for some time) or – I don’t even know what kind of perverted – methods people used to use before that. It has worked relatively well in my very own projects and inadequately in the company space. I think the case is quite similar for many projects, the switch from CVS to SVN is now followed with another switch.

In the last few years I’ve heard rumors about the next-generation SCMs like Mercurial, Git and Bazaar. I’ve heard huge open source projects moving out from SVN to one of those. As far as I understand, they’re all based on similar approach: You clone the whole repository to your computer as a local repository, make your commits to the local repository and then push the changes back to the distributed one. Then the changes from the distributed repository are pulled to other people’s local repositories and so should you do with the changes they have made.

For a while I’ve thought that the next-generation has already become current and I should really get back on track with the revision control. Few weeks ago a perfect moment appeared when I begun an open source pet project with a good friend of mine. We decided to use some modern technology for sharing the sources with each other. So we went on with Google Code which meant Mercurial.

First thing I want to know when introducing a new programming-related practice is its integration with the practices I already use. Read: I want IDE integration. And I also want to reserve the ability to change IDE and it still works. Of course I want to run it on multiple platforms, at least Windows and Linux. I want the ability to change from Eclipse + Windows to Netbeans + Linux or any combination of these, also as IntelliJ IDEA is being open sourced why not expect support for it too. I have to admit that that’s a bit tough requirement for any system, but coming from Java world that’s kind of a default expectation.

Mercurial works nicely out-of-the-box on Netbeans, probably because they use it a lot internally at Sun, for example on OpenSolaris. The Eclipse support is not that qualified, MercurialEclipse’s integration with the Eclipse Team environment is not even close to the standards set by Subversive or Subclipse.

The things are completely the opposite on Git, the Eclipse plugin is quite good – despite being still in incubation – and integrates somehow with the Team features. The Netbeans integration is not even able to push to the master, which makes it mostly unusable.

As a conclusion, my expectations are not yet reached, and I expect to see bigger change in the near future, probably when the systems reach the integration level of SVN which might be the reason restricting wider adoption in the enterprise. I don’t know if there will be a single winner in the end. It seems that Mercurial and Git are quite even on features and adaption and Bazaar is not much behind. If there will be a single winner, as a Finn, I have to hope my countryman’s product Git to be victorious. (For the serious readers, former is intended to be a joke, I’m really not patriotic at all, though Linus is a nice guy) I want to wish a merry Christmas to all my few readers!

If you have been following my posts, you would have noticed that I made the decision on branching model for the web development. Usually ISP ’s provide a lot of choices on VCS (Version Control System)  - svn,mercurial,bazzar and git. I wanted to choose one that makes the dev process efficient and fast enough for frequent releases.

Having used powerful and popular commercial VCS which is centralized by the way, I was leaned towards going with svn because it is same as them except for lack of gui tools to admin, locking and painful multiple merges between branches. But it seemed to be the popular open source version control in the dev community now. But i wanted to step back and evaluate other options of git and mercurial which i’ve not used before even as a developer.

I was stumped when i attempted to evaluate git and mercurial. Really these two are DVCS (Distributed Version Control System) which is quite different from the CVCS (Centralized Version Control System) like cvs,perforce and svn.  Honestly there are some basic fundamental difference between the distributed & centralized that you need to be aware of.  Better explained here.

You need to first get some of the centralized concepts( the painful branching and merging) out of your brain for a while and look at DVCS in a pure mind. I’m planning to install and play with git on os/x as it looks promising among DVCS choices.

If you have sometime to kill take a look at this youtube video where Linus talks about Git. Key take away from this video is that DVCS is better than CVCS and not about the git vs other source controls.

There are two classes I wish I took back in college: Dinosaurs & Their Relatives and Operating Systems. The latter especially bugs me, because every time I see something about a browser-based operating system, I want to scream, “No, no, I don’t want that!” and then curl up in a corner for three months making something I do want.

I don’t have the know-how to do the latter, but I do have a blog, so I can do some virtual screaming.

My beef with a browser-based OS is simple: I LIKE DOWNLOADING THINGS. My WiFi connection throws a hissy fit every 10 minutes (can’t tell whether I should blame Netgear, Comcast, or tiny gremlins). Or sometimes I’m on the road or on a plane or in some place where I want to do something on a netbook and I don’t have net access. I bet I’m not alone. I’ve heard this plenty of times: “I want a netbook. All I do is browse the net anyway. Oh, and I want Microsoft Word. And I need to be able to sync my MP3 player with it. And I want to watch some movies I’ve ripped. And I want to play World of Warcraft. And if it’d scratch my back, that’d be nice too.”

As in, people basically want a tiny full-featured laptop for $300. Sure, you can do all the above stuff with your fancy Gears / HTML5 / Extensions / etc., but you’re spending so much time reinventing the wheel. Hey look everyone, I can drag and drop in my browser! Whee! I’ve only been able to do that in my operating system since at least Windows 3.1!

This isn’t exactly a new experience for the industry either. When the iPhone launched, Apple was all, “You don’t need apps! You have web apps!” Then they launched the App Store and pretended they never said that.

So what’s the best way to merge the “cloud” with a netbook’s operating system? IMHO, the solution has been around for a while. And no, it’s not the iPhone, it’s version control.

I like to download things, but I also like the idea of being able to use multiple machines and have everything — my settings, files, and apps all sync up intelligently. Download an MP3 on one computer and listen to it on another. Version control lets you do that.

Version control has traditionally been a very difficult-to-understand programmer-oriented concept, yet it’s not hard to automate much of it and slap a decent UI on it. I mean, Dropbox has managed to pull it off.

I like Dropbox, but I want Dropbox’s mechanics to extend to the entire operating system, and then some. Some ideas on what that might look like:

(1) Package package package.

Let’s keep everything in their self-contained packages — no need for a single app to insert stuff into a registry, modify environment variables, or scatter files across /usr/bin, /usr/lib, and six other folders. One single package, just like good lord intended. Why? Well, it’s a good idea generally, and it also makes syncing less of a headache — imagine syncing an app from one computer to another only to realize you forgot a DLL.

(2) Infinite Undos

It’s kind of the reason version control was invented.

(3) Share the sandbox

People love sandboxes. It sounds fun and magical, and it also happens to good for security. We like holding dangerous animals in captivity for our amusement — so long as said dangerous animals can’t jump out of their pen and devour Little Timmy. The problem with the sandboxes is that sometimes apps need to share stuff. Some apps are designed to enhance others. Other apps are meant to edit external files. Proper sandboxing lets the user decide what apps can access what, but users are fallible.

One way of building a flexible sandbox is to borrow concepts from version control — e.g. branch, clone, and merge. If App A wants to change a setting that App B depends on, rather than letting A directly edit the setting, the OS simply provides A with its own private copy. If A and B are both aware of each other and need to see the changes the other is making, the OS lets A and B merge their two copies. If A or B is at any point unhappy with the merge, either is free to revert back to their original copy. Granted, user interaction may still be required at some point here, and explaining version control to the average user is hard, but Dropbox has done a decent enough job of it so far with simple files.

And yes, multiple copies of things will eat up memory quick but (1) deltas are small and (2) disk space is getting cheaper.

(4) Standards

A lot of this could be built using existing VCS standards — SVN, GIT, Mercurial, and so forth all do what they do well enough. If the cloud that your OS depended on disappeared or became evil, then anyone capable of setting up a version control repository could build their own replacement cloud. It maximizes user-control and decreases dependencies.

—–

Now I could be talking out of my ass here — I have yet to compile my own kernel, and while I love GIT, I’ve also never poked around in its source code and done any performance testing. Still, if this feasible, someone, please make it.

UPDATE: Lots of useful comments at news.yc

Sometimes I need to check the version (build, release) of an application for which I need  to create/update test scripts or run a test.

For this I created a test script that fetches application names and their versions for the current day and outputs this into an Excel file . That is I have a separate file for each day.

It is useful if you need to check quickly which builds were deployed into the test environment yesterday evening or when the current build for an application was deployed into the environment.

You need an environment and have decided that Amazon is the way to go. There are some things that should be considered before building your OS on, or uploading to the cloud.

1. Getting started

There are a couple of ways to get started with Amazon Cloud Hosting. You can build your own AMI ( Amazon Machine Image ) or you can modify an existing public AMI.
I am not going to give a tutorial on how to go about either of these options, that is well documented by Amazon.

Firstly you need to select your operating system, Amazon has a decent selection of supported OSs, you have the main Linux flavours that you’d want to see:

• Red Hat
• Fedora
• Ubuntu
• Debian
• Gentoo
• openSUSE

along with:

• OpenSolaris

and one token Windows version:

• Windows Server 2003

After selecting your OS see if there is a public image available of the version you would like to help you make the decision of whether to build your own.

If you want to go with a public AMI you need to decide if you can trust the provider, you can boot up an instance and check out the machine, see what is installed, ensure there is no proprietary software or at least you know the software that is installed. This should be thought about as well as the considerations to be applied to building your own AMI.

The easiest way to boot your first AMI is through the AWS Management Console. It is a fairly intuitive front end to the commands you can run when you install the EC2 tools, once you learn your way around the console, it will  give you a better insight into the tasks you need to get used to when using the command line tools.

If you want to build your own AMI then you will be in control of exactly what is installed, but you should consider the next point.

2. Is it Repeatable?

Now you have decided how to get started, are you able to rebuild your environment to the same specifications in a short period of time? If the worst happens and for whatever reason your server gets shutdown, lost, deleted, can you re-provision it?

Whether you are building your own or modifying an existing AMI there are ways to make the process repeatable. The simplest way is to keep a record of the process, a list of what is installed, the location of the binaries, where they are installed and any configuration steps.

If you have read the Amazon docs, and I suggest you do, then it should be fairly clear that all of the processes involved in either building your own or starting with an existing AMI are scriptable ( at least on the Linux side of things ). Using Amazon’s EC2 tools you can control AMI instances through scripts quite nicely, boot instances, attach EBS volumes, upload files and execute commands, everything necessary for automated environment creation. This in theory could be the beginning of your own Amazon DSL, but that is another blog post…

When creating your AMI through a loopback file, once again you have almost infinite control which should make the scripting easier. Once you have finished with your masterpiece you need to be sure to bundle it and upload it to Amazon.

3. Persistence

Now you know how you want to create your AMI and make sure the process is repeatable you need to think about the persistence of your data. By default none of your information will be saved when your AMI instance is shut down, unless you re-bundle that instance to a new bucket or with a new manifest name everything you did since you booted will be lost.

Obviously this isn’t workable, Amazon have a solution, and they call it Elastic Block Storage (EBS). Again there are Amazon docs for how to create and use EBS volumes. EBS volumes are attached to your instances as devices which you can mount as you would any other device, they are easy to use and it is simple to take backups by using the snapshot feature.

The real consideration is how to structure the installation of your software so you get the benefits of data persistence. This is obviously going to vary massively on the type of system you are running and the software you are installing, but components like web applications and databases and such should be configured to use EBS for information storage. If you plan carefully you can have EBS volumes with discrete services installed that can be cloned and attached to new instances at will, allowing for a truly scaleable architecture.

4. Versioning

How can you keep a track of differences in your environments from one upload to the next? There are a couple of really manageable ways, if you are scripting your setup then you can use version control and a release or tagging process of the script to keep a track of environmental changes.

If you are only at the stage of keeping a track of changes through component lists then it is possible to bundle your AMIs to a bucket appended with a version, or indeed specify a manifest file versioned in the same way, meaning you will keep your historic changes as an entire environment. If you are version controlling your scripts I would suggest uploading to unique bucket names appended with versions to begin with anyway, it just means you can immediately revert to the old environment if anything goes wrong.

After using Amazon EC2 for a while it will become clear how environments can be used as just another architectural component, they can be provisioned on demand, have services attached and started and within minutes you have a custom “box” available for use in the cloud, which can be torn down and brought back to its initial state very quickly.

One more thing, whatever you do, don’t block your SSH port

Etherpad is (but not for long) an awesome tool that lets you do real time collaborative editing for free without creating an account. My favorite use case for Etherpad is for conducting phone screen interviews because it allows me to copy and paste pre-written questions for the candidate to read, and also enables me to ask coding questions to software engineers.

It came as an unpleasant surprise on Friday (2009-12-05) when a friend discovered that Etherpad disabled new pad creation because they had been bought by Google to work on Google Wave. This sparked a multi-hour discussion with some fellow engineers about how we might implement Etherpad since we had been unable to find any suitable alternatives that matched our requirements (free, no sign up, real time editing). Our discussion ultimately converged to a solution that we later found out was very similar the Google Wave implementation. Fortunately, Etherpad came back online the next day so I decided to investigate how Etherpad had solved this very interesting problem.

To keep everything in sync, the Etherpad server holds the authoritative copy of the document. When an Etherpad client updates the document, it sends a changeset to the server. The client does not receive any information back when it posts a changeset, nor does it poll the server for updates, so it appears that the client must initiate some sort of semi-persistent connection to the server on page load that allows the server to stream updates to the client.

While a client is updating the document it sends changesets to the server every N milliseconds until there are no more changesets to send. In practice this means most changesets are only a few characters depending on how fast you type. The changesets that the client is sending are quite cryptic but I was inspired to do some reverse engineering.

Actions:

1. Type “hellodude” on line 2.
2. Highlight “hellodude” and PASTE “bye” (I had copied “bye” to my clipboard outside of Etherpad)
3. Highlight “bye” and delete it with one keystroke

Changesets that Etherpad generated:

1. "changeset":"Z:2>9|1=1*0+9$hellodude"
2. "changeset":"Z:c<6|1=1-8*0+2$by"
3. "changeset":"Z:6<3|1=1-3$

Analysis:

• “Z:2″, “Z:c” and “Z:6″ are the length of the current document in base 36 (1,2,…,8,9,a,b,…,y,z,10,11,…,18,19,1a,1b,…) BEFORE the change occurred. The “length” is the number of distinct positions where your cursor can exist.
• “>9″ means the net effect of the changeset was to add 9 characters. “<6″ means the net effect of the changeset was to delete 6 characters (deleted 8 characters and added 2 characters).
• “|1″ is the line number that the changeset applies to (this parameter is omitted for the first line which has index 0)
• “=1″ is the starting position of the cursor for the changeset (index starting at 1 for each line)
• “-8*” are the absolute number of characters that were deleted (omitted if no characters were deleted)
• “*0″ is either present or not present in everything that I have tried so I can’t figure out it is is more than just a delimiter.
• “+9$”, “+2$” and “-3$” are the absolute number of characters that were added
• “$hellodude” and “$hi” are the characters that were added. If characters were deleted and no characters were added then this is empty (which means the client is requesting to delete the character at a specific position, regardless of its value)

I assume the server is streaming back similar changesets to the client but I don’t know for sure since the persistent connection isn’t visible in Firebug and I am not interested in busting out Wireshark. This isn’t a complete picture (it gets a little bit more complicated for multiline changesets, and I don’t care about text formatting) but I am satisfied that I was able to figure out the basics.

Here’s a simple script which can be useful if you want to version control the Firefox profile folder, or just to keep it slim. Please see the embedded documentation for more information.

cleanup.sh

#!/bin/sh
#
# NAME
#    cleanup.sh - Clean up the Firefox profile folder
#
# SYNOPSIS
#    cleanup.sh [options] [directory]
#
# OPTIONS
#    -v,--verbose    Verbose output
#
# EXAMPLE
#    ./cleanup.sh -v
#
#    Cleanup in ./Profile
#
#    /path/to/cleanup.sh /path/to/Firefox/profile
#
#    Cleanup in /path/to/Firefox/profile
#
# DESCRIPTION
#
#    "Vacuums" the .sqlite files and sorts cert_override.txt and persdict.dat
#    alphabetically, to make them suitable for version control.
#
# BUGS
#    Email bugs to victor dot engmark at gmail dot com. Please include the
#    output of running this script in verbose mode (-v).
#
# COPYRIGHT AND LICENSE
#    Copyright (C) 2009 Victor Engmark
#
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
#
################################################################################

# Output error message
error()
{
    if [ -z "$2" ]
    then
        error_code=$EX_UNKNOWN
    else
        error_code=$2
    fi
    echo "$1" >&2
    exit $error_code
}

usage()
{
    error "Usage: ${cmdname} [-v|--verbose] [directory]" $EX_USAGE
}

verbose_echo()
{
    if [ $verbose ]
    then
        echo "$*"
    fi
}

# Use for mandatory directory checks
# $1 is the directory path
# $2 is the (optional) error message
directory_exists()
{
    if [ ! -d $1 ]
    then
        error "No such directory '${1}'
$2" $EX_NO_SUCH_DIR
    fi
}

# Make sure an executable is available
# $1 is the path to the executable
# $2 is the (optional) error message
executable_exists()
{
    if [ ! -x $1 ]
    then
        error "No such executable '${1}'
$2" $EX_NO_SUCH_EXEC
    fi
}

ifs_original="$IFS" # Reset when done
IFS="
" # Make sure paths with spaces don't make any trouble when looping

PATH="/usr/bin:/bin"
cmdname=`basename $0`
directory=$(dirname $(readlink -f $0))

if [ -z "$profile_directory" ]
then
    profile_directory="${directory}/Profile"
fi

# Exit codes from /usr/include/sysexits.h, as recommended by
# http://www.faqs.org/docs/abs/HTML/exitcodes.html
EX_OK=0           # successful termination
EX_USAGE=64       # command line usage error
EX_DATAERR=65     # data format error
EX_NOINPUT=66     # cannot open input
EX_NOUSER=67      # addressee unknown
EX_NOHOST=68      # host name unknown
EX_UNAVAILABLE=69 # service unavailable
EX_SOFTWARE=70    # internal software error
EX_OSERR=71       # system error (e.g., can't fork)
EX_OSFILE=72      # critical OS file missing
EX_CANTCREAT=73   # can't create (user) output file
EX_IOERR=74       # input/output error
EX_TEMPFAIL=75    # temp failure; user is invited to retry
EX_PROTOCOL=76    # remote error in protocol
EX_NOPERM=77      # permission denied
EX_CONFIG=78      # configuration error

# Custom errors
EX_UNKNOWN=1
EX_NO_SUCH_DIR=91
EX_NO_SUCH_EXEC=92

# Process parameters
until [ $# -eq 0 ]
do
    case $1 in
        -v|--verbose)
            verbose=1
            shift
            ;;
        *)
            if [ $# -ne 1 ]
            then
                usage
            fi
            profile_directory=$1
            shift
            ;;
    esac
done

# Announce that we're running
verbose_echo "Running $cmdname at `date`."

# Make sure the directory path doesn't end with a slash
profile_directory="${profile_directory%\/}"

# Preliminary checks
executable_exists "/usr/bin/sqlite3"
directory_exists "$profile_directory"

verbose_echo "Profile directory: '${profile_directory}'"

cd $profile_directory

# SQLite cleanup
for sql_file in *.sqlite
do
    verbose_echo "Cleaning '${sql_file}'"
    echo "VACUUM;" | sqlite3 $sql_file
done

# Sorting files
for file in cert_override.txt persdict.dat
do
    verbose_echo "Sorting '${file}'"
    sort --output=$file $file
done

cd $directory

verbose_echo "Cleaning up."
IFS="$ifs_original"

verbose_echo "${cmdname} completed at `date`."
exit $EX_OK

we all know last moment changes always result in that dreaded 3:00AM call from support guys,this has been proven infact there is a phd[pdf] thesis on it,Andreas Zeller from saarland university has done research on change inducing fixes,the research[pdf] can be simplified as linking version control and bug database together to track when the bugs are introduced in system and how they relate to some changes made recently.

below is the google tech talk given by Andreas Zeller where he discusses concept of delta debugging and software archaeology[pdf] further.

I recently found out how satisfying it is to delete old code. Why? Because it improves the signal to noise ratio of individual code files and of the code repository as a whole.

Imagine you are a new hire who is trying to get familiar with a repository and learn how things are done. If your repository is cluttered with old code files then looking for good examples of how to do something is like walking through a minefield of “// DEPRICATED!!!”  and “// TODO: get rid of this hack”.

Not only is old code clutter, it is dangerous. Consider a function calculateFoo() which was written assuming X was true (a reasonable assumption at the time). What happens if X ceases to be true at some point in the future? If calculateFoo() is still being used, then it creates a bug (which may or may not be noticed in a timely fashion).

Worse, if nobody is using calculateFoo() then the bug turns in to a time bomb waiting to explode on the next person who needs to calculate foo. This person will probably eyeball the code and if they don’t happen to notice the potentially subtle bug, then they will start to use calculateFoo() assuming that it works.

Another possible scenario is that a new programmer needs to calculate foo but needs the result formatted differently then how calculateFoo() formats the result. A cautious programmer might not want to modify the behavior of calculateFoo() for fear of breaking any existing uses of the function (even though there aren’t any). Therefore this programmer might add a new function calculateFooNewFormat() that calls calculateFoo() and then formats the result in the desired way. The code is now more complex then necessary because it handles a non-existent case.

The solution? Delete code that is old and unused!

Today I bumped into a solution for deleting deleted Team Projects (yip, that’s right: deleting Team Projects that should have been already deleted). A while ago we noticed that Team Projects in TFS2008 SP1 that were deleted with TFSDeleteProject.exe were still visible in the Source Control Explorer and re-running TFSDeleteProject.exe for the same Team Projects resulted in an error message that the Team Project could not be found on the Team Foundation Server.

But due to this entry in the TFS forum I was finally able to remove the Team Projects from Source Control Explorer as well.

Solution:

• Create new Team Project with exactly the same name as the Team Project that wasn’t completely deleted and specify the option to not create the version control folder.
• After the Team Project Creation wizard finishes, you need to run TFSDeleteProject.exe for the newly created Team Project.

No sign anymore of the deleted Team Projects in Version Control! Sounds logical?!

We ran across the article “Overcoming Resistance to Change by SCM” by Ben Weatherall on CMCrossroads. The excerpts below exemplify the need for tools like AccuRev that support changing processes and Agile adoption. Ben Weatherall has kindly allowed us to re-post these excerpts so that we may share his insight with all our readers.

Excerpts from “Overcoming Resistance to Change by SCM”

By Ben Weatherall

“As an industry, SCM is conservative – we hold the corporate jewels in our hands and we are reluctant to let either processes, procedures or personnel have a chance to mess them up. In fact, when they do get messed up we tend to lose our jobs. Then along came Agile and the need to support it while maintaining both technical and professional integrity.”

“Some of the old versioning tools could not handle merges of codebases where one or more of them had been refactored, so many of us had to rapidly switch tools. This often required a lot of hidden work to get triggers, wrappers, interfaces, metrics data collection, etc. working with the new tools. At the same time, there was a push to use tools that supported such things as backlogs and user stories instead of “just” defects. It was often the case where a development organization would bring in their own tools and tell SCM to either use them or get out of the way.”

“All of a sudden SCMers had to start rolling out their own tools, integrations, customizations, etc. with a speed even greater than the typical Agile development organization. And with significantly less personnel. We now leveraged what we could of the Agile methodology. We did rapid prototyping knowing ahead of time that things would have to be completely redone at a future time. Our assumptions, ones that made sense at the time, was that once a branching/stream structure and new tracking tools were in place to support this new way of developing software, it would stabilize and we could catch up.”

“The end result of this is that SCM has to adapt and do so at a rate that would have been considered impossible just a few years ago. We must adapt technologically, but we have to maintain the core integrity of principals of SCM: identification, reproducibility and traceability. A typical development sprint team consists of 5 people of which at least one represents the Quality organization. A typical SCM “team” consists of only one person. Across the software development industry as a whole, the SCM headcount is probably only 1-2% of the combined Development and Quality “workers.” For those of you in larger companies or those who are involved in admin-heavy tools, this probably sounds low; but from empirical evidence it is not. This is especially troubling for those who have to support multiple teams who are using different methodologies or variants of them.”

Ben Weatherall is currently based in Fort Worth, Texas where he practices Practical CM on a daily basis supporting a modified Agile-SCRUM development methodology. He uses a combination of AccuRev, CVS, Bugzilla and AnthillPro (as well as custom tools). He is a member of IEEE, ASEE (Association of Software Engineering Excellence – The SEI’s Dallas based SPIN Affiliate), FWLUG (Fort Worth Linux Users Group), NTLUG (North Texas Linux Users Group) and PLUG (Phoenix Linux Users Group).

For the entire version of this article on CM Crossroads, please visit: http://www.cmcrossroads.com/cm-journal-articles/13047

Your new project has been given the green light. You need to get your team up and running quickly.  Could a cloud based development environment be the answer? This blog discusses some of the options  and issues for moving to a  hosted development environment.

One of the first tasks for any Agile project team is to establish a robust, reliable set of development tools and associated infrastructure. Generally on any new development endeavour the following needs to be put in place:

• Locate or create a source code repository and import your skeleton project.

• Locate or create a continuous integration server to build your source code, run your tests and notify the team via email of any failures.

• Locate or create a task/user story tracking server and/or issue management tool and add your project to it.

• Locate or create your test environments for developer smoke, integration, and functional testing.

Now you might be lucky and have much of this infrastructure in place, in which case adding some extra users, a new project and associated set of build jobs might be relatively straightforward. However, if it isn’t you will need to   allocate a decent chunk of time to setting things up yourself. This may involve procurement of hardware, installation of an appropriate OS, installation of the relevant applications, providing secure access to project team members and so on. This gets more complex if the team is distributed and the infrastructure must be accessible beyond a LAN.

Often these development tools are open source, which means that while the cost of acquisition for the software may be zero, the ongoing maintenance and support will probably require specialist knowledge.  Any time your team spends doing this is time that could be spent progressing the project.

With either option, server space and administration time is required and there are obviously costs associated with this, and the costs may be disproportionately large to small and medium-sized organisations.

In the same way that SaaS offerings for email and collaboration suites (such as Google Apps) have sought over the past few years sought to turn these services into low-cost, click-and-go commodities, there are now equivalent hosted solution options available for Agile development infrastructure.

Hosted version control solutions have been available for a while and the market has expanded rapidly over the past few years. Collabnet (the people behind Subversion) and CVSdude are probably the best known. Both companies have expanded their offerings, and not only provide version control but are bundled with or integrate to a host of other tools.  These now compete with newer entrants like Beanstalk, Assembla and Unfuddle (which also does Git hosting). GitHub itself has seen huge growth over the past year, with over 400 new users and 1,000 projects being added each day.

Coming from the other direction are suppliers of Development collaboration and productivity tools, Atlassian are probably the biggest player here and their hosted JIRA studio suite based around the very popular Jira issue tracking tool was launched in May 2008.  Fogbugz is another alternative, based around an integrated Project Management, Wiki, Issue Management and Helpdesk set of tools.

Each vendor has a different focus with JIRA studio and Collabnet’s Team-Forge perhaps the most fully featured for development teams. Both offer a very comprehensive stack and are moving towards the idea of an Application Lifecycle Management (ALM) suite. It will be interesting to see how these platforms develop, and how the traditional Enterprise tool vendors (IBM Rational, Microsoft and Borland) respond.

If  a platform sounds too restrictive and you want to “pick and mix” your own set of tools you can.  There are also some great tools which focus on a specific area – Basecamp for Project Management and Lighthouse for Issue Management are a couple of well-known examples worth looking at. Most of these tools have open APIs that enable you to integrate easily with others, so getting together an integrated  set of tooling is easily achievable.

My advice here is to be clear about what you want from these tools. Some are very feature rich and developer centric while others do a great job of providing a clean and simple process and interface. So which tools suits you will depend on your project, your team, and your organisation. What is clear though is that there is growth in this sector, increased competition and greater integration between the providers. All of this can only be good for those who are happy to outsource their development environments – increased choice and competition against a backdrop of  decreasing hosting costs.

Moving on from  managing your project to testing it, the use of externalized environments that allow teams to deploy a release of a web-based application and run functional tests against it, is trickier. Depending on the nature of the application and its associated runtime dependencies this may require the creation of a bespoke environment. However, recent developments in cloud computing should soon make this much easier. Google’s App Engine for example allows you to run Java (and Python) applications on their infrastructure. So if the AppEngine is your production target, creating a test environment that is a clone of production should be a relatively straight-forward activity. More recently in August this year, SpringSource launched their Cloud Foundry which allows you to rapidly deploy a test (and also production) environment for your Java web application with a few mouse clicks and Microsoft have also weighed in with Azure. Both Google and Microsoft are promising tighter integration with the IDE, and I’ll be watching these platforms closely.

One area which is less mature is hosted continuous integration. There are currently only a small number of pioneering providers in this space, which may surprise some, as the practice of continuous integration is at the heart of the Agile development process. The SaaS multi-tenant application model does not fit easily with the requirements for continuous and often complex software builds. It is computing resource intensive activity, especially for programming languages such as Java, and this will inevitably impact the cost of such a service to the end-user. Mike CI is one of these pioneers and there is a good analysis of the others here.

Now I’m not saying trusting your code to a 3rd party is a simple decision to make. There are often legal, security and organisational hurdles to consider. It isn’t be for everyone and for many large corporates it might be a step too far. But for many people the cost, convenience and management overhead of maintaining it all yourself does not stack up. Your team is in place to write great software. For your next project, I’d recommend that you seriously consider using these low-cost, on-demand hosted services.

git is too complicated for me. It’s supposed to be awesome, but I just can’t figure it out.

Visual Source Safe, CVS, SubVersion, 그리고 Git
한국 사람으로써 발음도 힘든 Git, “깃”인가 “짓”인가? 나는 “짓”이라고 발음하는데 NSCoder Nigher에선 “깃”이라고들 하더라. 근데 어떤 사람들 보면 또 “짓”이라고 하기도 한다.

근데 도대체 이렇게 버젼 컨트롤 프로그램들이 다양한 이유가 뭘까? 도대체 뭐가 다르길래 자꾸만 새로운 것을 내놓을까? Visual Source Safe가 문제가 많다는 것이야 뭐 MS 제품에 빠진 사람들 아니면 다들 아는 것이고, CVS와 Subversion은 CVS 사람들도 Subversion을 Better CVS라고 부르니 뭐 그다지 궁금하지도 않고. ( 사실 좀 차이가 있다. SubVersion이 파일 깨지는 것에 덜 취약하고 속도가 빠르다고 알려져 있다. 그 이유는 CVS는 파일간의 차이를 저장하는데 반해, SubVersion은 완전하게 그 다른 버젼들을 저장하는 것이라고 한다. 내가 읽은 것을 기억하는게 맞다면.. ) 그렇다면 Subversion이 훌륭하게 좋은데 왜 Git이란 새로운 것이 나왔으며, 요새 Git을 많이들 쓰는가? 적어도 NSCoder Night에선 Git이 내세인거 같다.
Git은 다른 것들과 달리 분산형 시스템이다. 즉 commit을 자체에 하는 샘이고, 어떤 주기가 오거나 어떤 명령이 오거나.. 혹은 네트워크에 연결이 될때, 서로 간에 통신을 통해서 transparent하게 그 버젼을 동일하게 유지시켜 준다. 그러므로 빠르다. 또한 항상 네트워크에 연결될 필요도 없다.
근데 그 외는 무슨 차이가 있을까? 우연히 다음의 문서를 보게 되었다. GitHub를 통해서..
관심있으신 분들은 읽어보시기들 바란다.

Git for Computer Scientist

Yesterday (during the Source Code Management session of Brian Harry at TechEd 2009 in Berlin), I noticed for the first time that merging partial changes in a changeset is also clearly visible afterwards. I just found some time between sessions to test it out …

In the screenshot above you may notice that changeset 9 (dev branch) is merged to the main branch. But look at the yellow color of the bar after the merge and the tooltip that is shown when I hover over changeset 11. Changeset 9 consisted of 2 different files that were modified while the merge operation took only 1 modified file in changeset 11. Valuable information!

Currently in TFS2008 when you create a new branch in your version control structure, all files are copied to your local worspace and marked as pending changes. This means that a branch in TFS2008 is not seen as an atomic operation. You were required to check-in the local changes before the branch got created on the server. If your branch would consist of a huge set of files, branching could become a time-consuming operation and the system would not prohibit you to do some modifications to the pending changes.

In TFS2010, a branch is now seen as an atomic operation: it doesn’t copy all files into your local workspace and the branch action is immediately checked-in – no option anymore to do some local modifications. The child branch will always be an exact copy of the selected parent branch folder. You will get a messagebox that will notify you of this action when you hit the branch button …

You will need to request a get latest on the newly created child branch to have a local copy of the sources.

Some simple changes can make a big difference!

Late Binding mode provides more flexibility for Parallel Development compared to being in Online mode. One major difference is nobody needs to keep the file checked out in branch (e.g. 0.1.1(R)). So setting the correct context (CA SCM) part is delayed till the users commit their changes. Only when someone needs to commit the changes, it requires a CA SCM context (Package, Check In Process and other processes).

Let’s assume, CA SCM Administrator has helped initializing the creation of a project (CA SCM Project) and baselined the source code in that project. And it has the below structure:

====== Source Code Baselined ======

CA SCM Broker: BrokerMachine

CA SCM Project: ParallelDev

State for Development: Development

Check In Process: Check In on Branch and Trunk

View Path : \Repository\Source Code\Common

======= Pre-requisite Information ========

Solution File: Solution.sln  (under \Repository\Source Code\Common)

Install CA Software Change Manager client

Install the Visual Studio Plug-in for CA Software Change Manager

SCM Explorer: Available from “View -> SCM Explorer” menu in Visual Studio IDE.

======= Late Binding Mode – First Developer – user1 ========

User1 opens Visual Studio IDE and then opens SCM Explorer as being instructed. Clicks on “New Broker Connection”, provides login details and connects to CA SCM broker (BrokerMachine).

Navigates to CA SCM Project “ParallelDev”, and navigates till repository path “\Repository\Source Code\Common”. He finds the solution file “Solution.sln” in the list view in right pane.

Right clicks on the solution, and performs “Open Solution”. Wow! The solution is in his own machine now and it is open in his Visual Studio with managed glyphs (icons beside files in Solution Explorer).

Next important thing is – start working as I don’t need any context now.

User1 is least bothered about who else is working on which file this time. He started changing in several files.

======= Late Binding Mode – Second Developer – user2 ========

User2 opens the solution in his machine in Visual Studio IDE from CA SCM. The solution and all its files are shown as managed.

User2 also updated few files from the IDE.

======== Commit – Set Context ========

Before any of the users commit their changes, it requires a context (CA SCM), so let’s set the context. If you see, I don’t even have set the check out process this time. And for check in process, I have kept both the option Branch and Trunk enabled, so I will explain both the cases separately.

==== User1 – Context ====

Package: dev1

Check In Process: Check In on Branch and Trunk

==== User2 – Context ====

Package: dev2

Check In Process: Check In on Branch and Trunk

====== Commit – On Branch ======

If both the users are committing changes on Branch, chances of getting a conflict in local workspace is absent. Only when those branch versions are merged in CA SCM (using CA SCM Workbench), there it may create merged (M) tag versions, which we need to resolve and create a normal (N) tag.

====== Commit – On Trunk ======

What if both are committing on Trunk, now it may find conflicts (both started working on 0′th version of the file, both users have changed the same file from their own workspaces, and one of them has committed the change to repository on Trunk). So, second user has to resolve the conflict from within IDE. Only after that it allows to commit the change (conflict file) to the repository. In this case, users don’t have to resolve any merged (M) tag from CA SCM Workbench.

======= All Set Now! – Let’s Change Few Other Files ======

Late Binding Mode and Online Mode - These are the two modes of working that this plug-in supports. Both has its own merits and demerits.

Online mode means you are always connected to the CA SCM (Software Change Manager) server or rather broker. Any change that you make in the local workspace in Visual Studio is immediately reflected in the CA SCM repository. So, you check out a file to start working in it. Really good thing for someone who wants to ensure that the fellow developer has started working on something at certain date and time. Also, in certain situation it restricts two users from updating or modifying a single file. What will happen if for some silly reason you are far far away from that giant server! You’ve got to be disconnected, but this mode does not allow that.

As a developer I will be little bit biased towards Late Binding for the simple reason of its simplicity and flexibility. Moreover, it does not force me to stay connected 24×7 to that giant server that I just talked about a little earlier. What a relief, may be now I can disconnect from the server, take a nab, wake up and do some changes in code (of course from Visual Studio). When I come back near to that roaring server, I can connect back to it and commit all my code changes to CA SCM repository. New versions of the changed files are now created.

Make your choice!

Command-Line Fear

I have been windows user from past 8 years anode Graphical User Interface. I was in dire search for a good source control because Source Safe was bringing nothing but disappointment to my life. After researching n the market, I learned that Subversion (SVN) and Git are two which are popular but as soon as I found out that it’s a command-line version control and mostly used by Linux and Ruby users, I opted to and started learning Subversion. After couple of months, I noticed that Git was gaining a lot of traction so I logged into my TekPub account and started on the Git screen casts. I have to tell you, I switched my source control right away with no sign “command-line fear” and full of excitement. If you are a windows user and looking for a good source control, I request you to read this post with “open-mind” which hopefully will help you decide if Git is for your not.

Why use Git?

1. Git is Distributed: You don’t have to have a centralized system.The repository can be locally on your computer. You can have centralized system, but there is a lot of freedom with it.
2. Git is Fast: Git has a faster and different way of tracking file changes.
3. Git is Reliable: Your repository is local which can be publicly shared as needed so there is not centralized server which can blow up and take your source history.
4. Git is Cheap: Open Source with no centralized server to maintain make it cheap.
5. Git is Easy: Git is very easy to use
6. Git is Fun: Easy, Cheap, Reliable, and Fast which helps you be more efficient …..sounds like fun.

How Git Emereged?

Git was initially designed and developed by Linus Torvalds.It was one of his personal project to support Linux Kernel because Linus was didn’t like any source control software out there.Therefore, Linus rolls his own source control in April 2005 which has speed and efficiency, distributed, and bullet proof.

I have asked myself several times “How Subversion makes copies of each revision”. Whenever, a new change has been committed a new version of the current files will be automatically created. One commit creates a one copy of the whole respository folder.

Does it make Subversion respository a huge monster of hard disk ? Who eats your disk daramatically in sense of a large project.

Today, I found answer for that kind of wonder that is Subversion uses Cheap Copies.

“                 Cheap Copies

“