Science Journalists … hmmm

“…Microsoft researcher Byron Cook and his colleagues did the unthinkable — they hacked a fix. When Cook tried to describe the workaround, however, he found it impossible to explain with existing mathematical symbols. His only option, he decided, was to invent new ones…” – from Wired

Software applications are based on layers and the communication between them, so the main task in the design of an application architecture will be the identification and creation of layers and then to specify a communication language and set the rules and guidelines of the interaction between these layers.
To start the design of an application architecture, the architect should keep in mind the following points:

• Separation of concerns
• Clearness in the communication rules between layers.
• Implementation of loose coupling between layers using abstraction (Interface)
• Separation of component between logical layers (UI-Business-..)
• Keeping the data format consistent within a layer or component

If we really take into consideration these points, the application will be easily extensible and testable, and its components will be clearly identifiable and reusable.

He estado un poco desconectado luego de haber creado mi log debido a que he pasado por un proceso de cambio en lo laboral.

Actualmente trabajo, desde el 16/11, para un empresa que brinda ingeniería en varias especialidades. Particularmente, con 4 personas más que conforman el equipo, comenzaremos a desarrollar el software de gestión para dicha empresa. Al llegar me he encontrado con un grupo que tiene un líder pero que fue armado con perfiles totalmente diferentes y todos los analistas venimos de grupos de trabajo distintos y con “costumbres” también distintas. Creo que lo más complicado de este nuevo empezar es poner todo a punto para “comenzar con el pie derecho”. Hace falta mucho pero en resumen creo que el primer punto a definir es el modelo operativo o metodología de desarrollo que utilizaremos.

Durante mi experiencia laboral me ha tocado vivir muchas situaciones diferentes en cuanto a los modelos utilizados y también he tenido el privilegio de introducirme en algunos. Entre todo lo que he vivido y leído finalmente me he decidido a definir mi propio modelo que, en retrospectiva, pienso que es le mejor modelo que hubiese podido utilizar en todos los proyectos en los cuales he participado y actualmente aplico para mis proyectos de desarrollo web que realizo fuera de mi actual trabajo. Las diferentes etapas son las siguientes:

• 01 Requerimientos
• 02 Análisis
• 03 Diseño y Prototipado
• 04 Desarrollo y Pruebas de Sistema
• 05 Pruebas de Integración
• 06 Pruebas de Usuario
• 07 Implantación

La particularidad del proceso es el hecho de que hace un “mix” entre varios modelos ya conocidos (3 en realidad) y es por ello que lo he bautizado XthreemiX (X3miX). Debido a que no cuento con gráficos (al menos en el corto plazo debido a la escacez de mi tiempo) voy a tratar de explicar el método con palabras (prometo exponer imágenes a futuro para mejor comprensión).

Para comenzar, hay que tener en cuenta que los 3 métodos que se utilizar son: espiralado, scrum y lineal. Espiralado es usado en las etapas 01, 02 y 03, scrum en la etapa 04 y lineal en las etapas 05, 06 y 07.

Para hacer la explicación más fácil tomaremos como ejemplo el desarrollo de un módulo de gestión de imagenes (álbum de fotos )

Pasemos a explicar la pimera parte (etapas 01, 02 y 03). La clave de esta primera etapa y del éxito de la misma es la rapidez y sencillez aplicada a las etapas 01 y 02. El objetivo de la etapa completa es brindar una especificación técnica del producto por medio de iteraciones entre todas las sub-etapas que la componenen. Nos enfocaremos ahora en cada etapa en particular.

El objetivo de la etapa 01 es definir los “requisitos de alto nivel” (entiendase por requisito de alto nivel cada uno de los aspectos generales que debe satisfacer el producto). Por ejemplo:

• REQ.1: El módulo debe permitir gestionar fotografias
• REQ.2: El módulo debe permitir generar álbumes de fotos en formato PDF

Hay que tener en cuenta que los requisitos deberán ser definidos junto al cliente y que, seguramente, de cada reunión con el mismo nos llevaremos nuevos requisitos pero no debemos nunca esperar hasta tener definidos “todos” hasta pasar a la siguiente sub-etapa; si no, por el contrario, debemos para a la próxima para listar las “funcionalidades de alto nivel” para el conjunto de requisitos que han sido definido con el cliente. Aquí presentamos a la sub-etapa 02.

Como he anticipado, la etapa 02 nos dará la lista de funcionalidades de alto nivel para cada uno de los requisitos definidos. Al finalizar cada reunión con el cliente, y con un nuevo conjunto de requisitos en la mano, deberíamos comenzar a definir la lista de funcionalidades que se desprende de cada uno. Ejemplo:

• AN.1: El módulo debe permitir crear nuevas fotos. Requisitos relacionados: REQ.1
• AN.2: El módulo debe permitir crear álbumes. Requisitos relacionados: REQ.1
• AN.3: El módulo debe permitir modificar álbumes. Requisitos relacionados: REQ.1

Estas son solo algunas de las funcionalidades que se desprenden pero, como pueden observar, la definición de las mismas está definida haciendo uso de un alto de nivel similar al de los requisitos.

Como podrán imaginarse, definir estas listas no debería llevar más de 3/4 jornada (la 01 en la reunión con el cliente y la 02 desprendida de nuestro análisis aunque tal vez debamos validar con el cliente) y al finalizar ese tiempo estamos listo para comenzar a diseñar y prototipar en la siguiente sub-etapa.

La sub-etapa 03, diseño y prototipado, tal vez demande un poco más de tiempo que las dos anteriores debido a que el objetivo de la misma es dar especificaciones técnicas para cada una de las funcionalidades definidas y desarrollar un prototipo de un conjunto agrupado lógicamente para poder presentarlo al cliente y comenzar con la validación de nuestro producto (en realidad, de cada parte o paquete de nuestro producto). aquí deberíamos utililizar herramientas que nos permitan presentar prototipos “usables” a nuestros clientes para que los mismos puedan iteractuar con nuestras “propuestas”.

Para resumir, el proceso espiralado se dará de la siuiente forma: 01->02->03->01->02->03->01->02->03->…->01->02->03 hasta que tengamos el producto que necesita nuestro cliente “conceptualmente definido” y validado por el mismo.

Es evidente que la rapidez con la que debe ser llevada a cabo cada una de estas partes es punto clave para el éxito del modelo pero también puedo asegurarles que, habiendo utilizado otras metodologías anterioremente, creo que la facilidad con la que puede ser ejecutada cada una de estas sub-etapas y el poco nivel de “burocracia” que tienen los entregables de cada una de ellas hará de ustedes y su grupo de desarrollo un equipo muy eficiente.

En la próxima entrega me dedicaré a explicarles la etapa 04 pero ahora debo ir cortando debido a que, como les decía al inicio, mi tiempo es escazo. Nos volveremos a ver…

While researching user experience design techniques, I stumbled upon some nifty whiteboard magnets for prototyping called GuiMags as well as a complementing book called The Unplugged.

GuiMags look like the nicest way to prototype something before going to HTML and CSS. Labor intensive forms of prototyping don’t seem to add much value, and paper and traditional whiteboard prototyping only works until you’ve changed your mind about something and have to throw your work in the trash or erase half the board.

Although I decided to postpone a magnet purchase until I am doing design again, I was able to get my hands on the book. Its premise: we limit ourselves by the technologies we use. Instead of thinking outside the box, we’re often thinking and functioning in it. A large part of this thinking inside the box is how we develop software.

Although, everyone interested in the topic should pick up the book, here are a few of my takeaways:

• Every major form of art that involves technology (music, film, video games, graphic design) starts outside technology. Artists do not limit themselves by their technology but by the limits of their own minds. As a software engineer, you often limit yourself by the technology you use day-to-day.
• Spend as much time as you can iterating on concept and design before going to implementation.
• Design the software front-end not the back-end first.
• Just like there are code freezes, freeze the product when it has passed the design phase.
• It is often wise to outsource the implementation.
  • This serves as a peer review of the design before it goes to implementation. Software developers traditionally think about the back-end first.
  • Different cultures have different strengths: “England and Western Europe are great at design, Ukraine and Macedonia have amazing and prompt developers who can think for themselves, the Netherlands always emails back the same day, India is extremely polite, etc.”
  • Work can be done while you are sleeping. “This can cut the development time in half.”
  • Because you already know what you want and won’t be constantly changing the design, contractors will want to work with you even if you pay less.
  • Only be satisfied with five-star developers.
  • Pay more than you agree to pay.
  • Do one-week sprints. Longer sprints end up getting delayed, with excuses.

With the last (sub)point in mind, I think this methodology is well-suited for an agile development process.

There is a lot to gain from reading the book, so make sure to grab a copy for yourself.

Test driven geliştirme yapmak, eğlenceli olduğu kadar uzun süren bi işlemdir malumunuz. Bu konu ile ilgili bir yazım olacaktı.

Önce test yazacaksınız, sonra testi geçirecek kodu yazacaksınız. Başka bir test yazıp asıl işi yapacak kodu yazmaya zorlarsınız kedinizi. Sonra duplicationları gidermek için refactor edersiniz. Bunları yaparken naming convention a dikkat edersiniz.

Derken ne oldu? Hepi topu 3 ile 5 i toplayıp 8 döndüren bir method yazdınız daha. Bu methodu yazabilmek için en az 4 test yazmış olmamız lazım. Yani sadece basit bir method için 4 test yazıyoruz ki, büyükçe bir projede bu sayı nerelere çıkar varın siz düşünün.

TDD yazıyorsak en çok yazılan şey doğal olarak testtir. Farkettim ki bizim Visual Studio içinde bu durumla ilgili bir snippet yok. Ben her test için şunu kendim yazmak zorundayım:

[Test]
public void TestMethodTest()
{
    Assert.Fail("Not implemented yet!");
}

Halbuki bir code snippet olsa ve biz 1-2 tuş ile bunu şak diye yazmış sayılsak diye düşünerek bir snippet yazdım. İtiraf etmeliyim ki “prop” snippettını referans alarak yazdım.

İndirmek isterseniz buyrun.

Bu dosyayı “My Documents\Visual Studio 2008\Code Snippets\Visual C#\[İstediğiniz bir folder]\” altına atabilirisiz.

Test driven geliştirme yapmak, eğlenceli olduğu kadar uzun süren bi işlemdir malumunuz. Bu konu ile ilgili bir yazım olacaktı.

Önce test yazacaksınız, sonra testi geçirecek kodu yazacaksınız. Başka bir test yazıp asıl işi yapacak kodu yazmaya zorlarsınız kedinizi. Sonra duplicationları gidermek için refactor edersiniz. Bunları yaparken naming convention a dikkat edersiniz.

Derken ne oldu? Hepi topu 3 ile 5 i toplayıp 8 döndüren bir method yazdınız daha. Bu methodu yazabilmek için en az 4 test yazmış olmamız lazım. Yani sadece basit bir method için 4 test yazıyoruz ki, büyükçe bir projede bu sayı nerelere çıkar varın siz düşünün.

TDD yazıyorsak en çok yazılan şey doğal olarak testtir. Farkettim ki bizim Visual Studio içinde bu durumla ilgili bir snippet yok. Ben her test için şunu kendim yazmak zorundayım:

[Test]
public void TestMethodTest()
{
    Assert.Fail("Not implemented yet!");
}

Halbuki bir code snippet olsa ve biz 1-2 tuş ile bunu şak diye yazmış sayılsak diye düşünerek bir snippet yazdım. İtiraf etmeliyim ki “prop” snippettını referans alarak yazdım.

İndirmek isterseniz buyrun.

Bu dosyayı “My Documents\Visual Studio 2008\Code Snippets\Visual C#\[İstediğiniz bir folder]\” altına atabilirisiz.

BizTalk Custom Pipeline Components

Business Case:

A Sweden based ICT Company were facing a problem in archiving of invoice message (TIFF files). They had to do many manual processes for the archiving which required automation.

Requirement:

The requirement of the client was to create a custom pipeline component in BizTalk to support archiving of invoice message (TIFF files). The proposed solution has to interact over secure FTP (SFTP). The invoices have to be transferred to the second party and acknowledgement of the receipt is to be recorded. If there is any error in the process then it has to be notified by email. A strict requirement was that the invoice and accompanying files should be transferred as a single set. If there is any error then the sending of invoice should be cancelled and notified by email.

Decos Solution:

BizTalk Pipelines allow you to customize the processing of XML documents received or sent via the various BizTalk adapters. Custom Pipeline components extend the behavior of Pipelines to include processing data of virtually any format. They can be a powerful solution if you support legacy systems that require integration with other products, but your legacy data format does not follow standards. It may contain carriage returns in odd places or records spanning any number of lines of text, for example. The only stipulation is that the data emerging from the Custom Pipeline must be some form of XML document.

The client’s requirement was to build custom pipeline component in BizTalk to support archiving of invoice message. The invoices which were TIFF files were generated at the client’s side. First step taken by Decos Team was creation of windows services to interact with SFTP server at source and destination sides. And then the Enhancements to third-party nSoftware component were done to search/create directories recursively. To complete a Pipeline, one or more Pipeline components from within Visual Studio .NET were needed to be selected and then they can be added to the Pipeline. None of the existing BizTalk Pipeline components satisfied the client requirements, so Decos had to build a custom pipeline component. A custom pipeline component for archiving the invoices was created. After creating the custom Pipeline component, the pipeline was deployed to the C:\Program Files\Microsoft BizTalk Server 2006\Pipeline Components directory so the custom Pipeline component can be added to the Pipeline in the BizTalk 2006 project. After deploying the custom Pipeline component, the component was added to the Pipeline component toolbar. The logic in BizTalk Orchestrations was implemented to ensure that files are received as a single set and error conditions are detected and notified. Technologies being used for this solution: BizTalk Server 2006 R2, nSoftware SFTP adapter, SSH library

The Result:

Decos team delivered this solution in 20 days and is currently working on the development of additional features. Client was delighted to have an advanced solution based on Biztalk Server 2006 to get relief from their pain area of archiving the invoice messages.

GET A FREE QUOTE

more: www.decos.in

]]> http://computersciencesource.wordpress.com/2009/11/23/year-2-software-engineering-software-design-intro/ Mon, 23 Nov 2009 23:16:54 +0000 Shaun http://computersciencesource.wordpress.com/2009/11/23/year-2-software-engineering-software-design-intro/ http://computersciencesource.wordpress.com/2009/11/23/year-2-software-engineering-domain-modelling/ Mon, 23 Nov 2009 21:09:18 +0000 Shaun http://computersciencesource.wordpress.com/2009/11/23/year-2-software-engineering-domain-modelling/ http://kswenson.wordpress.com/2009/11/22/kanban-for-software-development/ Sun, 22 Nov 2009 21:05:16 +0000 kswenson http://kswenson.wordpress.com/2009/11/22/kanban-for-software-development/

Last Wednesday I got a full scale indoctrination into the agile software development methodology called Kanban, loosly based on the Toyota Production System (TPS) mechanism with the same name.  Toyota uses the kanban as a mechanism to allow for just the right amount of parts to be ordered and to be delivered just in time (JIT) in order to avoid overproduction and waste in the production line.  Kanban Software Development Methodology (KSDM) brings the same lean ideas to a development team.

The Kanban track at QCon, the SD conference in San Francisco, had a line of speakers coordinated to give a thorough explanation: David Anderson, Jeff Patton, Henrik Kniberg, Chris Shinkle, and David Laribee.  What I found impressed me as an approach that may answer some of the problems I have been having in moving teams to agile development.

My first thought was that it should not be called “Kanban” since there is no kanban card: neither physical nor virtual nor electronic.  Instead the focus is on “leveling” (also known in TPS as “heijunka”) which is a way of keeping the amount of work in each category steady.  Kanban is the mechanism in TPS for achieving this leveling, but in KSDM achieves this leveling without a Kanban token itself.  I won’t dwell any more on the name, because it is in widespread use, and the ideas attached to the name as so important.

Resources

Being new to the topic, I refer you right away to resources with a more thorough treatment of the subject:

• LimitedWIPSociety – a focal point for this philosophy.  “WIP” means “Work In Progress”.
• The kanbandev group at Yahoo has discussions on these topics.
• Kanban Applied to Software Development: from Agile to Lean – a good article from Kenji Hiranabe at InfoQ that lays things out in good detail.
• The Kanban Primer: A Cultural Evolution in Software – a good overview article from David Anderson
• Taiichi Ohno Reinterpreted – my post from a few weeks ago visiting the fundamentals of Toyota Production System

My goal will be to simply cover my impression of what is different or unique about this approach to agile software development — there is a lot more to it than I can cover here.

Key Concepts

The key to any Lean (with a capital L) method is to eliminate waste.  In software, work that is partially complete is waste.  This partially complete work is known as “Work In Progress” or WIP.  There is a necessity to have a certain amount of WIP, but the point is to minimize that to the degree possible.  You minimize WIP, and as a result that work gets accomplished quickly.  It only stands to reason: with a fixed amount of workers, reducing the number of things being worked on will allow them to spend more time on those fewer things, and get them to a completed state more quickly.

Let me  emphasize how important this is.  Development projects that go for 8 months with no visible results are dangerous on many levels.  By taking such a big bite of work at once, you commit the entire department to a direction without knowing whether it is going to work.  There is no way to gauge the progress of the team, except in vague status measures which can be manipulated by incompetent players to hide reality.   KSDM is about nibbling away at the work.  Each small unit is completed and made customer ready BEFORE the next unit is started.  In my reading of Taiichi Ohno, this production in small continuous batches, instead of huge batches, is the essential ingredient of Just In Time manufacturing.

The focus is on “flow” of development activities. Focus on flow does two thing: first it obviously help to continuously get things completed.  The other thing is that it helps to expose trouble.  Whenever there is a disruption in the flow, when the flow does not work quite right, it is an indication of a problem that needs addressing.  If works starts to pile up at one step, this is an indication that you should focus effort on that step and find out what is not working well.

How it Works

This seems in many ways contrary to the punctuated approach emphasized by Scrum and other strict iterative development.  This apparent difference is an illusion caused simply by scale.  KSDM allows for finer grained control.  Scrum is a fairly course grained approach, e.g. all work gets done in a three week cycle.  If you look at the larger scale, over the year, Scrum is providing a steady flow of features into the product.  The 3 week iteration is a mechanism to assure that there is not a huge huge backlog of incomplete work.  All the work, of the complete development cycle must be started and finished in that one sprint, which can be difficult if you have people that specialize in certain phases of development.

KSDM takes this control to a different level.  You break the work process in a series different activities (phases).  You then set a limit of how many job units you will allow at any phase.  A simple rule of thumb: you can have a few more work units as you have people doing that job, so that each has one thing to work on at a time, and a small cache of completed jobs.  The people in a given phase will do their work on a job unit to completion, so that it is ready for the next phase of work.

This is where the somewhat brilliant key idea behind KSDM comes to play.  The completed job does NOT free the person for working on another job, until that job is pulled into the following phase and work is started there.  If work is piling up at a particular phase, those people are NOT ALLOWED to work ahead.  As Taiichi Ohno makes so clear, that working ahead is waste.  Instead of working ahead, they can look around to see what is wrong.

“I feel a disturbance in the flow…”   – Obi Wan Kenobi on software development

To put this in concrete terms, consider a process which involves (1) detailed design, (2) coding, (3) testing, and (4) documenting.  Each of these stages you place a limit on the number of jobs, and for the sake of example lets say that limit is 4.  Say for example that the coders have finished coding on their four job units, and are ready to take a new one.  But the testing is backed up for some reason, still working on the last four job units, and are not ready to take a new job.  The developers are not allowed to pull in a fifth job unit.  There is no point in coding up more features when the earlier features are not getting tested or documented.  It is also possible that because the developers are not pulling jobs from design, that the design phase becomes filled up with completed tasks.  When work backs up in this way, one should go and figure out why testing is stuck.  Maybe the real problem is that the documentation is blocking test.  Whatever it is, the primary job of the entire team is to identify the problem with the flow, and fix it.  Do not simply work ahead accumulating a huge pile of work for “someone else”.  Instead, focus on the big goal, which is to get features completed to a customer ready state as quickly as possible.

Reflection

This idea of setting limits on the number of discrete jobs that can exist at any given phase at a time is so simple, and yet so profound, that I want to format these paragraph all in bold italics. It allows people to specialize into different role, to focus on their particular work, while the mechanism assures that the primary goal is not being lost.  Some teams can take a feature (a story actually, a small separable part of a feature) from design to completely implemented, tested and document in only four days.

Kanban works in a car factory because the time to complete a particular job is well known.  A Corolla comes off the line every 97 seconds.  The factory is set up to produce a car’s worth of parts every 97 seconds as well.  The parts are highly repeatable, so if it takes more than 97 seconds to build four doors, then you need either multiple steps, or parallelism, whatever makes the most sense.  Once you have figured out exactly how long it takes to build a door it can be done again and again.  Software is not so repeatable.  Every software job is different and unique.  How does that work with Kanban?  The interesting thing: it does not turn out to matter.  You break features into stories that are approximately the same size, but if one story takes three times the effort as another story, no problem.  With software the job does not need to be timed to fit into 97 second slots.  We can be flexible in the time dimension: one story takes one day, and another takes three days.  What is important is that that story, no matter how long it takes, is completed before another is pulled in by that resource.  With this understanding, my biggest concern was eliminated.

Clearly the method requires that features be broken down into fine grained stories.  If you did not do this, you run the risk of “starving the line”.  That is, one phase is filled with long running jobs, and the following phase has completed all their work.  One area to explore is when the feature designer claims that the feature can not be broken down into small stories.  There are surely techniques to address this, including job classification and prioritization, minimum marketable features, and other details which I can’t cover in this post.

Finally, it appears to me that a teams with a long practice of waterfall development might be able to evolve into this method.  There is no big disruptive change. Just make the process visible, and limit the amount of work that is in progress at any given time.  This allows Taiichi Ohno’s concept of  “autonomism” to take over, and team can self organize to become more efficient.   This make a lot of sense.

Are you looking for a good book on Architecture? Then, Microsoft Application Architecture Guide, 2nd Edition is a ‘must-read’. Though it is tuned to working with Microsoft technologies, much of topics covered have larger relevance. What I like about this book, compared to other books on software architecture, is practical relevance and ease of application

Hats off to Microsoft for bringing architecture from the elite glass house to developers’ workspace!

As part of my last year in computer science studies, my master’s thesis is concerned with the generating medical documents from models of clinical pathways. A clinical pathways is defined as a list of task to do allowing to guide the care of patient affected by a pathology. It can be represented by graph with boxes (the tasks) link together by arrows defining the sequence in which the tasks happened.

From this clinical pathways, some information will be extract to supply a medical document bound to different kind of stakeholders (patient, doctor, physician, hospital attendant, …). The specificity of this generating process is that the extracted information will be fitted to each class of stakeholder. So a lot of complex filtration mechanisms will be defined to do this job. In order to closely specify these mechanisms, we need to prepare and practice some interviews of the stakeholders. For this master’s thesis these interviews will be practice in cooperation with the service of radiotherapy of St Luc (UCL) and the general hospital of Peltzer (Verviers) .

The main part of the work will consist in specification and designing of such a generator.

Carnegie Mellon’s Software Engineering Institute offers free membership to military personnel. Discounts are given for courses taken by active members.

http://www.sei.cmu.edu/membership/benefits/