Dando continuacion al post anterior, seguiremos con los gestores de descarga para GNU/Linux:

Uget: Dejando a un lado el diseño de la pagina del proyecto (mis ojos), Uget antes urlgfe es un cliente de descarga que ademas de contar con las características de otros gestores, permite clasificar las descargas en categorías, permitir hacer batch de descargas, importar archivos desde texto html. Pueden descargar el programa desde aca, para Ubuntu hay un paquete deb.

wxDownload: Otro programa en wxWidgets y C++, por lo que permite ser multiplataforma. Las características son ser multilenguaje, soporte para ftp y http, normal y a través de proxies, poder descargar enlaces metalink, ordenar las descargas, y calcular el MD5 y SHA1 de los archivos. Podemos descargar el codigo fuente del sitio, y compilarlo para nuestra distro. Para ubuntu y debian se encuentra paquetes deb (inclusive para la arquitectura 64 bits).

Aria2: Este programa manejado a traves de la linea de comandos es uno de los pocos gestores que te permiten descargar a través de bittorrent ademas claro a través de http, ftp, https y metalink. Caracteristica aparte es que si tienes multiples descargas te permite tenerlas al mismo tiempo. El código puede descargarse de este lugar, aunque la pagina nos muestra binarios para diferentes distros (como Ubuntu)

Jigdo: AKA jigsaw downloader, en si este programa se denomina como utileria para la descargar de imágenes de disco CD y DVD o archivos muy pesados. Este programa esta inspirado en los problemas que tienen los usuarios de Debían en la distribución de Cds. Lo que hace este programa es convertir la dirección de descarga en un archivo en extension .jigdo, seguido de esto al descargarse con la herramienta jigdo-lite este busca el mismo archivo en otros servidores y lo parte en pedazos muy pequeños, lo que permite descargar cada archivo individual. Podemos encontrar el codigo fuente desde la pagina, para Debian y Ubuntu se encuentra en los repositorios.

Bueno aquí este pequeño recopilatorio. Espero que algunos de ellos le sirvan.

Sayounara

01 Desember 2008, saya mau download ISO DVD Linux Fedora 10 yang baru seminggu direlease. Karena connect ke speedy (unlimited) maka server download paling cepat (setahu saya) adalah http://dl2.foss-id.web.id/. Ternyata tidak tersedia file .iso di sana, untuk DVD tersedia file berekstensi .jigdo (padahal pengennya yang torrent).

http://dl2.foss-id.web.id/fedora/releases/10/Fedora/i386/jigdo/Fedora-10-i386-DVD.jigdo

Akhirnya tanya mbah google, apa sih itu…dan inilah rangkumannya.

Jigdo Download Manager memungkinkan kita mendownload ISO Fedora dan lainnya, termasuk file ISO berukuran besar. Tool ini mendownload semua file penyusun image kemudian menggunakannnya untuk membentuk ISO berdasarkan pada template pre-created. Tulisan ini menjelaskan cara mendownload dan membangun image ISO Fedora menggunakan Jigdo Download Manager.

Jigdo: Download dan bentuk file image, bahkan untuk berukuran besar

Apa yang diuraikan di sini dapat diterapkan untuk Fedora versi 6.0 ke atas. Jika menggunakan Windows maka silakan download versi win32 (zip) dan jalankan file jigdo-lite.bat.

Apa yang diperlukan?

1. Jigdo Download Manager, silakan download di: http://atterer.net/jigdo
2. File .jigdo yang akan didownload, misalnya http://dl2.foss-id.web.id/fedora/releases/10/Fedora/i386/jigdo/Fedora-10-i386-DVD.jigdo
3. Jika memanfaatkan repository lokal, pastikan mengacu ke file .jigdo pada lokasi yang tepat. Periksa URI file pada bagian [path] untuk memastikan anda telah memasukkan path dengan benar.
4. Sumber data http://, ftp:// atau file:// yang akan digunakan.

Saatnya mulai…

Pertama, tentu dimulai dengan menginstal Jigdo Download Manager yang telah didownload. Instruksi di bawah ini digunaan untuk menginstal jigdo melalui yum pada salah satu versi Fedora.

Menginstal Jigdo

1. Buka jendela terminal, maksudnya masuk ke console gitu loh.
2. Instal Jigdo Download Manager:

su -c ‘yum install jigdo’

3. Buat suatu working directory untuk Jigdo, misalnya (ini hanya misal, contoh):

mkdir ~/jigdo

Menggunakan Mirror Publik

Jika mendownload image Fedora, maka image jigdo default diset untuk mendownload dari public mirror Fedora. Lokasi download default akan secara random mengirimkan kita ke mirror berbeda yang terdekat dan mempunyai file-file yang perlu didownload. Tentu kita boleh menentukan mirror tertentu yang akan dijadikan sebagai sumber file.

Penentuan Mirror (kalo mau)

1. Silakan pindah ke working directory jigdo:

cd ~/jigdo

2. Jalankan jigdo-lite, sertakan nama file .jigdo:

jigdo-lite http://example.org/example-fedora-iso.jigdo

3. Ulangi langkah-langkah ini untuk setiap image .jigdo yang akan didownload, satu demi satu.

Mengakses Repository Lokal atau Khusus

Jika mendownload image jigdo dari sumber lokal untuk merangkai image maka harus melakukan perubahan terhadap file .jigdo (file konfigurasi download image) agar mengacu ke sumber (lokal) yang diinginkan. Sumber data dapat berasal dari File, HTTP dan FTP.

Pemilihan Mirror Manual

1. Pindah ke working directory jigdo:

cd ~/jigdo

2. Download file .jigdo:

wget http://example.org/example-fedora-iso.jigdo

3. Gunakan editor favorit dan edit file .jigdo. Pada bagian [Server], lakukan perubahan pada baris Core dan Updates, arahkan ke sumber data lokal atau mirror yang diinginkan:

Awalnya, misalnya:

[Servers]
Core=http://example.org/fedora/releases/7/Everything/i386/
Updates=http://example.org/fedora/updates/7/i386/

Diubah menjadi (contoh):

[Servers]
Core=file://mnt/repomirror/fedora/releases/7/Everything/i386/
Updates=file://mnt/repomirror/fedora/updates/7/i386/

Itu anjuran dari http://fedorasolved.org, namun saya mengubah baris-baris pada session [Image] dan [Servers] pada baris berawalan Base (bukan Core dan Update…yang ada hanya Base di bawah [Servers])…sehingga dapat mendownload file-file dari server foss-id.web:

[Image]
Filename=Fedora-10-i386-DVD.iso
Template=http://dl2.foss-id.web.id/fedora/releases/10/Fedora/i386/jigdo/Fedora-10-i386-DVD.template
Template-MD5Sum=m4zEizzSZykB9F9gcH8jng

[Servers]
Base-i386=http://dl2.foss-id.web.id/fedora/releases/10/Fedora/i386/os/
….

4. Jalankan jigdo-lite, umpankan file .jigdo yang tekah diedit tadi:

jigdo-lite example-fedora-iso.jigdo

5. Ulangi langkah-langkah ini untuk setiap image jigdo yang ingin didownload.

Memanfaatkan Data yang telah didownload

Cara paling mudah dan nyaman menggunakan jigdo adalah mem-”patch” ISO yang telah ada – telah didownload sebelumnya. Ini akan mengurangi jumlah data yang diperlukan selama download. Langkah pemanfaatan ISO yang telah didownload diterangkan di sini.

Jika berbentuk file ISO:

1. Mount file imagetersebut menggunakan perangkat loop:

mount -t iso9660 -o loop /path/to/oldimage.iso /path/to/mount/point/

2. Beritahu jigdo-lite dimana file ISO tersebut dimount, saat ditanyakan. Anda dapat pula melewatkan opsi –scan ke suatu direktori untuk menscan file-file di dalamnya.

Jika image sudah ditulis ke CD atau DVD?

1. Masukkan media (ya ke drive-nya lah), ingat kemana media tersebut dimount. Pada Fedora biasanya di bawah direktori /media/.
2. Beritahu jigdo-lite dimana media dimount, seperti pada cara sebelumnya.

—demikian duloe ya—
Lebih detail mengenai jigdo, silakan menuju: http://atterer.net/jigdo

NOTA: Esta es una experiencia UNICAMENTE orientada para aquellos que tenemos instalado Intrepid Ibex RC desde el Alternate CD o tenemos la ISO descargada. Si bien está declarada como una manera eficiente de tener la release final del 8.10 (el proximo 30 de octubre), no se recomienda para aquellos que no esperen ALGUN ERROR, CRASH o no puedan soportar el hecho de reinstalar el sistema nuevamente. Aquellos que no se sientan en condiciones, abstenerse de usar este método, y obtengan la ISO de Intrepid Ibex final, de las maneras que solían hacerlo. (torrent, descarga directa, net update).

Comenzamos?..

Jigdo (JIGsaw-DOwnload) es un programa que puede ser usado para descargar eficientemente builds o actualizaciones de una ISO existente sin la necesidad de re-descargar el CD entero, con cada nuevo cambio (Actualización Incremental). Simplemente, verifica por cambios o actualizaciones entre una imagen de CD existente (en tu PC) y la imagen más reciente lanzada (en algún servidor). Este proceso solo será funcional para actualizaciones de un mismo lanzamiento, como sucede entre la RC de 8.10 y la versión final del 30 de Octubre.
Para empezar, instalamos JigDo:

sudo apt-get install jigdo-file

Acto seguido, y contando con la ISO de 8.10 RC ya descargada, iniciamos el proceso (el 30 de octubre, una vez sepas que ya está libre la versión final).

jigdo-lite

El sistema ahora va a solicitar la URL del archivo .jigdo necesario para iniciar la descarga. Este .jigdo deberá ser obtenido de la lista de descargas disponibles el 30 de octubre, en el sitio oficial de descargas de Ubuntu. Ingresen alli para encontrarlo. Copienlo y peguenlo en la consola, cuando se les pida.

To resume a half-finished download, enter name of .jigdo file.
To start a new download, enter URL of .jigdo file. <<<
You can also enter several URLs/filenames, separated with spaces,
or enumerate in {}, e.g. `http://server/cd-{1_NONUS,2,3}.jigdo'
jigdo:

Mientras escribo, el archivo es:
http://releases.ubuntu.com/releases/intrepid/ubuntu-8.10-rc-alternate-i386.jigdo (Alternate CD). Seguramente, este puede cambiar el día del lanzamiento final.
Una vez ingresada la URL del .jigdo, el programa solicitará la ISO actual en tu PC, ya sea montándola en alguna carpeta o desde el cd quemado, indicando la dirección al dispositivo.

If you already have a previous version of the CD you are
downloading, jigdo can re-use files on the old CD that are also
present in the new image, and you do not need to download them
again. Mount the old CD ROM and enter the path it is mounted under
(e.g. `/mnt/cdrom').
Alternatively, just press enter if you want to start downloading
the remaining files.

Para montar el .ISO, debes hacer:

sudo mkdir /media/intrepid
sudo mount -o loop ruta/a/intrepid-alternate-i386.iso /media/intrepid

Jigdo, ahorá comparará la lista de paquetes y sus versiones, en el CD de tu pc, y en la lista dentro del servidor suministrado. Verá cuales paquetes debe descargar e instalar y cuales puede extraer del CD, ya que son los mismos alojados en el servidor.

Found xxxx of the xxxx files required by the template
Copied input files to temporary file `intrepid-alternate-i386.iso.tmp' - repeat command and supply more files to continue

Continúa el proceso presionando 2 veces ENTER. Jigdo continuará descargando todos los paquetes actualizados necesarios. Finalmente, obtandrás una ISO actualizada, similar a la alojada en el servidor.
De esta manera, se ahorrará significativamente la descarga de paquetes redundantes y de alguna manera, liberando también el ancho de banda del servidor de Ubuntu, el cual SIN DUDA estará muy concurrido!!!

De todas maneras, puedes usar los TORRENTS, el próximo 30…

Visto en Ubuntu Tutorials y en Jigdo How-To.

With every passing day more and more stuff is available to download. Average download rate per day has also increased drastically since broad band is made available to more and more users.  I remember I was using download accelerators to download anything more then couple of Mega Bytes on my dial up connection.

Have you ever tried or thought about downloading a CD image of approximately 600 – 700 MB 5 years back? Most probably not, right? But now forget about CD Images we can easily download DVD images of 4.7 GB within couple of days even with slowest broad band connection. Now a days almost everyone uses torrents and P2P clients to download huge files. This helps you to get your files faster and reduce the load on the servers as more then one computer host full/partial files.

Till now we were only looking at it from end user point of view, as an end user we want to download the files as soon as possible. But what about the servers on which these files are hosted? No wonder we have hard disks available in tunes of Tera Bytes now. Just look at the number of videos uploaded to You Tube everyday, or constantly increasing size of the inbox, or average size of the software’s distributed. Everything requires lot of storage space and that results in constant upgrades to server storage.

For commercial sites it should not be a problem to upgrade the infrastructures because they get some money for there product, services or advertisements they offer. Unfortunately our open source/free software friends does not have that flexibility as they are dependent on volunteers and donations to host their contents.

By this time you might have forgotten the purpose of this topic as it doesn’t contain a word about Jigdo. You are right but that was essential to understand and appreciate this tool. In this post I will stick to very basic information about Jigdo as all the details are explained in very clear and easy manner on Jigdo Website.

What is Jigdo:

In short Jigdo is a tool to create ISO images from set of files, this will help you to remove the actual ISO images from the server as the ISO image will be reconstructed on the user’s computer by downloading individual files present in the ISO image. This tool is developed by Richard Atterer.

Lets us try to understand this with an example:

Say you want to distribute you Linux destro, your complete destro consist of files of 1GB. Lets say you have four variants Complete, Desktop, Server and Minimum. Obviously these variants will be created using the subset of your overall 1GB data.

If you want to host the individual files and one ISO image each for each variant, total space requirement on the the server would be:

This is nothing but duplication of data on the server, all files are stored multiple times, once as a individual file and one or more times as part of ISO images. Jigdo helps you to prevent this by creating the ISO images from individual files so you no longer need to store the ISO images on the server.

How Jigdo Works:

Lets continue from our example, you will create an ISO images from the individual files stored on the server and give this ISO file to Jigdo. Jigdo will create two files for you, first is .jigdo file and second is .template file.

.template file contains metadata about the ISO image i.e. .template file contains everything else in ISO image except actual files. this includes information about where the file offsets are in an ISO image, there checksum, directories etc. As this file contains binary data you will not be able to understand its content using text editor.

.jigdo file contains links to the actual files on the server along with there md5 checksum, this will be a gzipped text file that can be viewed using zcat.

Now instead of ISO image you will provide .jigdo and .template files to the end user. End user provides this file to jigdo. Jigdo will download each file separately using wget and place it in the local ISO file by using the information in the .template file. Thus jigdo will be able to reconstruct the bit by bit copy of the original ISO image.

How Jigdo Helps End User:

There are many advantages for end users

1. Jigdo allows you to download individual files through different mirrors, so if you find that your current mirror is slow you can switch to different one, If you do this with ISO image hosted on server you need to download the whole thing again.
2. If you already have a CD/DVD image of particular destro, Jigdo can use unchanged files from this CD/DVD rather than downloading them again, this will be very useful if you have beta version and want to download next beta or final version as not many file will change between these versions.
3. You can stop and re-start the download process anytime
4. Your download will be comparatively faster
5. If you want to create your own distribution, you need to host only changed and new files again, rest of the files will be taken from the original server.
6. Jigdo is file format independent you can host any big file using Jigdo on multiple servers
7. Last but not least is you will help the open source/free software community to save some money on hosting services

Please go through the links below for details about Jigdo and using Jigdo.

Who is using Jigdo:

• Debian
• Fedora Core
• Ubuntu Beta

Links Related to This Post:

• Richard Atterer
• Jigdo Website.
• Jigdo How-To
• Jigdo FAQ

I needed to modify a package supplied by Fedora 9 DVD and make a DVD of my own. Creating a new iso Image out of source files is called remastering.

Now let me explain how can you do it too!

1. Modifying the required package.

Install the Source Rpm (SRPM) of the required package using yum-install. Once you do that, the Source Packages will be installed in /usr/src/redhat. There you’ll see folders such as SOURCES, SPECS, BUILD, etc.

Now, go to the source folder. You’ll see a lot of configuration files, a Source tar ball etc. Make whatever change you want. I’ll wait.

Now, once that is done, go to the SPECS folder. You’ll find a file called <package_name>.spec. That is the RPM specification file which describes how the RPM should be built. Build binaries by using rpmbuild -bb <spec_file> ( -bb stands for build binary). This should build the RPM file. If everything goes well, you’see a  new shiny RPM file waiting for you.

2. Build the DVD iso image

Now, once the RPM file is ready, lets include it in our own remastered ISO. Its not at all simple. You have to make sure you get the bootrecord burnt properly. This requires that you remember numerous lengthy parameters to pass to mkisofs and genisoimage. Scary right ? Not so. Enter jigdo. If you looked at the link, I’m sure you’d be thinking ‘How the heck is a download manager going to help me create an ISO image. Crazy ?’. The same thing came to my mind when I came across jigdo for the first time. (If you don’t have it, go ahead and do a yum install )

Now, the interesting part – using jigdo to create our ISO image. Mount the original ISO through a loopback device. Copy all the contents into a separate directory. Find the directory called Packages and copy our newly created RPM into it overwriting the existing one. You need to get a jigdo file for your distro first. Fedora users, here’s one for a Fedora 9 DVD.

Now run a command called jigdo-lite. It will ask you a couple of questions on where your jigdo file is and where your DVD package directory is. Once you are done with this, wait and watch as jigdo-lite magically creates a new Fedora DVD ISO image! You need to have a working Internet connection active as jigdo-lite will attempt to download a Template File for the DVD ISO image.

It was fun! Wasn’t it ?

Yesterday Fedora 9 was announced. Using Fedora can give you a look at what may be in Red Hat Enterprise Linux down the road – and give you an exciting Linux distribution to boot.

There are a number of new exciting features to be found in Fedora 9. First, everything is updated to the latest versions, including GNOME 2.22, KDE 4.0.3, and Xfce 4.4.2.

Fedora 9 introduces the new filesystem ext4 as an option. While ext4 remains an experimental filesystem, it may be good to try it out. Like ext3, it remains compatible in both directions (an ext4 filesystem can be mounted as ext3, and vice versa).

Fedora 9 also replaces the System V initd process with an event-based replacement, upstart. Upstart was created and developed for Ubuntu Linux, and has spread to Fedora and Debian. Each process is started through a response to an event, and each process may generate another event.

Fedora 9 has several different spins or variations based on different sets of packages. For example, there could be a KDE spin, a GNOME spin, and a Xfce spin for example. The Fedora project has a page tracking spins for those who might be interested in custom spins.

This version of Fedora introduces support for Jigdo, which is a CD distribution mechanism that the Debian project has used for years. I’ve not used Jigdo, but the description given in the release notes suggests a large speedup if you have most of the data already.

It sounds like a very exciting distribution; I’ll be looking around my electronic wasteland to see where to install it.

I first became interested in using jigdo (Jigsaw Download) during the test releases for Fedora 8. I was spending a lot of time getting the DVD ISOs over bittorrent and was annoyed that in all probability I already had packages necessary to compose the release, but not in its official form from the rawhide trees I mirror locally.

Jigdo has the ability to start with a base image and build ISOs based on local (or remote) packages. I gave it try with some of the Fedora Unity Respins, helped test one of the Fedora 8 respins, and recently built the Fedora 9 Alpha ISO myself using jigdo.

There has been noticeably more interest in jigdo on the fedora-list@redhat.com mailing list over the past few months. This is interesting considering how summarily dismissed jigdo has been as “not being needed” by some of the regulars on the fedora-devel-list. The first time I heard jigdo mentioned was at the Red Hat Summit 2007 during the Revisor session. When I asked some of the Fedora veterans why Fedora didn’t use it I was told it was unreliable, full of memory leaks, and “that old thing from Debian”.

Is this the same disconnect between the folks who think CD releases of Fedora don’t matter and those that do? A classic “developer doesn’t think user wants or needs what the user says they need”? CD releases will be back for Fedora 9 and I think one of the reasons was the complaints from users on fedora-list.

The other arguments I’ve heard against doing jigdo is that it would not benefit enough users or would cause confusion by adding “yet another method to obtain Fedora”. I don’t find any these arguments strong enough to believe Fedora should not provide jigdo as a download option.

Looking at the Fedora Unity’s jigdo statistics people are using jigdo to get Fedora. How many more people could we get Fedora to if it was a mainstream option and how big is the risk if we try?

Speaking from personal experience I think providing direct jidgo support by Fedora would be a great thing. I have never had problems with jidgo or problems with the ISOs it creates. It provides a good alternative for people who cannot use bittorent or have limits on how much bandwidth they can use in a given period.

Who knows, maybe it will become a reality: http://fedoraproject.org/wiki/Features/JigdoRelease

Kumar, a dedicated Debian dude has written following and I’m sponsoring the first ever Sponsored Post to my blog:

Thanks to Kartik Mistry for `sponsoring’ this blog post. I wanted him to do this because I don’t blog, and his blog has wide reception on Planet India FLOSS and Planet Debian. So, here goes!

Jigdo is a wonderful way of constructing CD/DVD ISOs by dowloading smaller units (like deb packages) to construct an image on your machine, rather than download a 4.3 GB monster over internet only to realize that the connection stops half way or some such thing.

I would also take this opportunity to publicize IIT Madras’ mirror, which mirrors quite a bit of free software. It has a (somewhat) offical Debian (x86, amd64) mirror at

http://ftp.iitm.ac.in/debian
ftp://ftp.iitm.ac.in/debian

And am x86 mirror for Ubuntu as well:
http://ftp.iitm.ac.in/ubuntu
ftp://ftp.iitm.ac.in/ubuntu

It is also (most of the time) ftp.in.kernel.org and www.in.kernel.org, since it is an official Linux kernel mirror (the only one I know of in India).

Anyway, let’s jigdo now! We are to prepare the DVD for Etch 4.0r1 (i386) using jigdo. Here are the steps:

0. Get Jigdo. You can get the “lite” version for any GNU/Linux distribution or Windows here:
http://atterer.net/jigdo/

But Debian/Ubuntu users just need to apt-get install jigdo-lite.

1. Visit http://cdimage.debian.org and go to the “Download CD/DVD images with Jigdo” section.

2. I need the official i386 “stable” DVD image for i386, so I click that. That takes me to a page with a directory listing with many “.jigdo” files and “.template” files. While the jigdo file is a gzipped and formatted list of files in the CD/DVD, the template file is one with a description of the image itself. See TechDetails.txt in the jugdo distribution (or /usr/share/doc/jigdo-file) for details.

3. I download debian-40r1-i386-DVD-1.jigdo and debian-40r1-i386-DVD-1.template to a directory on my computer, and run this:

jigdo-lite debian-40r1-i386-DVD-1.jigdo

4. First, I am asked for files to scan. Since I don’t have any files on my disk already (or don’t want to use them), I just press Return.

5. I am then asked for a mirror. I specify the fastest mirror available:
ftp://ftp.iitm.ac.in/debian

And that’s it. It is supposed to write a blank image and start populating it with Debian packages! Simple, huh?

Depending on the speed of your connection, it should take a few hours (or days) to get a DVD image done. A CD would be much quicker. For people inside IIT Madras, we get 5-7 MBps to the mirror, so it takes around half an hour for the DVD image. So, now you know whom to ask for those Debian DVDs and CDs, if you’re near Chennai, right?

Troubleshooting suggestions are available on the Debian CD-Image
website here: http://www.debian.org/CD/jigdo-cd/

So, hope this was useful.

Kumar Appaiah