Fugu: A very good open source (Free BSD) GUI tool to do file transfer securely. It makes life easy when you do some one time things over doing it via command line.

You don’t really need this if you are very comfortable with the OS/X built-in command line tool: sftp.

Vi höll ju en omröstning om de bästa gratisprogrammen och nu är det dags att börja presentera tio i topp. Givetvis är det här en högst ovetenskaplig lista och jag räknar med att ta upp det här temat igen, så de som inte fick med sina favoriter på listan eller ens hann med att nominera behöver inte hänga läpp. Nog snackat!

Tiondeplatsen tog FileZilla, som är ett filöverföringsprogram som hanterar FTP, SFTP och FTPS. Programmet finns i Ubuntus programcentral, men finns också till Windows och Mac.

Min bedömning är att det här är ett bra alternativ för er som behöver ett filöverföringsprogram, till exempel om man har en webbsajt som man måste ladda upp filer till, men det är inte ett program som alla behöver.

Du hittar FileZilla på http://filezilla-project.org/

Läs även andra bloggares åsikter om Gratisprogram, fri mjukvara, öppen källkod, filöverföring, ftp, sftp, ftps, ladda upp, ladda ner

Ubuntu Server 9.10 kurmus olmamdan henuz 1 hafta bile gecmemis olmasina ragmen bu sure zarfinda sisteme 3 kez format attim, Mac’e VMware uzerinde Ubuntu kurdum, CentOS kurulumu denedim, Ubuntu’yu iki kez daha kurdum. Anlasilan FTP sorunu bir dizi problemin sadece bir baslangiciymis!

Internet en buyuk yardimcim oldu bu surecte ve sonunda mukemmele yakin calisan bir sisteme kavustum. 1 gun boyunca test server’i uzerinden yayin dahi yaptim. Iste simdilik karsilastigim sorunlar ve workaround’lari:

→ Apache Virtual Host Problemi

Ubuntu’yu toplamda 3 kez kurmamin ana nedeni aslinda buydu. Bircok web sitesini ayni server’da tek IP uzerinde barindirdigim icin Apache’nin virtual host ozelligini kullanmam gerekti. IIS’te bunun icin ek hicbir islem yapmama gerek yokken Apache icin bir konfigurasyonlar dizisini modifiye etmem gerekiyordu. Ancak internetten buldugum ve Ubuntu Server 9.10 icin olan hicbir tutorial isime yaramadi. Her defasinda Apache coktu ve her seye bastan baslamam gerekti.

En son kurulumumda Webmin modulunu yukledim ve tum islemleri, kurulan bir web arayuzu araciligi ile yaptim. Baslarda bu bile sorunlara neden oldu ki ornegin bircok virtual host tanimlamama ragmen a.com b.com c.com, adrese ne yazarsaniz yazin hep ayni siteye yonlendiriyordu Apache kendiliginden. Bunun ise basit bir Port hatasi oldugunu gordum, Virtual Host yaratirken Port belirtmediginiz taktirde Apache en son kurulan virtual host’a yonlendiriyor adres ne olursa olsun. Sanirim bir tur Bug.

Linux icin olan bircok FTP programini denedim, bunlara VSFTPD, SFTP, VFTPD ve PURE-FTPD dahil. Pure-FTPD haric hepsinde yine Virtual User yaratma sorunu ile karsilastim. Ardindan Pure-FTPD haric hepsinde kullanici yaratabildim ancak bu sefer de login bilgilerini degistiremedim.

Pure-FTPD kurulumu sorunsuz gecti, istedigim kadar user olusturup kisisel ayarlarini degistirebiliyorum. Yalniz dun aksam 2 saatligine anlam veremedigim bir sorunla karsilastim: Gerek network icerisinden gerekse disaridan baglanti kuramiyordum. 425, Sorry Wrong Address hatasi aliyordum devamli. Gizemli bir sekilde birkac kez remove & reinstall isleminden sonra bu sorun da cozuldu.

En onemli problemlerden birisi: PHP ile mail gondermek. IIS’te bir turlu ustesinden gelemedigim problemdi. Ne yaparsam yapayim, ne kendi server’imdan ne de baska bir SMTP server‘dan email gonderemiyordum. Linux’ta yine boyle bir sorunla karsilasmaktan korkuyordum. Ancak PHP PEAR ile Google Apps‘deki hesabimdan SMTP ile mail gondermeyi basardim. Local olarak kendi SMTP server’imdan gondermeye vakit ayirir miyim bilmiyorum, onceki deneyimlerimden dolayi IP adresimin whitelist‘te olmamasindan dolayi bircok mailbox’a SPAM olarak gittigini biliyorum.

This is just a brief overview. The installation process is pretty easy on these. Same as with most patches, IBM UpdateInstaller “update.sh” was used to install the service “pak” files. These patches must be done in order. Patch the WAS installation before patching WPG.

All WebSphere services must be stopped to install the WAS updates. On a standard installation, bcguser must be used to stop the service

[bcguser@WPGhost ~]$/opt/IBM/bcghub-simple/bin/./bcgStopServer.sh

We do not use ‘/opt’ for our WebSphere location, so change this if yours is different.

Next, use Update Installer to patch the WebSphere Application Server
[user@WPGhost ~]$ sudo /opt/IBM/WebSphere/UpdateInstaller/./update.sh

There is a gotcha here that had me “chasing my tail” for about 10 minutes. When going to install the WebSphere Partner Gateway fix pack, the Partner Gateway and WAS server must be started. Installation of the update will fail with error “user input validation”.

So before installing the WPG update, re-launch the Application Server and Partner Gateway

[bcguser@WPGhost ~]$ /opt/IBM/bcghub-simple/bin/./bcgStartServer.sh

Once done, launch IBM Update Installer again, passing the customized responce file for your environment. This needs to be executed as the root user, so sudo was used to allow xforwarding from a non-root account

[user@WPGhost ~]$ sudo /opt/IBM/WebSphere/UpdateInstaller/./update.sh -options /opt/IBM/bcghub-simple/responsefiles/bcgupdate_en_US.txt

Those are my miscellaneous notes about the update installation. Everything went fine here and I hope this fixes some of the SFTP issues we have been having.
This brings the WebSphere Partner Gateway Console form version 6.2.0.0.273 to 6.2.0.1.333

Notes: Here is the link to IBM’s website that lists the fixes that are provided in the update.

Voici un des moyens d’automatiser le téléchargement avec FTP,SFTP ou le lancement d’une commande avec ssh ou telnet, dans l’exemple, j’utilise le module python pexpect qui se connecter sur un serveur SFTP puis telecharge des fichiers. Pour ssh on peut utiliser le système de clés mais pour FTP, telnet ce n’est pas possible et si vous n’etes pas aussi l’admin du serveur distant.

Voici de code, évidemment il faut installer les paquets openssh-client et python-pexpect si vous utilisez ubuntu ou debian


#!/usr/bin/env python

import pexpect

p=pexpect.spawn(’sftp -C -R 128 thierry@113.231.167.XXX‘)
p.expect(“thierry@113.231.167.XXX’s password: “)
p.sendline(‘MyVerySecurePassword’)
p.expect(’sftp> ‘)
p.sendline(‘get foo.bar /home/thierry/’)
p.expect(’sftp> ‘,timeout=None)
p.sendline(‘bye’)

Ceci est un simple exemple vous pouvez l’améliorer en ajoutant un expect s’il demande d’accepter la clé du serveur …

How to back up or migrate your blog.

The internet is not safe. If you have your data on a webhost, anywhere, it’s probably not safe. If you have a security specialist on your team (and none of us here do), then maybe it’s somewhat safe from malicious attacks, but even then, it’s still vulnerable to plain old hardware failure and general cockups. And the most vulnerable data of all is the data that is hosted on bottom-end hosts like Dreamhost where potentially hundreds of other people are sharing disk space and server resources with you, and as with everywhere on the internet, a sizable proportion of them are guaranteed to be some combination of incompetent and malicious. It’s only a matter of time before one of them accidentally or deliberately brings the server down, accidentally corrupts your files, or breaks in to your filespace and uses it to subsidise their pornography business. It’s happened to me, more than once. Moreover, as I’ve said before, if you are using FTP instead of SFTP, it means that you don’t really care about your data anyway, and pretty much anyone with enough dedication can come along and steal your password and make free with your files if they try hard enough.

Everyone needs backups. Backup. Backup all the time. Remember, I can’t give you extensions if the Ukrainian Pr0n Mafia eats your homework.

Those are the risks – but here is an clarifying certainty: when the semester is over, this web hosting will be turned off and your files will be deleted. So, quite aside from everything else, if you wish to keep your work, you will need to make at least one backup to get it off Dreamhost at the end of the class.

OK, that’s the motivation. Now… how to do this backup thing?

There’s a few options. Too many, as always. Personally, I store all the files I am working on using github, which keeps a record of every change I make, and is very very powerful. However, that’s a bit complicated, and it won’t work easily from UTS computers, so I’ll leave that one as an exercise for the advanced students.

More easy, you might want to keep all your work offline on your home computer, and only upload it at the last minute. That’s pretty possible too. You can run web servers on your home machine and just use whatever your normal home backup system is. (you do have a backup system for your home computer, right?) If you want to do that, you might find it easiest to download a preconfigured web server. There’s a free one for windows called XAMPP, and one for Mac called MAMP. I certainly do offline development myself, although I don’t use those preconfigured web server packages, since I don’t develop on Mac or Windows, but use a nice fresh linux system called Ubuntu which manages it for me automatically. All three of these systems are quite easy to use, although I should warn you that WordPress makes the uploading to a new server a bit fiddly.

But enough broad-brush options. I’m going to get down to the nitty gritty on the simplest option of all: Backing up over (S)FTP.

If you are uplaover cyberduck, you are already part of the way there. The trick is:remember that when you edit your files on the dreamhost server instead of your local computer, you now have some custom changes which you probably want to keep a copy of.

this way lies pain if you ain't careful

Same goes for wordpress’s built-in editor, and its plugin installer. Every time you use that, you are modifying files on the server which leaves them at the whim of Dreamhost. Nothing wrong with that, if you want to see the changes you make take instant effect, but it can be bad if you want to make sure you get to keep that change if something goes wrong.

So the trick to avoid losing stuff here is to use Cyberduck’s “Synchronise” feature, which will synchronise files on your local computer and the server, and keep them as similar as possible. This handy little feature exists on the “actions” menu of cyberduck

Here's your lifesaver

Once you click on that, you get to choose a local folder on your computer to keep in sync with the server.

cyberduck's synchronise window

Now, choose whether to do update the server with your local changes (“upload”), update your local copy with changes from the server (“download”), or both at once (“mirror”). Press “continue” then wait until it’s finished. And now you’ve backed up your files. Hurray!

so many choices, so little time

Note that Dreamweaver can also do this. The adventurous among you are welcome to try that, but I’m not going to show you. Why? Well, for a start, all the staff here use Cyberduck in preference to Dreamweaver. For another, just because dreamweaver can do almost everything doesn’t mean it can do everything well, and one of the things that annoys me is how incredibly slow its file synchronisation system can be. For a final reason, it doesn’t hurt to learn how to use a couple of different tools, especially free, open-source ones like Cyberduck.

But, are you done here? Well, no. Not unless you are building a site in pure HTML. The thing about WordPress, or joomla, or any of these modern blogs and things is that they store information in a database on the server (in our case, MySQL) which is completely distinct from the files on the server. It’s a whole other way of storing information, and it’s completely different to back up. We won’t be going in to MySQL in detail this semester, but you will need to know at least that it’s there and how to make copies of the stuff it stores on your behalf. Here goes:

You need to find the admin interface for you database. The easiest way to do this is to log in to the dreamhost control panel and go right there.

You’re looking for the “web administration” link, which for all of our databases is though a special web application called “phpMyAdmin”.  You can also usually work out how to get there, if you rememebrd your MySQL hostname, by just putting “mysql” in front of your site address. So for my site, at http://dan.map-netcultures.net, the databse admin is at http://mysql.dan.map-netcultures.net. It will be different for you, mind.  You’ll need your database username and password to login, mind. You did write it down, didn’t you?

That will take you into the convoluted bowels of phpMyAdmin. There’s lots of buttons here, which you can gleefully ignore. The key thing is to select the database that you are using for your blog (you did write that down, right?) In my case it’s “dan123″.

Next, you want to do an “export” of that database, which is, predictably, under the “export” menu – the little guy highlighted in red below.

Now, finally, it’s time to do the deed. The expor menu has terrifyingly many options, most of which you don’t need. But you do probably want to “save as a file” and moreover a “zipped” file. Select those options and press the “go” button.

The red circle highlights the only controls I suspect you will need unless you are feeling rather advanced indeed

Bam! you now have a database backup called something like yourname.sql.zip

Put this somewhere safe

Where this downloads to depends on your browser. the important thing is that you should put it somewhere safe so that you can find it again later. And while we’re here- you can also restore backups. This is handy if you make a horrible mistake while you are editing your blog and want it to go back the way it was. That part is straightforward if you’ve gotten this far – to reload your exported database, you simply “import” it:

The opposite of exporting is importing. If you dump your exported file into the import box, you will get your database restored to the way it was when you exported it.

Too easy. Now, go forth and make radical changes to your blog, secure in the knowledge you can always put it back the way it was.

Bonus points – wordpress-specific tips

These instructions so far have been completely generic. But if you are using wordpress, you have a couple of extra options, and a couple of extra difficulties.

Firstly, let me re-iterate that full blown wordpress install is a bit fiddly, but the instructions are
here. If you’ve done anything at all complex to your blog, this is the avenue open to you. It’s pretty well documented, but takes a few minutes.

There’s also the wordpress-special option of exporting and re-importing your blog. This only works if you haven’t done any massively self-hosted word-press specific things – if you haven’t customised your themes, if you aren’t embedding custom javascript, and you haven’t installed any except the wordpress.com preferred plugins. If all those hold true, there is an easy option: Export your blog posts, and import them again later. Instructions are here. Two steps. 2 minutes. Easy.

Configure Dropbear SSH server

SSH server can’t work independently, it require SSH server (like dropbear) to run. Dropbear require libz & libcrypto to run, so please first install them (if not install already). To install the dropbear SSH server, follow the steps given below.

-bash-3.2# tar –xvf dropbear-0.52.tar.bz2

-bash-3.2# cd dropbear-0.52

-bash-3.2# ./configure

-bash-3.2# make PROGRAMS=”dropbear dbclient dropbearkey dropbearconvert scp”

-bash-3.2# make PROGRAMS=”dropbear dbclient dropbearkey dropbearconvert scp” install

It will install the dropbear SSH server. Now you can create the softlink of dbclient as ssh. It is not mandatory, you can also use dbclient instead of ssh command.

-bash-3.2# ln –s /usr/local/bin/dbclient /usr/bin/ssh

To start the dropbear SSH server, first create the dss & rsa keys (for the encryption of SSH packets) in /etc/dropbear directory. SSH server does work without encryption, so it is mandatory to create the keys.

-bash-3.2# dropbearkey –t dss –f /etc/dropbear/dropbear_dss_host_key

-bash-3.2# dropbearkey –t rsa –f /etc/dropbear/dropbear_rsa_host_key

Now dropbear can be started simply by following command.

-bash-3.2# dropbear

If above will done properly you can connect your system via SSH.

View Complete Topic

     Sometimes you need to be sure that your FTP transfers are not going to get hijacked or intercepted by a third party. Most FTP applications are not designed for secure transfers over SSL which is why WinSCP was developed.

    The WinSCP program (for Windows users) is a great alternative FTP client you should really try out.  It is an open source free SFTP client and FTP client for Windows. Legacy SCP protocol is also supported.Its main function is safe copying of files between a local and a remote computer.

    For starters, WinSCP is not the easiest FTP client to use. Because of the extra security features, it can appear overwhelming at first although it does feature a graphical user interface. It also supports integration with Windows in terms of drag & dropping files, pasting in URL’s and shortcuts.

More Features:-

• Graphical user interface
• Translated into several languages
• Integration with Windows (drag&drop, URL, shortcut icons)
• U3 support
• All common operations with files
• Support for SFTP and SCP protocols over SSH-1 and SSH-2 and plain old FTP protocol
• Batch file scripting and command-line interface
• Directory synchronization in several semi or fully automatic ways 
  Integrated text editor
• Support for SSH password, keyboard-interactive, public key and Kerberos (GSS) authentication
• Integrates with Pageant (PuTTY authentication agent) for full support of public key authentication with SSH
• Explorer and Commander interfaces
• Optionally stores session information
• Optionally supports portable operation using a configuration file in place of registry entries, suitable for operation from removable media
  

    Not sure where to start, after you downloaded it?  They have guides and useful tutorials to help you out. There is Portable as well as installer version is available for download.

  If security is your concern, you won’t be disappointed by WinSCP.

Download WinSCP v4.1.9 Installer | Portable | Source Code

I have researched several vendors who provide FileTransfer over SFTP, FTPS or HTTPS to send large binary files to customers, other departments, vendors etc.

This will be published soon

FTP jest protokołem typu klient-serwer, który umożliwia przesyłanie plików z i na serwer poprzez sieć TCP/IP. Jego główną wadą jest brak szyfrowania transmisji co wiąże się z przesyłaniem haseł oraz nazw użytkowników w postaci jawnej. Wada ta dyskwalifikuje ten protokół jeżeli zależy nam na bezpieczeństwie. Na szczęście istnieją alternatywy wykorzystujące SSH do komunikacji oraz gwarantujące szyfrowanie przekazywanych danych. W tym wpisie postaram się pokazać jak wdrożyć sFTP lub SCP na swoim serwerze pozwalając odejść poczciwemu FTP na emeryturę.

Trochę teorii z wikipedia na początek :

sFTP – protokół typu klient-serwer, który umożliwia przesyłanie plików z i na serwer poprzez sieć TCP/IP.
Jest on pozbawiony wad, które posiada zwykły FTP. Przesyłając plik przy użyciu protokołu FTP uzyskujemy dobre przepływności, ale nie zyskujemy bezpieczeństwa, nasze hasła nie są szyfrowane. Znaczną poprawę bezpieczeństwa przynosi protokół SFTP, który nie wymaga na danym hoście posiadania serwera FTP, wystarczy konto SSH, którego jest on rozszerzeniem, używa jego struktury oraz przez niego się łączy.

SCP – protokół SCP jest bardzo podobny do protokołu RCP (BSD), jednak w przeciwieństwie do RCP SCP korzysta z szyfrowanego połączenia podczas transferu, dzięki temu podsłuchanie transmisji jest zdecydowanie trudniejsze. Sam protokół SCP nie zapewnia uwierzytelniania, opiera się on na protokole SSH. Protokół SSH zajmuje sie tylko transmisją plików, jego przewagą nad protokołem FTP jest to, że oprócz szyfrowania potrafi również przekazać razem z plikiem jego podstawowe atrybuty (np. uprawnienia).

sFTP przy zastosowaniu Scponly
Scponly gwarantuje nam bezpieczne przesyłanie plików dzięki SSH jednak wyłącza możliwość korzystania z samego shella. Tracimy możliwość wykonywania czy robienia czegokolwiek z poziomu SSH. Dobre rozwiązanie jeżeli chcemy dać dostęp do “ftp” ale nie chcemy dawać możliwość korzystania z shella.

Instalacja :
sudo apt-get install scponly

Konfiguracja użytkownika :
sudo chsh -s /usr/bin/scponly user

Od tej pory możemy logować się tylko przy pomocy klientów sFTP na konta który ustawimy. Jeżeli zalogujemy się przez SSH na konto któremu ustawiliśmy sFTP zobaczymy jedynie krótką informację :
WinSCP: this is end-of-file:0

sFTP + SCP przy zastosowaniu rssh
rssh jest restrykcyjną powłoką działającą w oparciu o ssh pozwalającą jedynie na łączenie się przy pomocy SCP lub sFTP wyłączając, jak poprzednio, możliwość logowania się do konsoli. Dodatkowo rssh pozwala na wymianę plikami przy pomocy rdist, rsync, lub cvs.

Instalacja :
sudo apt-get install rssh

Konfiguracja :
Musimy dodać dodatkowy wpis do /etc/shells o treści
/usr/bin/rssh

Oraz edytować plik /etc/rssh.conf w którym wybierzemy jakie rodzaje połączeń będą akceptowane :
#allowscp
#allowsftp
#allowcvs
#allowrdist
#allowrsync


Na koniec musimy ręcznie zmienić powłokę użytkownikom który mają korzystać z rssh. Dokonamy tego w /etc/passwd, przykładowy wpis dla usera bartek
bartek:x:1001:1001:,,,:/home/bartek:/usr/bin/rssh

Jeżeli zależy nam na posiadaniu dostępu do konsoli oraz bezpiecznej wymienianie plikami to nie robimy nic poza sprawdzeniem czy posiadamy wpis
Subsystem sftp /usr/lib/openssh/sftp-server
w configu sshd ( /etc/ssh/sshd_config )

Na koniec warto by było wspomnieć o klientach sFTP/SCP. Skupię się jedynie na kliencie dla systemów z rodziny Microsoft. Godnym polecenia wydaje się WinSCP.
Możliwości programu :
* Graficzny interfejs użytkownika
* Kilka wersji językowych
* Integracja z systemem Windows (przeciągnij i upuść, URL, ikony skrótów)
* Wszystkie typowe operacje na plikach
* Obsługa protokołów SFTP i SCP
* Skrypty i linia komend
* Synchronizacja katalogów na kilka automatycznych i częściowo zautomatyzowanych sposobów
* Wbudowany edytor tekstu
* Wsparcie dla haseł SSH, keyboard-interactive, public key and Kerberos (GSS) authentication
* Integracja z Pageant (PuTTY Agent) dla pełnego wsparcia public key authentication(klucza publicznego)
* Do wyboru interfejs wzorowany na Eksploratorze Windows lub Norton Commanderze
* Opcjonalny zapis ustawień sesji.
* Opcjonalnie wspiera pracę bez instalacji używając pliku konfiguracyjnego zamiast wpisów rejestru, nadaje się do pracy z nośników wymiennych

Linki
http://www.sublimation.org/scponly/wiki/index.php/Main_Page
http://dragontoe.org/rssh/
http://winscp.net/eng/docs/lang:pl
http://pl.wikipedia.org/wiki/File_Transfer_Protocol
http://pl.wikipedia.org/wiki/SFTP
http://pl.wikipedia.org/wiki/SCP
Bezpieczne korzystanie z danych na kontach FTP
Bezpieczne wgrywanie plików na konto FTP

TextWrangler 3.0 (2538) (2009-08-24)
Bare Bones Software, Inc.

오랜만의 업데이트. 그동안 밀린 버그 수정과 Mac OS X 10.6 Snow Leopard 호환성 확인이 주 목적인 듯.

You can view a file in the remote machine using vim (or vi) or compare it with a local file using vimdiff by just using scp:
vi scp://kousik@remote.server//home/kousik/remote-file

vimdiff scp://kousik@remote.server//home/kousik/remote-file local-file

You could have copied the remote-file to the local machine first, but this way you don’t have to do it explicitly.

By the way, sftp also works fine.

SCP is a secure protocol to transfer files based on SSH, SFTP (SSH FTP) is a newer SSH FTP with more directory and file operations integrated.

ECMerge does integrate a FTP supporting multiple parallel connection, but does not yet integrate a SCP or SFTP protocol client. However you can still use ECMerge to compare your files updated through these protocol. WinSCP is well known SCP/SFTP client which let you compare your files with the differ of your choice. The base differ is simply “fc” which is really basic. 

To configure WinSCP for ECMerge, go to the Preferences panel, then the Commands page, the “File Compare” can be configured this way:

c:\progam files\ellié compuing\merge\guimerge.exe --mode=diff2 "!" "!^!"

Have a good week
Armel