Saturday, September 16, 2006

Rsync on Windows

When several computers are networked together, you can take advantage from this - you can setup rsync to synchronize data between computers. This can be particularly useful when you want to sync laptop with desktop


Note: Maybe this tutorial is over-verbose, but i'm used to assume nothing from the _user_.


I'll be talking about Rsync usage in Windows... Although i am currently writing this in Vim editor under Linux.


After you have them installed, follow the following steps:


First, you have to install both cwRsync packages from here . After you have them installed, modify the batch source below to suit your needs. :)
Don't worry, the batch file example is reasonably commented and places that have to be changed are marked by "CHANGE HERE" lines


-------Start Batch example-------

:: This is first to suppress command themselves echoing to the screen
@ECHO OFF

:: Make environment variable changes local to this batch file
SETLOCAL

:: This command specifies directory cwRsync is installed to
:: Specify different path if different from default
SET CWRSYNCHOME=C:\PROGRAM FILES\CWRSYNC

:: Set CYGWIN variable to 'nontsec'. That makes sure that permissions
:: on your windows machine are not updated as a side effect of cygwin
:: operations.
SET CYGWIN=nontsec

:: Set HOME variable to your windows home directory. That makes sure
:: that ssh command creates known_hosts in a directory you have access.
SET HOME=%HOMEDRIVE%%HOMEPATH%

:: Make cwRsync home as a part of system PATH to find required DLLs
SET CWOLDPATH=%PATH%
SET PATH=%CWRSYNCHOME%\BIN;%PATH%

:: Example : C:\WORK\* --> /cygdrive/c/work/*
::
:: Example 1 - rsync recursively to a unix server with an openssh server :
::
:: rsync -r /cygdrive/c/work/ remotehost:/home/user/work/
::
:: Example 2 - Local rsync recursively
::
:: rsync -r /cygdrive/c/work/ /cygdrive/d/work/doc/


:: Simple menu for choosing synchronization direction

:: CHANGE HERE!
:: You are free to change the computer's name from Foo to something more sensible
:: Do one search and replace run on this menu.
echo 1. Retrieve Wiki from Foo
echo 2. Send Wiki to Foo
CHOICE /C:12 /N Make your choice:
IF ERRORLEVEL == 2 GOTO TO_FOO
IF ERRORLEVEL == 1 GOTO FROM_FOO

:: You must replace paths, IP adresses and usernames
:: An explanation for the clueless:
:: Replace user with user which was defined in secrets file
:: Replace IP with, well, Foo's IP adress
:: Replace path with your path
:: Note about cygwin's path: It consists from /cygdrive/x/Path
:: where x is your drive's letter, eg C drive
:: Another thing to note about UNIX paths in general
:: is that they use forward slashes, not backslashes
:: EG, C:\Windows\system32 in this format would be /cygdrive/WINDOWS/system32
:: Yet another difference is that path names are CASE SENSITIVE,
:: Meaning that Doc, doC and doc are three different filenames

:: CHANGE HERE!

:TO_FOO
rsync --del --progress -av "/cygdrive/c/Place/That/Needs/To/Be/in/sync/" user@192.168.1.1::Wiki/
GOTO END

:: CHANGE HERE!

:FROM_FOO
rsync --del --progress -av user@192.168.1.1::place/ "/cygdrive/c/Place/That/Needs/To/Be/in/sync/"
GOTO END

:: Here are the explanations of command line flags
:: used (taken from manual of course :):
:: -a, --archive archive mode; same as -rlptgoD (no -H)
:: --del an alias for --delete-during
:: --delete-during receiver deletes during xfer, not before
:: -v, --verbose increase verbosity
:: --progress show progress during transfer



:END
echo -----------------
echo I hope the sync was succesful! :-)

-------End Batch example-------


When you're done with that batch file, you should save it as C:\Program Files\cwRsync\sync.bat and you're done with the client part...


The daemon (server, service) part needs two files for configuration:

rsyncd.conf and rsync.scrt

The first file contains configuration, folders for sync et cetera.

Rsyncd.scrt contains plaintext pairs of usernames/passwords. You can name rsync.scrt whatever you want to, as long as you don't forget to update pointer in rsyncd.conf.



-------Start rsyncd.conf example-------

use chroot = false
strict modes = false
hosts allow = *
log file = rsyncd.log
pid file = rsyncd.pid
secrets file = rsync.scrt

[place]
path = /cygdrive/c/The Place/Docs
read only = false
auth users = user
transfer logging = yes

-------End rsyncd.conf example-------

-------Start rsyncd.scrt example-------

user:password

-------End rsyncd.scrt example-------


These configuration files must be unique for both hosts. I.e., 1 host must target 2nd host in batch file, et cetera.


When all of the configuration is done and config files are in place... start the daemon:

---

sc config RsyncServer start= auto
net start RsyncServer

---
(Enter this to cmd prompt...)
(no, the weird space in "start= auto" is NOT a typo. That's actually correct syntax... Duh)


Or, if you prefer the tedious rodent-GUI way:

Go to Start>Settings>Control Panel>Administrative tools>Services, find RsyncServer, go to properties, make startup type automatic, start it.


When finished, launch that "sync.bat" or whatever you have named it, and it should give you a proper mini-menu for choosing synchronising direction. Enter the choice number, and the synchronisation shall commence... Happy synching :-)

Edit (2006-09-20): Sorry, the "choice" command isn't included in Win2k/XP by default; It is part of Microsoft's NT Resource kit... Choice.exe can be downloaded from here. Put it to C:\(Windows|WinNT)\system32 or to directory where is that rsync script located (folder where CwRsync is installed)

Thursday, September 07, 2006

Mysteriously unreachable websites

Today i solved a great annoyance i had for a loooong time... I couldn't access quite many websites:

www.python.org
www.ubuntu.com (No, i am NOT an ubuntu user. It is just that that i couldn't reach their website)
www.lilypond.org
www.ubuntuforums.org
www.delilinux.de

and a whole lot of other domains. I have regarded it as mystery and blamed ISP for that... Until today.

Today, i decided to look at them more closely and my quest was to find similarities between these hosts.

The whole log of how did i diagnose and fix it (maybe oververbose as always):


#This is my router box running Slack10.2
#First, i tried to nslookup and then follow the packets
root@EN6350:/var/log# nslookup python.org
Server: 86.100.0.8
Address: 86.100.0.8#53

Non-authoritative answer:
Name: python.org
Address: 82.94.237.218
root@EN6350:/var/log# traceroute python.org
traceroute to python.org (82.94.237.218), 30 hops max, 38 byte packets
1 123.76.34.95 (123.76.34.95) 2990.280 ms !H 2993.109 ms !H 2999.948 ms !H
root@EN6350:/var/log# traceroute 82.94.237.218
traceroute to 82.94.237.218 (82.94.237.218), 30 hops max, 38 byte packets
1 123.76.34.95 (123.76.34.95) 2995.091 ms !H 2990.231 ms !H 2999.924 ms !H
#This is fake /|\, my gateway's ip is different
#nslookuping once again, to see possible similarities
root@EN6350:/var/log# nslookup lilypond.org
Server: 86.100.0.8
Address: 86.100.0.8#53

Non-authoritative answer:
Name: lilypond.org
Address: 82.94.241.173
root@EN6350:/var/log# nslookup ubuntuforums.org
Server: 86.100.0.8
Address: 86.100.0.8#53

Non-authoritative answer:
Name: ubuntuforums.org
Address: 82.211.81.186
root@EN6350:/var/log# nslookup ubuntu.com
Server: 86.100.0.8
Address: 86.100.0.8#53

Non-authoritative answer:
Name: ubuntu.com
Address: 82.211.81.166
#So, strangely, all of nonreachable hosts are in 82.0.0.0/8 netmask.
#Let's look at route and iptables
root@EN6350:/var/log# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.0 * 255.255.255.0 U 0 0 0 eth0
82.0.0.0 * 255.0.0.0 U 0 0 0 eth1
loopback * 255.0.0.0 U 0 0 0 lo
default 82.135.162.1 0.0.0.0 UG 1 0 0 eth1
#82.0.0.0 route causes all packets sent to some ip address which starts with 82, be sent to null
#So i will delete that route.
root@EN6350:/var/log# route del -net 82.0.0.0 netmask 255.0.0.0 dev eth1
#Now, let's try tracerouting packets
karolis@EN6350:~$ traceroute python.org
traceroute to python.org (82.94.237.218), 30 hops max, 38 byte packets
1 c01.int.balticum.lt (212.59.18.1) 8.700 ms 10.213 ms 9.128 ms
2 c02.int.balticum.lt (86.100.2.17) 62.339 ms 40.145 ms 9.255 ms
3 e01.int.balticum.lt (86.100.2.25) 10.151 ms 10.296 ms 9.614 ms
4 213.226.138.245 (213.226.138.245) 13.482 ms 12.103 ms 14.310 ms
5 213.226.156.60 (213.226.156.60) 232.569 ms 237.165 ms 159.932 ms
6 s6-0-0.11-8-0.ar4.DEN2.gblx.net (208.51.117.249) 196.053 ms 362.947 ms *
7 so1-0-0-2488M.ar1.AMS1.gblx.net (67.17.65.242) 285.953 ms 361.452 ms 242.792 ms
8 * XS4ALL.so-7-0-0.nar1.AMS1.gblx.net (146.82.33.178) 484.540 ms 282.054 ms
9 0.so-6-0-0.xr1.3d12.xs4all.net (194.109.5.1) 249.593 ms 141.258 ms 261.868 ms
10 0.so-3-0-0.cr1.3d12.xs4all.net (194.109.5.58) 356.662 ms 210.328 ms 298.223 ms
11 dinsdale.python.org (82.94.237.218) 91.372 ms 290.529 ms 276.604 ms


So i solved this problem only after about a year since it appeared... Because I couldn't imagine why those sites wouldn't work. But it's better later than never, I guess