Please see this post for details of the latest beta available for download.

I'm writing a fairly small program, called FolderSync2. This post is to see if there is sufficient interest to make it worth my while polishing it, writing documentation & releasing it.

But what does it do, and why would you be interested in it?
Say you have two computers, which you share files between. You are happily copying files from one computer to the other, editing some of them, and later copying some of the edited files back to the other computer when you need them. Or perhaps you are downloading stuff on one computer, and then copy it over to the other one.

But once you have a lot of files & folders, it starts to get confusing about which computer has the latest version of a file, or maybe you can't remember where you put the latest version. Or perhaps you aren't sure if the other computer has all the files. This is where FolderSync2 comes in - it handles all that for you! How? By ensuring that both folders (one on one computer & one the other) are kept "synchronised". i.e. The folder you have on one computer is exactly identical to that on the other one. So if you can see it on one computer, then you know where to find it on the other.


FAQ
First question you might ask is "Does FolderSync2 need to be running all the time?" The answer is "No!" You merely run FolderSync2 when you want to update both folders (computers) with the latest changes.

Second question you might ask is "Can I only modify one folder?" Again, the answer is "No!" You can change files & folders on both computers, and during synchronisation both computers will be updated simultaneously. This is what makes FolderSync 'unique', and much more powerful than a simple backup program.

The next question you might ask is "Can I edit the same file in both folders (computers)?" The answer is "No. After editing on one computer, you must synchronise before editing it on the other computer." This is because FolderSync2 has no idea what is inside a file, so it can only copy a whole file from one computer to the other.

You might then ask "What happens if I *do* edit the same file on both computers?" FolderSync2 will warn you about this, and will ask you what it should do. (Or it can simply ignore problematic files.)

You could also ask "Does FolderSync2 need both computers to be Amigas?" Again, the answer is "No". As long as your Amiga can access files on the other computer, it should work. Currently I am using FTP Mount to access my Windows PC's files. It should be possible to get it working with SMBFS (to access a standard Windows file share, aka Samba), but I have not yet tried.

FTP Mount does present some 'interesting' limitations, which means that currently you need WinUAE running on Windows, but I aim to get rid of that need. (Also, that need should be removed if FTP Mount gets a certain bug fixed.)


What's the "2" at the end of "FolderSync2" all about?
Some years ago I wrote a program called just "FolderSync", which had much the same aim. Eventually I stopped releasing updates, due to lack of feedback/interest, so the version on Aminet is really old & crappy.

FolderSync2 is much better than the old FolderSync:
* Easier & more user friendly - I am almost tempted to say "Idiot proof"!
* Much more robust - it should be completely reliable, even if you do something silly.
* More flexible, and easier for me to extend further.
* Rewritten from scratch using PortablE, so it will have native OS3, OS4 & AROS versions.


Are there other programs which do something similar?
Not on the Amiga, as far as I know. I have seen one or two crude attempts to do something similar, but they are hardly better than a plain backup program.

Microsoft has released the program SyncToy for Windows, which is very similar. But FolderSync (1) predates SyncToy :)


A reminder about the purpose of this post
If you think you would find FolderSync2 useful, then please let me know. If you are NOT SURE, then also let me know. (Please don't express false interest - if you really are not sure, then please say so, rather than exaggerating your interest.) If you have any question, then please ask away :)

Last edited by ChrisH on 17-Jan-2011 at 07:43 PM.
Last edited by ChrisH on 18-Jun-2009 at 04:08 PM.
Last edited by ChrisH on 10-Jun-2009 at 08:26 PM.
Last edited by ChrisH on 04-Jun-2009 at 08:31 PM.
Last edited by ChrisH on 04-Jun-2009 at 08:30 PM.
Last edited by ChrisH on 30-May-2009 at 09:29 AM.
Last edited by ChrisH on 17-May-2009 at 06:10 PM.
Last edited by ChrisH on 25-Mar-2009 at 09:16 PM.

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

@ChrisH

sound very interesting and usefull.

Did it all kind of data-files? (image, text, music..etc?)

Image and Text would be interesting for me.

_________________
I´m back in 2023 on Classic Amiga with my A1200/Blizzard1230IB@50Mhz, 32MB RAM, AmigaOS3.2 and ROMs, Indivision AGA MK3, Author of Amiga-Flipclock (OS4)

@ChrisH

As you may remember, I was trying to use FolderSync some years ago to keep the local and server copy of my website in sync without copying the whole thing every time.

We never got it working due to FTPMount, and subsequently I started using a script which only copies files if newer than the last time I ran it. This has limitations.

If FolderSync2 works with FTPMount then - as they say on Dragons Den - "I'm in!"*

Chris
* Actually, do they say that? I think it's either "I'm out" or "Let me make you an offer...", neither of which is appropriate here.

Last edited by Chris_Y on 25-Mar-2009 at 09:47 PM.

_________________
"Miracles we do at once, the impossible takes a little longer" - AJS on Hyperion
Avatar is Tabitha by Eric W Schwartz

@ChrisH

Sounds nice and useful.

Is it possible to avoid using the "both ways" feature, i.e. to lock one folder so it acts as the master, forcing the other one to become its mirror?

What does it do with links? Soft/hard links, links pointing to another file within or outside the sync'ed tree, links with absolute/relative paths?

Best regards,

Niels

@fingus
FolderSync2 has NO IDEA about what files contains, it just copies them when they have been updated. The reason this is a "big deal" is that (unlike a backup program) it does this in both directions simultaneously.

So images, text, music, etc is fine. Just don't expect it to MERGE two files together, because it won't happen. Files are just "black boxes" as far as FolderSync is concerned.

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

@nbache
Yes, there is a "backup mode", which allows you to force one folder to become the master. But I only added it because I needed it during development, and do not expect it to be needed, unless there are some synchronisation bugs.

But if you will ONLY be using the "backup mode", then you might as well just use a normal (incremental) backup program. It will probably be more efficient & easier to use.


I have NOT yet considered soft/hard links, mainly because I don't use them. The easiest thing would be for me to treat them as normal folders. Trying to actually duplicate soft/hard links sounds rather difficult & error prone, although I haven't actually thought about it. Some specific examples of their use might help me here.

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

@Chris_Y
Now that I fully understand all the limitations of FTP Mount, I can say that it would be *impossible* for the old FolderSync to work with it. But FolderSync2 is a complete redesign, which is (in general) a lot more tolerant of filing system limitations, so (touch wood) it is working well synchronising several GB of data (I only just got it working, so there may still be some glitches to work around).

I guess you'd only be using it "Backup Mode"? I'm really surprised that there isn't ANY incremental backup program that will work with FTP Mount. You could even write one yourself - incremental backup is so *incredibly easy*, that a simple one barely constitutes a program at all. (That said, I find text parsing "incredibly simple" (ish) now (see the ToyInterpreter included with PortablE), so it may simply be a case of "It's easy when you know how".)


BTW, FolderSync2 is also built on my second attempt at portable file/folder OOP classes (FolderSync1 was built on my first attempt), and these should also be a lot more robust & tolerant of weird filing systems (in general).

Last edited by ChrisH on 25-Mar-2009 at 11:18 PM.
Last edited by ChrisH on 25-Mar-2009 at 11:17 PM.
Last edited by ChrisH on 25-Mar-2009 at 11:16 PM.
Last edited by ChrisH on 25-Mar-2009 at 11:15 PM.
Last edited by ChrisH on 25-Mar-2009 at 11:13 PM.

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

@ChrisH

There's this kind of a program in os4depot.

_________________
Rock lobster bit me. My Workbench has always preferences. X1000 + AmigaOS4.1 FE
"Anyone can build a fast CPU. The trick is to build a fast system." -Seymour Cray

@Tomppeli
While I haven't tried that program specifically, it appears to be the kind of thing I was referring to when I said "crude attempts to do something similar". Such programs are not smart about how they synchronise, as they use some very simple rule(s), and this generally means that they need a LOT of care to do the job I described in my first post (or more likely they can't do it at all). I wanted a program that made my life easier NOT harder!

Please try out Microsoft's SyncToy in "Synchronize" mode, to see what REAL (intelligent) folder synchronisation is like.

Last edited by ChrisH on 25-Mar-2009 at 11:52 PM.
Last edited by ChrisH on 25-Mar-2009 at 11:50 PM.

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

@ChrisH

Quote:

Please try out Microsoft's SyncToy in "Synchronize" mode, to see what REAL (intelligent) folder synchronisation is like.

Does it work on Linux or AmigaOS ?

_________________
Rock lobster bit me. My Workbench has always preferences. X1000 + AmigaOS4.1 FE
"Anyone can build a fast CPU. The trick is to build a fast system." -Seymour Cray

@ChrisH

What about to use a webspace to synchronize the files between two different computers on online basis? Similarly to DropBox. I am sick of using the USB when transferring files from PC to Amiga or vice versa.

Last edited by AmiKit on 25-Mar-2009 at 11:56 PM.

_________________
Modern Retro Experience

@ChrisH

If this can also be used as a way do a "live" replicated backup of your system (many folders or drives at once) to another system/drive, then I think it sounds very useful. If it's only one folder at a time then not so much. I confess I haven't read the details on it yet.

Plaz

@ChrisH

I've used something similar on the PC for years called SyncBack. I'd be interested in such a program. It would be even more useful if it had a "backup" option too, i.e., one computer has the master copy, and the other is the backup (a one-way sync).

Hans

_________________
http://hdrlab.org.nz/ - Amiga OS 4 projects, programming articles and more. Home of the RadeonHD driver for Amiga OS 4.x project.
https://keasigmadelta.com/ - More of my work.

@ChrisH

I would love this proggy on OS3.x

I have spent many hours trying to find a good program to do this!!!! (on any platform). I have found several crappy ones but no good ones yet.

FolderSync2 is 4 me!

Long live E!

_________________
Wanna try a wonderfull magical Amiga strategy game?
Total Chaos AGA

Sounds interesting alright, it's one of the few features I actually miss when I go home to my Amigas - having to manually copy updated files...

_________________
RobTheNerd.com | InstallerGen | SMBMounter | Atoms-X

Well, generally hard to say if there is a need for such a program. Amiga is a hobby for most of us... Cannot think of a situation, where syncing files could be handy for a hobby. Syncing files is needed, if you work professionaly with your computer (e.g. sync word docs) with your machine at work. Or syncing files needed for a windows or mac machine with USB Stick... I cannot think of many situation where I need an AMIGA file on any other machine... This might change sometimes, if we see "professional" apps (like wordprocessing, spreadsheet or customer relation management) on our AMIGA.

BUT: I would install such a proggy, even if I wouldn't use it much. Realistically seen it at the moment it might be useful for:

Pagestream Users (Laptop and AMIGA - as one of the few left over professional apps)
Sourcecode (programming under UAE if you are away from your AMIGA)
Webprogramming
Pics
Maybe Hollywood presentations

PLUS FolderSync should be able to sync Linuxpartionsm, ftpmounts and SAMBA shares as well.

So yes, go for it I am in !!

_________________
Geomarketing at
www.geobiz.de
www.gebietsplanung.net
www.geomarketing-consultant.de

@AmiKit

Quote:

I am sick of using the USB when transferring files from PC to Amiga or vice versa.

One way is to send files in email to yourself.

@ChrisH
Could you explain in words what's that "intelligence" in SyncToy (read my previous post).

Last edited by Tomppeli on 26-Mar-2009 at 04:06 PM.

_________________
Rock lobster bit me. My Workbench has always preferences. X1000 + AmigaOS4.1 FE
"Anyone can build a fast CPU. The trick is to build a fast system." -Seymour Cray

@ChrisH

Backup is fine for one site, but I have another where content is potentially edited both web-side and on the Amiga - that needs a proper sync ideally.

Quote:

I'm really surprised that there isn't ANY incremental backup program that will work with FTP Mount.

I don't know of any incremental backup programs full stop.

Not that check the dates anyway, I think most check the archive bit and I'm never sure what AmigaOS does with that. Others put the backup into some kind of archive file.

Name one that might do what I want and I'll try it!

My script works OK but it tends to re-copy files unnecessarily in certain circumstances, doesn't create directories and can miss files if the connection is lost during copying.

Chris

_________________
"Miracles we do at once, the impossible takes a little longer" - AJS on Hyperion
Avatar is Tabitha by Eric W Schwartz

@ChrisH

Quote:

ChrisH wrote: @nbache Yes, there is a "backup mode", which allows you to force one folder to become the master. But I only added it because I needed it during development, and do not expect it to be needed, unless there are some synchronisation bugs. But if you will ONLY be using the "backup mode", then you might as well just use a normal (incremental) backup program. It will probably be more efficient & easier to use.

Yes, that's what I do now. I'm using MirrorCopy in a script which can run overnight to mirror half of my HDs to the other half. But it doesn't handle links very well, and neither does my other favourite (for slightly different tasks), Cloanto's DirDiff. So that's why I wanted to know if your program had good link support and could also work unidirectionally.

Quote:

I have NOT yet considered soft/hard links, mainly because I don't use them. The easiest thing would be for me to treat them as normal folders. Trying to actually duplicate soft/hard links sounds rather difficult & error prone, although I haven't actually thought about it. Some specific examples of their use might help me here.

Yes, you'd have to analyze all the combinations of the cases I mentioned above (and I forgot about links to directories vs. to files) and make sure you know what to do and how to accomplish it. At least for some of the combinations it becomes necessary to either make multiple traversals of the directory tree or to save some information elsewhere and go back and fix the links afterwards.

But I haven't made the full analysis myself either

Best regards,

Niels

@ChrisH

Rsync is a fairly mature open-source program you might be interested in getting ideas from:

http://en.wikipedia.org/wiki/Rsync

I have to think that this kind of thing only really works well when the machines are very regularly connected to the same network (or the Internet) and the syncing occurs without user intervention.

A good network backup program would be more interesting to me.

Tom