Hello! Let me add the SyncEvolution list, because the technical information may be relevant. For those who see this for the first time, it started with an open letter that I sent to the OpenSync list asking whether it really still makes sense to continue with two different projects instead of focusing on one: http://sourceforge.net/mailarchive/forum.php?thread_name=20110103221846.G... I was suggesting that SyncEvolution has the better baseline to continue from. Of course this requires further explanation, which this email is about. It is in reply to Graham because he raised several interesting technical questions. On Mo, 2011-01-03 at 16:57 +0000, Graham Cobb wrote: [...] SyncEvolution has grown organically over time (one could call it evolutionary...), instead of shooting for a grand design covering everything, like OpenSync did for 0.40. The main advantage is that there have been regular stable releases since the very beginning four years ago. On the other hand, features for which there was no real need yet are still missing. There have been different phases: 1. SyncML client for Evolution 2. SyncML client for additional storages (iPhone, Mac OS X, file) 3. backends contributed by external developers (Ove Kaaven: N900 calendar, Franz Knipp/m-otion.com: XMLRPC) 4. SyncML server (direct syncing with phones), both via Bluetooth and HTTP, using the Synthesis engine 5. non-SyncML protocols The last point is the goal for SyncEvolution 1.2, in development right now. It still uses the Synthesis engine and everything that it provides (data conversion, conflict handling). SyncML is also still in use, but only as internal protocol between two peers. What a developer above the engine sees is the storage plugin (aka data source) interface. Conceptually such a plugin must provide: 1. change tracking (otherwise only slow syncs work) 2. data import/export, either in the internal Synthesis format (field list) or in a backend specific text format that the engine understands Further references: * introduction to the Synthesis engine and its data conversion: http://syncevolution.org/development/pim-data-synchronization-why-it-so-hard * convenience class for a data source which has id + revision string for each item and exchanges data as text: http://meego.gitorious.com/meego-middleware/syncevolution/blobs/master/sr... * base class with maximum freedom: http://meego.gitorious.com/meego-middleware/syncevolution/blobs/master/sr... * fully functional example backend: http://meego.gitorious.com/meego-middleware/syncevolution/trees/master/sr... * configuration handling: http://syncevolution.org/development/configuration-handling * communication patterns and server mode: http://syncevolution.org/development/direct-synchronization-aka-syncml-se... * local sync: http://www.mail-archive.com/syncevolution@syncevolution.org/msg01419.html Ove and Franz were able to implement their backends with very little assistance, so the documentation can't be that bad, although there's no doubt that documentation could always be better. Yes, sync is always between two peers. One storage should be the designated "master" copy of the data. Any data which cannot be stored by that "master" will get lost. The master could be in a capable system like EDS or Akonadi, or in the file backend, which can store anything that the sync engine itself can handle. A sync topology is created by defining several of these 1:1 relationships. The master itself might be the client of another server, as long as there are no loops. There is currently no logic for keeping several of these peers in sync, but that could be added at a meta level (keep syncing until all changes have been distributed). Unknown extensions are currently dropped. This could be changed, but leads to additional questions that would need to be sorted out: should such extensions be sent to all peers, or just the one who created them? What if different peers have a different understanding of "X-FOOBAR"? It is safer to limit syncing to the data that is fully understood and modeled in the Synthesis configuration file. Currently this covers vCard 3.0 + extensions and iCalendar 2.0 (including UID + RECURRENCE-ID, VTIMEZONE, VALARM, but not attachments). Yes. The Synthesis engine has dealt with that for 10 years and contains a large collection of tools that can be used to deal with such problems, ranging from different data profiles to a full scripting language that can modify data on-the-fly. The Synthesis engine uses capability descriptions to determine which properties are supported by an unknown peer and has smart merging techniques for individual properties. For example, consider the case where a VEVENT was modified like this: 1. event in sync on peer A and B 2. DESCRIPTION is extended on peer A 3. SUMMARY is modified on peer B 4. syncing recognizes the conflict and resolves it by using the SUMMARY of peer B (because the item on B is more recent) and the DESCRIPTION of A (because the description of B is a subset of it) These two properties are handled differently because the conflict resolution policy is configured differently to reflect the difference between single-line and multi-line text. In such a case only slow syncs are possible. The Synthesis data modeling defines which properties are compared to find pairs. The drawback of a slow sync is that data removed on one side will be recreated. I have thought a bit about that over Christmas, because I am now in that situation: I can modify the address book on my FRITZ!Box 7390 router, but it is an XML file with no unique identifier for each entry. My idea is to do synchronization in multiple steps: 1. keep a local mirror of all contacts 2. do a slow sync against that mirror to find pairs; items in the mirror which have no corresponding entry on the router can be removed 3. two-way sync between the mirror and my master data 4. upload copy of the mirror to the router The simpler alternative would be to pick some properties and use those as key, perhaps with hashing to keep the key size small. See above. Client-wins/server-wins/most-recent-wins are all configurable. SyncEvolution itself uses most-recent-wins, with smart merging of some properties. We would need to list those, but I'm fairly sure that much of it has been considered already. My own summary, more at a meta level than the details above: * don't reinvent the wheel, use a mature engine (Synthesis) * add features in small steps (more manageable, immediately useful) -- Bye, Patrick Ohly -- Patrick.Ohly(a)gmx.de http://www.estamos.de/