On 09/04/14 21:39, Patrick Ohly wrote: It is this concept which I have found hard to grasp. I think part of my problem is that there is no "object" on which to hang these "per-peer source" properties. I think a diagram (or even a UML description) would help. In the absence of a diagram, let me see if I have grasped this... Imagine a context (call it @Ctx) which contains three sources (S1, S2, S3) and two sync configs representing peers (P1@Ctx, P2@Ctx). Then there are a a bunch of objects as in this matrix: | S1 | S2 | S3 | ---+----+----+----+---- P1 | X11| X12| X13|P1 sync config properties ---+----+----+----+---- P2 | X21| X22| X23|P2 sync config properties ---+----+----+----+---- |S1 |S2 |S3 | |shrd|shrd|shrd| |prop|prop|prop| Where Xnm presents the "non-shared" (=== "per-peer") source properties of Sm for sync config (===peer) Pn. The property called "sync" is a "non-shared source property", so it is a property of each of the objects called Xnm in my matrix. Is that right? If so, I think the terminology description needs to be updated to define these Xnm objects as first-class objects and give them a name. After all, they are the objects of which the "non-shared source properties" are properties. And then there are three types of properties: properties of the three types of objects (Pn, Sm, Xnm). I don't see how that makes sync "primarily" a source property. Doesn't exactly the same logic make it primarily a sync property as there are multiple values for it, one for each peer (isn't that what per-peer means)? The truth seems to be that it is a matrix property with values that are per-source-peer-combination, and pretending they are some special sort of source property just causes confusion. [I have re-ordered Patrick's email here] OK, that makes sense. This is setting a property of one of my Xnm objects. So, what happens when I specify "--configure sync=two-way memotoo@default"? Is that an error (because sync is not a property of my Pn objects)? Just like "--configure sync=two-way @default addressbook" is an error because sync is not a property of my Sm objects?

On 09/04/2014 22:39:06, "Patrick Ohly" <patrick.ohly(a)intel.com&gt; wrote: I'm trying to follow the logic. What is considered the non-advanced scenario then? If I were to sync a single local data store with a single phone, is that considered to be a sync with different databases? And I had visualized local-to-multiple database sync as a hub-and-spoke scenario; if it is indeed a hub-and-spoke, I thought that would be the same as a repeated master-slave sync. Regards, Emile

On Thu, 2014-04-10 at 07:03 +0000, Emiliano Heyns wrote: The original SyncML based syncing with EDS as local storage is the easy case. Only the four standard databases (one of each kind), no need to choose anything other than a template (or syncURL/uris if no template exists) and credentials. The world has moved on since then, so it would be a good time to revisit concepts and adapt them where needed. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On 10/04/2014 12:11:34, "Patrick Ohly" <patrick.ohly(a)intel.com&gt; wrote: P1 here is a peer then, and P1@Ctx (aka a sync config) describes within Ctx exhaustively how P1 can sync in Ctx? Or could a single peer (sensibly) have multiple of these within a single context? So: P1@Ctx has sync config properties that describe all relevant sync properties for the peer except the sync mode of that peer with any single data source within that context, which is held by the Xmn entity. If so, for the sake of current discussion, I would propose to call that SyncMode. But above you said that Xmn specifically holds the "sync" mode. Is this different from the "sync" property? My data visualizing skills are non-existent, so I've resorted to the only tool I know in this domain: SQL. Could you guys have a look at http://reichenhack.github.io/syncevolution-model.html to see if this is going somewhere? Regards, Emile

On Thu, 2014-04-10 at 15:31 +0000, Emiliano Heyns wrote: Both works for syncs initiated by the user of SyncEvolution. When SyncEvolution acts as SyncML server, the protocol forces us to have a single sync config for the SyncML client with all sources enabled that it is meant to have access to. Apart from that there's nothing wrong with having different sync configs for the same peer, whether it is in the same context or in different ones. The original goal was to have just one sync config (say, "google" instead of "google-contacts" and "google-calendar") because it allows a single command to sync all data of that peer ("syncevolution google" vs. "syncevolution google-contacts && syncevolution google-calendar"); ultimately that's up to the user. Side note: traditionally, a SyncML session (and that includes a local sync) allows multiple sources to be synced at once. For SyncML over HTTP that has some performance benefits (less message exchanges in particular when no data needs to be transferred), but it also increases the complexity of the sync considerably. I am contemplating to run syncs under the hood such that only one source is active at a time. See https://bugs.freedesktop.org/show_bug.cgi?id=57004. However, for the user nothing would change (the "syncevolution google" command would still sync all active sources, just one after the other) and thus the current discussion is not affected by this implementation detail. "uri", "syncFormat" and "forceSyncFormat" also fall into the same category. Calling them "SyncMode" isn't really appropriate. No, it's the same thing. With sync mode I meant the value of the "sync" property: one-way, two-way, etc. "forceSyncFormat, sync, syncFormat, uri" are not placed correctly in that model at the moment. Other than that this looks right. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On 10/04/2014 21:03:05, "Patrick Ohly" <patrick.ohly(a)intel.com&gt; wrote: OK, given this then, "peer" and "sync config" are different concepts. A peer can have multiple sync configs, and a peer loosely translates to a device or service or other syncml instance, but is merely a notational convention to mentally cluster several sync configs. Correct? OK, I've moved them and simply renamed that entity to SyncSource for now. The 'sync' property is currently located on the DataSource -- that is still correct? So this 'sync' property is then not part of Xmn (which I dubbed SyncSource for now), but of the data source. But then (currently) the target-config cannot hold one either, as it's just a "special sync config"; I've updated the schema to show that simply exactly one of the Context's sync config can special in that it is considered its target-config. Updated. Regards, Emile

On Thu, 2014-04-10 at 19:19 +0000, Emiliano Heyns wrote: Correct, and the terminology section describes them as different things, too. Correct. I wouldn't describe it like that. A peer typically really exists in the physical world, so it is more than just an abstract convention. No. What you had is "mode" in "SyncMode" is the "sync" property value. The "sync" property lives in the intersection of data source and sync source, so it *is* part of Xmn. I would even say that it is the most important part of it, although there are others ("uri", "syncFormat" and "forceSyncFormat"). -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

BTW, when I request --source-property ? and --sync-property ?, databasePassword and databaseUser do not show up. Where should I put these in the model? In the data source? Regards, Emile

On Thu, 2014-04-10 at 19:48 +0000, Emiliano Heyns wrote: Huh? They show up here. $ syncevolution --version SyncEvolution 1.4.1 ... $ syncevolution --source-property '?' ... databaseUser = evolutionuser (no default, shared), databasePassword = evolutionpassword (no default, shared) authentication for backend data source; password can be specified in multiple ways, see SyncML server password for details Warning: setting database user/password in cases where it is not needed, as for example with local Evolution calendars and addressbooks, can cause the Evolution backend to hang. Yes.

On 11/04/2014 09:03:08, "Patrick Ohly" <patrick.ohly(a)intel.com&gt; wrote: OK, so setting 'sync' in the special target-config SyncSource (Xmn) is possible but useless. In all other SyncSources this is useful. I was just thrown off by the "in one of the Xmn boxes", which I read as "in any one of the Xmn boxes, at any line". Emile

On 11/04/2014 10:02:05, "Patrick Ohly" <patrick.ohly(a)intel.com&gt; wrote: Hey! Is this documented? SUPER! Emile

After trying to bring the conversation together with what I've learned from rooting around in the .config folder, this is my current understanding. I know the sync description is wrong, it's just what I understand of it right now. I would appreciate corrections towards common usage. We have: contexts. Everything in SyncEvolution happens in a context. A context has a name, and holds some commone properties that are applicable to everything in that context: logdir, maxlogdirs, and deviceId. These all have sensible defaults, so we won’t be bothering with these right now.sources. A source lives in a context, has a name, and describes a database of (either) contacts, events, tasks or memos, using the following properties: backend: the storage engine (such as file, caldav, activesync, …)database: the location (file path, url, whatever) that describes where the database can be found, possibly relative to the peer syncURLdatabaseFormat: the format of the items in the database, in case more than one type is allowed by the backend (currently only relevant for backend ‘file’, which supports text/vcard and text/calendardatabaseUser: the username to access this source, if not specified already by the peer (more on this later)databasePassword: the password in the same vein as databaseUserpeers, which describe the ‘remote side’ of a local-remote sync. This only describes how to reach and access the peer in general, not (yet) how to reach any source inside it; more on that later. These contain loads of properties, but the most interesting are: username: the username to use when accessing the a source associated with this peer. Note that for example you could access the same source with different credentials (and thereby possibly getting different items returned from the same source, based on an authorization scheme)password: like usernamesyncURL: specifies how to contact the peer, for example the caldav or EAS urlsyncs, which tie together a remote peer, a local source and a remote source. This has two properties of interest: sync: defines which way the sync goes, can take values ‘none’, ‘one-way’ or ‘two-way’ (one-way being seen from the local side of the sync)uri: specifies the source for the remote peer.

On Sun, 2014-04-13 at 07:42 +0000, Emiliano Heyns wrote: It depends. As a special case, the data source can use these three properties as fallback when nothing is set specifically for the source. What that means is that "--print-items @foo source" relies on properties in the source alone (because there is no peer mentioned) but "--print-items target-config@foo source" allows the source to use these extra three properties. I am talking about the behavior as implemented in the WebDAV backend. If I remember correctly, the ActiveSync backend always expects to be used together with a peer and does not support "backendUser/Password" at all. This is only a limitation of the implementation; it would be better if it worked like the WebDAV backend. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On 13/04/14 19:45, Heyns Emiliano wrote: That is my fault, for causing confusion. Although @Exchange is, indeed, a context, the name "Exchange" is not being used to mean that in this command. "Exchange" is, in this case, also the name I used for the peer config within the @Local context. Two different entities (of different types) with similar names. And completely unrelated as far as SE is concerned (they get linked later by using the local:// url). But I always think of the peer config as pointing to the peer. So, in my mind, I tend to name the peer config in one context with the same name as the context name it will eventually link to. But I probably shouldn't have done that in the example, to avoid confusion. It might be useful to always include the @ at the front of names of contexts, to reduce this confusion. There is nowhere in the command line where the context name ("Local") can appear without the @ in front -- so you may as well consider the @ part of the name. Graham

------ Original Message ------ From: "Graham Cobb" <g+syncevolution(a)cobb.uk.net&gt; To: syncevolution(a)syncevolution.org Sent: 13-4-2014 21:32:41 Subject: Re: [SyncEvolution] Getting the concepts clear (Was: Configuring a target) So that line means "set sync=two-way, uri=calendar on the Xmn with context=Local, Peer=Exchange, source=calendar." That makes sense (conceptually, anyhow). No, that's OK. Now that I know this, I can fold that into the howto. Emile

On 13/04/14 21:58, Heyns Emiliano wrote: Yes Not quite. If I understand correctly (Patrick?), the Xmn's for each source within target-config@context have a uri property. So the uri in the sending end's Xmn get matched against one of the Xmn's in the receiving end's target-config to select the source on the receiving side. I normally set up Peer@Source with the local URL and the peerIsClient. But there is no reason it wouldn't work the way round you describe. No... I think it means... - side a (sending side): {context: @Peer, peer: Source@Peer}, connects to - side b (receiving side): {context: @OtherContext, peer: target-config@OtherContext} No. It syncs every Xmn from Source@Peer (which has "sync" not equal to "none") and tries to pair it with a relevant Xmn from target-config@OtherContext. So... - side a (sending side): {context: @Peer, peer: Source, Xmn: each one in turn (with sync!=none), URI: taken from each Xmn}, connects to - side b (receiving side): {context: @OtherContext, peer: target-config, Xmn: the one with the URI matching, source: the one corresponding to the Xmn} You can have several sources in the context, because it is the Xmn's which pair up and which have the important properties (like uri). You would only need separate contexts if you needed to specify different usernames. Almost, but not quite. You would set up LocalFolders@Exchange and LocalFolders@Google, and give them both the same target-config in @Local. The uri's would be set up in the Xmn's (for example "LocalFolders@Exchange contacts", "LocalFolders@Google contacts" and "target-config@Local contacts"). That would be the hub-and-spoke system. No. You could also set up "LocalFolders@Exchange calendar", "LocalFolders@Google calendar" and "target-config@Local calendar", using the same contexts and peers (but different Xmn's). The uri's are on the target-config Xmn's. You are probably right about the username/password. Graham

I appreciate the extra info, but not so much the implication that I'm hard to deal with. The documentation might be meaningful to people deeply knowledgeable about syncevolution and its internals. For people trying to get up to speed, they're a disaster. Damn right it's amazing how much effort it takes to get the simplest of things figured out - and not just Patrick's, mine too. It took what, weeks? for Graham to stumble upon the xmn idea. Without that the documents don't make any kind of sense. Even to figure out what counts as simple and complex is damn hard to figure out from the documentation. I'm sorry if this has been a burden on Patrick. It hasn't exactly been a picnic for me, either. I have a master's degree to finish, a family to tend to, and a full time job. I have an irrational thing for open source which makes me put time into things like this instead of just fiddling with the settings until it works for me, but instead I try to figure out the conceptual mess spread throughout the docs so I can tie them together so others after me don't have to. My master's is philosophy. I am used to tying together complex concepts. What I'm not used to is a mishmash of ambiguous terms, silent defaults and position bound parameters, of which I should just hope that Patrick and others don't make any typos, because that just puts me hopelessly off track again. Most of all, I'm not used to having to tiptoe around people's feelings while I'm trying to clarify ambiguities. Bugger this for a lark. If syncevolution as a project thinks the documents are fine as-is, and anyone that can't silently muddle through them is dense, there's nothing useful I have to offer. On Apr 14, 2014 12:20 AM, "Ove Kåven" <ovek(a)arcticnet.no&gt; wrote:

On Sun, 2014-04-13 at 20:32 +0100, Graham Cobb wrote: FWIW, I do the same and think it makes sense. Agreed. I think I am doing that when I talk of contexts. If not, the text should be corrected. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Sun, 2014-04-13 at 20:58 +0000, Heyns Emiliano wrote: In both. You get two log files (one for each side), two data dumps for each pair of sources (again one for each side), and changes are made on both sides. You would use the same target-config for both Google Calendar and Google Contacts. Credentials are the same. The start URL for finding collections (aka syncURL in SyncEvolution) are not the same unfortunately, but this is a Google specialty that can be worked around by not using the syncURL in favor of setting each "database" directly. With other WebDAV servers, it is possible to use a single "syncURL=https://server.example.com" and SyncEvolution will find both CalDAV and CardDAV resources. The "uri" in a local sync is only relevant on the side where the sync is started. Use the actual name of the sources on the target side and their potential aliases (set via the target-config's uri values) don't matter - or more accurately, they are ignored completely. As for the username/password, as you said yourself, that could be in the source. The reason why a target-config is still relevant are the other sync properties which can't be set per source: loglevel, printChanges, dumpData, SSL checking. The target-config is optional because the default values of these properties can be used, but the values must come from somewhere. Taking them from the local side IMHO would drop valuable flexibility, like debugging at different levels (for developers) or enabling backups only on one side (for users). -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Sun, 2014-04-13 at 23:13 +0100, Graham Cobb wrote: No, "uri" on the local side gets matched against "source name" on the target side, not some other "uri" value there. "sync" gets ignored on the target side. The default sync formats are fine for SyncEvolution<->SyncEvolution syncs. So all in all the Xmns on the target side are not used at all. BTW, me might need a better name for the non-target side. "sending" is misleading, because both sides send. I tend to use "normal sync config" (see "target config" in the terminology section) but "normal" is not very descriptive. "originating sync config" perhaps? Native speakers to the rescue, please. "local" vs. "remote" gets introduced and could be used, but they can be ambiguous, too. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Mon, 2014-04-14 at 00:20 +0200, Ove Kåven wrote: Thanks, that's useful and spot on. I just have some very minor corrections and some anecdotal additions. Indeed. It came to be out of lack for alternatives (defining and implementing an entirely new sync protocol is no piece of cake), but it does the job pretty well (IMHO). I had to extend the sync engine occasionally (see PBAP and the new caching sync modes) and may have to do it again, but that was still less work than starting from scratch. Actually, the originating side acts as SyncML server and the target-config as client, hence the "peerIsClient=1" in the originating side. The property has to be set because at some point it might make sense to allow reversing the roles. If that happens and configs are in use where the property is not set, we wouldn't be able to use it. The reason for making the local side the SyncML server is that this role needs to talk with the database more often. It is faster to do that on the side that (usually) has the real local databases. Correct. "Peer" is such a fuzzy term. I couldn't avoid it entirely due to lack of alternatives, though (see that "peers" directory, "peerIsClient", "per-peer" config properties, etc.). -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

Den 14. april 2014 08:57, skrev Emiliano Heyns: No, me neither. Makes one wonder, doesn't it? With what? The documentation? As I mentioned, those who understand what's going on in SyncEvolution, can't put the finger on what's wrong with its documentation. I think I'm one of those, and so although I *could* try to write something, I'm really not sure what kind of explanation would be better than what's already there. So discussing with users is of interest, to understand what kind of explanation would work. Then I wish most other people also knew themselves that well. One of my handicaps is that I rely too much on my intuition. Because it's often right, I trust it, sometimes enough to do what it says even without consciously understanding why. I guess that question has three aspects: 1) what message I wanted to express, if anything; 2) what reaction I expected, if any; 3) why I wanted that reaction, if it mattered to me. Then my answers, for now, are: 1) I didn't say "sink in", I said "get through". (There's a subtle difference, in that "sink in" would probably mean it's just your fault, but "get through" has an alternative interpretation, that something about the communication or message could be at fault.) Anyway, I already said what it meant. That is, I'm not saying you've done anything wrong, I just found this stuff amazing. It strongly suggests that something is wrong and something ought to be done to improve the situation. That's both a negative and a positive. 2) Everyone reacts differently, and a person's reaction to things tell something about how they think. The expected typical response from a person with average self-esteem might have been mild irritation, tempered by mixed feelings about the absurdity of the situation, and some patience with other Internet people's curiously lousy social skills. (That is, an average person might say "ok, that was not nice, but I've seen worse, I'll just reply with a snide remark and forget about it afterwards".) 3) Consciously, I knew I might get a snide remark back, but didn't care. Intuitively, however, I think I did hope you might step back and think about not only the information itself, but the way it's organized, and if this mess can be sorted out by asking a different type of questions than you've been using so far. (Sometimes an emotional "kick" helps people think about things differently. Sometimes not.) Anyway, I've found this exchange interesting; I wanted, and expected, to learn something, and I did. So, thank you for that. And maybe I now have a slightly better idea about how information like this should be communicated. So, if there are no hard feelings, and considering that the Easter holidays may finally give me a little free time, then I could perhaps try to draft up a new description of SyncEvolution for you, if you like.

On Mon, 2014-04-14 at 11:30 +0200, Ove Kåven wrote: In a way it is, because refresh-from-client/server depends on the role of each side. It will always refer to the internal SyncML server resp. client, not the remote server that may or may not be used by a backend on either side. Consider direct syncing between an ActiveSync server and a WebDAV server and it becomes painfully obvious that the traditional "refresh-from-server" is no longer useful. Because they exposed implementation details, refresh-from-client/server were replaced by refresh-from-local/remote, with the better defined (although not always intuitive) "local" side being the one which triggers the sync. It would reverse the direction of these two options, so yes. refresh-from-local/remote would continue to work as before. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Mon, 2014-04-14 at 13:14 +0100, Graham Cobb wrote: Is there really a fundamental difference between your view and Ove's? You both view it as a SyncML client<->server sync. The difference is that Ove leans more towards considering that a hackish implementation detail, whereas you see it as a first-class concept that users need to know and understand. That's correct. WebDAV really isn't a sync protocol, it just has some support for concurrent access to item collections without offering any means for resolving conflicts. ActiveSync could be used as a sync protocol (by relinquishing control over the local copy of the data to the server); SyncEvolution doesn't use it like that and instead just uses it to get changes and reading/writing items. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On 14/04/2014 02:51:59, "Ove Kåven" <ovek(a)arcticnet.no&gt; wrote: Which is why I tried to fit the ideas in SQL. It's the formalized data language I'm most familiar with. My answer to that has been to try to capture the ideas in a tutorial, including the abstract concepts involved. Whether or not such an answer is proper is not for me to decide.

On Tue, 2014-04-15 at 11:50 +0200, Emiliano Heyns wrote: I suspect that running SyncEvolution as SyncML client is still more common (and thus "normal") than running it as SyncML server. For the sake of this discussion, "normal" sync is both, because it involves only one sync config. It's matched based on the SyncML device ID, stored by SyncEvolution in "remoteDeviceID" (for the SyncML client that connects to SyncEvolution) and "deviceID" (for SyncEvolution itself). The credentials are optional; one could also allow syncs without checking them. SyncEvolution skips the check when username and password are left empty. The SyncML client chooses which sources it wants to sync with. This is what SyncEvolution as SyncML client does via the "uri" property. It would be nice if SyncML had allowed some kind of automatic "I want to synchronize the default address book", but that never got standardized. Instead client and server must agree on the name (aka uri) of that default address book. Note that SyncML also allows for server-initiated sync sessions. This is used with phones connected via Bluetooth, where the user interacts with some software running on a more capable device (usually a computer) which then becomes the SyncML server. It's better to think of a local sync as being initiated from the server side, the "originating config" as I started to call it. It's the other way around. The Xmn properties of the originating config specify the sources and how they are paired with the sources on the target side. The syncURL pointing to that context is set in the originating config. The syncURL in the target config is not used for the local sync itself; it can be used instead as placeholder for a common URL for the involved sources on the target side. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

Hello! It's getting more accurate, but it's still not 100% there. On Wed, 2014-04-16 at 08:12 +0000, Emiliano Heyns wrote: It's more like a client in a server-initiated sync. "it" and "its" is ambiguous. I think you mean the right thing: "This incoming server must know what data stores the target config has available; this is registered in the incoming server's Xmn entries". "client" has to be "server" here if you want to be technically correct, and consistent with the user-visible "peerIsClient" property. Correct, except that client and server need to be swapped again. I am not sure how much it helps to explain local sync in terms of SyncML. It helped Graham, but one either has to be intentionally imprecise (swap client and server) or explain about server initiated SyncML syncs. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Wed, 2014-04-16 at 19:58 +0000, Heyns Emiliano wrote: I think that would be better. Note that it'll also cover syncing with phones, which uses the same approach. That's pretty much it. There's not much to add. In SyncML, there is a special message that the server sends to the client. It contains a list of pairs of uri (as named in the client) + sync mode, one pair for each database that the client is expected to sync. The client then starts a sync session just as it normally would. In SyncEvolution, a custom method call is used to convey the same information plus some additional details (for example, temporary config changes for the sync, like an increased loglevel). -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Thu, 2014-04-17 at 07:50 +0200, Emiliano Heyns wrote: I'm not sure I fully understand the question. Do you mean, "can SyncEvolution figure out whether a sync config is for a client or server"? No, not at the moment. This has to be configured explicitly via peerIsClient. Perhaps this can be changed, by making an educated guess based on the syncURL. Case 1: SyncEvolution as HTTP SyncML client. Do this if syncURL=http[s]://. Case 2: SyncEvolution as HTTP SyncML server. Config is found via device ID in first SyncML message, then acts as server. Case 3: SyncEvolution as Bluetooth server (via obexd plugin): first message is SyncML, same handling as for HTTP => act as server. First message is server-initiated sync message => act as client. Case 4: SyncEvolution as Bluetooth client. Do this if syncURL=bt-obex://. Okay, here's the problem. Should SyncEvolution ask the peer to act as SyncML server or SyncML client? This is determined based on the peerIsClient value. Case 5: local sync. Here the roles are currently hard-coded. peerIsClient has to be set because it might make sense at some point to reverse the roles without having to swap the configs. A pretty weak argument IMHO because it will always be possible to swap originating and target config around, too. So in essence, if we drop support for one rare case (Bluetooth sync initiated on a phone as client of a SyncML server on a PC), then peerIsClient becomes redundant. In all other cases, peerIsClient has to be set correctly and is (or should be) tested as a sanity check. Hmm, this might work. I need to think a bit more about this. Feedback from others about the whole discussion would be good. I am a bit reluctant to reuse an already existing term ("sync config") for something else, because there will inevitably be a lot of old documentation that doesn't use it according to the new definition. But legacy compatibility can't be allowed to hold back progress forever either. I remember that it was difficult in the past to talk about the client config or server config because it is ambiguous whether the client config is a configuration *on* the client or on the server *for* the client. Using the neutral "peer config" might avoid that. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Thu, 2014-04-17 at 10:02 +0000, Emiliano Heyns wrote: The target config might be (mis)used like that: set syncURL=bt-obex://... and sync with it -> it acts as SyncML server. Point to it elsewhere with syncURL=local:// -> it acts as SyncML client with a different peer. This is a special case, though, and not intended. In general, a sync config has exactly one role. One can have a hierarchy of terms: peer config / | | \ / | | \ SyncML client config | | SyncML server config | | | target config originating config I'm using "client/server config" here to illustrate the point. I'd prefer to not introduce them because of the ambiguity. I am not convinced that we really need separate terms for SyncML sync configs. "sync config" works for me, with originating and target config as special cases of that. Agreed. "peer" remains as defined before (some entity to sync with), and "peer config" is what SyncEvolution uses to achieve that. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Thu, 2014-04-17 at 14:16 +0200, Emiliano Heyns wrote: Beware of the of the two layers in SyncEvolution: 1. Configuration layer. 2. Engine layer. At the configuration layer, SyncEvolution only knows about sync and source configs (current terms). It has no way of doing any semantic checks. The engine layer is where something in SyncEvolution tries to use these generic configs. This could be a backend (for source configs) or the sync engine (for sync configs), at which point it becomes possible to identify mistakes. So when describing the configuration mechanism, one has to use the generic terms ("setting up a sync config"). When applying that mechanism for specific use cases, the more descriptive terms come into play ("setting up a SyncML server config"). It's ambiguous when talking about the phone and the PC at the same time. When saying "the SyncML client config", is that a config on the phone (because it is the client) or on the PC (because we set up a config for the client there)? The line in my ASCII diagram goes from "originating config" to "peer config". Perhaps that wasn't obvious when viewing with variable width font? That it is also the config of a SyncML server is an implementation detail. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

I've rebuilt my howto using the existing terminology, making only two changes: 1. I've used 'datastore' in place of 'data source' 2. I've used 'store-access' instead of 'unshared' for per-peer-source properties. Other than that, the howto does without the explanation using the syncml heritage, using only the minimal amount of concepts to get sync with EAS going. The purpose of this HOWTO is not to explain all concepts and all possibilities of SyncEvolution, and so it omits many of them; its purpose is to provide an understanding of how SynceEvolution works, broadly, so that can be used as a springboard to understand more complex concepts and use-cases (which I will describe in separate documents, probably re-writing existing howto's as I am doing so). The updated document can be found at http://reichenhack.github.io/syncevolution.html, the full text is pasted below: If you're getting into SyncEvolution, there are a few concepts that are crucial to understand. hostThe device or computer that SyncEvolution runs on.peerA peer is one of the counterparts in a data exchange. This can be a device (like a phone), a server (like Google) or even the host itself (useful for synchronizing two different databases).datastoreEach peer hosts one or more datastores holding either memos, events, contacts or tasks) that can be synchronized (Google for example offers Calendar and Contacts). This used to be called a source config ordata source, and you will still see many documents referring to it by one of these terms.backendA datastore is implemented by a SyncEvolution backend. The backend describes what storage technology is used for that datastore; it does not specify where that datastore lives. Some backends provide access to data outside of the host itself (CalDAV and CardDAV, ActiveSync).sync-configA sync-config defines a peer, implemented by SyncEvolution. A sync-config combines information about the peer and the datastores it can access, and provides a full specification of how those database are to be accessed during sync; how to locate the database, credentials to be used, etc.contextSync-configs and datastores are defined inside an configuration context which bundles them together. Sync-configs can only access datastores that are defined in the same context as the sync-config. There is always a default context that is used if none is specified explicitly. Let's set up a simple sync using just these concepts. As SyncEvolution is at heart a SyncML synchronization engine, the simplest case describes everything in terms of SyncEvolution as a SyncML client, talking to a remote SyncML server. I will set up a file-based datastore called local-contacts, and a file-based datastore called local-calendar, which I want to sync with a remote Horde SyncML server at https://horde.example.com/rpc.php. First, I set up the datastore for my contacts. Each datastore and sync-config must live in a context, in this case, as this data lives on my own PC, I call that context local: This means: on datastore local-contacts in context local, set the following properties: backend, database and databaseFormat. I did not explicitly create the datastore or the contexts; both are created when first referenced, and updated if they already exist. I also did not explicitly tell SyncEvolution that we're setting up a datastore (rather than a sync-config, which we'll set up with a similar command); SyncEvolution knows that we're configuring a datastore because we are passing it properties that are relevant to datastores. Likewise, I set up my calendar datastore: Next, I need to set up the local peer which will contact the remote SyncML server later by setting up a sync-config. This sync-config specifies, for example, the location of the server to sync with: Which tells SyncEvolution that I want to set up a sync-config (implied by the syncURL property being set) named Horde in context local, and that I want to sync the two datastores local-calendar and local-contacts using that sync-config. If you're setting up a sync-config, the first non-assignment on the command line is the name of the sync-config and the context in which it is to be created (Horde@local); all following non-assignments (local-calendar, local-contacts) are the datastores it will synchronize. SyncEvolution knows these must live in the local context, as sync-configs can only access datastores in its own context. We still need to tell SyncEvolution which direction we want to sync these datastores, so we set store-access ("unshared") properties on the sync-config: Which means "if I start a sync using sync-config Horde from context local, offer datastore local-calendar/local-contacts to the remote server, each for two-way sync". Now we should be all set to sync. The first time, it is required to do a slow sync: Which will sync all data-stores that have been tied to this sync-config using store-access properties. After this first time, you can do an incremental (much faster) sync with Synchronization beyond SyncML Synchronizing between a remote SyncML server and a local file-based store (or a local evolution datastore -- hence the name SyncEvolution) is not all that SyncEvolution can do. It can also synchronize with "server-based" datastores, such as Exchange (through activesync, aka EAS) or WebDAV-based servers that implement the CalDAV and/or CardDAV protocols. EAS and WebDAV are not considered sync engine technologies within the scope of SyncEvolution, but data storage engines. While it would be in principle possible to pair up server-based backends, for simplicity and debuggability, we set up a hub-and-spoke system with the hub a simple backend such as the file or eds backends. The hub will be SyncEvolution sync-config set up like the scenario above. We shall leave the setup from the first section in place, but add an Exchange server (which understands both contacts and calendars) to sync our datastores with. Since SyncEvolution doesn't consider EAS to be a sync peer but a datastore, we will need to set up a peer through which we can access EAS. Such a peer is called a target-config in SyncEvolution terms. We set up this peer by setting up a sync-config; this sync-config must be called target-config for SyncEvolution to recognize it as such. Exchange activesync EAS based stores take a little more tending to than file-based stores, but nothing beyond some mild head-against-concrete bashing. For EAS, you will need to have activesyncd installed in addition to syncevolution, and you'll need to store some configuration data outside SyncEvolution where activesyncd can find it: your username/password for the EAS server, and the activesync URL. These can be set up as follows: The EAS URL will usually look something like "https://owa.example.com/Microsoft-Server-ActiveSync"; if you don't know it, head over to https://testconnectivity.microsoft.com/, select tab Exchange Server, option Automatic detection of Exchange ActiveSync and let it discover the settings; in the resulting page, expand everything, and search for <Name>; in it will be the full EAS url. After we've set this up, we can go back to SyncEvolution. First we configure the contacts datastore: This means: in context exchange, create a datastore exchange-contacts (SyncEvolution again knows that we're configuring a datastore because we are passing it properties that are relevant to datastores), and on that datastore set the following properties: backend, database. In this case, we do not need to set up databaseFormat, since the backend eas-contacts can only store contacts. The database in this case is a little different than the file stores we set up above, since it is not a full specification of where to find your EAS contacts; it is instead resolved from at the moment SyncEvolution logs on. The name of your contacts folder can be seen in Outlook, or you can list the available folders using SyncEvolution: We set up our EAS calendar store in the same way: At this point we have two datastores in the exchange context, named exchange-contacts and exchange-calendar, but we still need to set up the peer by setting up a sync-config, which, as noted before, must be named target-config. We don't need to set a password, since we've already handed that to activesyncd, but SyncEvolution does need to know which activesyncd user to use, so <your exchange email address> is passed as a matching key; hence the <your exchange email address> in the gconftool-2 invocations. It is here where the database property from the datastore becomes fully meaningful in combination with the data we stuffed into activesyncd; there is no meaningful direct URL to any Exchange folder when it comes to activesync. In principle this could have been handled with databaseUser and databasePassword, but as activesyncd is a different software component, this probably got complex quickly. Giving the target-config something to talk about Now we still need to tell SyncEvolution which datastores can be synced through the target-config. These are specified as store-access properties like we did before: Which means "if peer target-config in context exchange is synced, it has datastores exchange-calendar/exchange-contacts available" We're setting sync=none here because it is a store-access property, which has the side effect of setting up store-access for the target-config. We could have used any value for sync, as its value is ignored; the sync value of the peer for the target-config takes precedence. Pairing the peers Now that we have our target-config set up, it's time to pair it up with a sync-config in the local context. For this, we'll need to create a new sync-config, and with it, the required store-access properties. First the sync-config. which means: create sync-config ExchangeSync in context local, and give it syncURL local://@Exchange. I could have named my client endpoint Exchange without any problem (and it would indeed be common practice to do it this way), just as I didn't have to use exchange-calendar or local-calendar but could have just used calendar for both (as they live in different contexts), but I want to distinguish between these two different things (context and sync-config) explicitly for clarity. As for the peerIsClient -- it is just required at this point. The URL is special, and we will get to that in a bit. Next, store-access for the peer: Which means: When I sync ExchangeSync@local, perform a two-way sync between local-contacts from context local with exchange-contacts from context exchange; we know exchange-contacts lives in context exchange because the syncURL from ExchangeSync (local://@Exchange) tells us that we're syncing with the target-config in that context. Likewise, perform a two-way sync between local-calendar from context local with exchange-calendar from context exchange. Finally, the sync And that was all she wrote. Just like the simple example, we must first perform a slow sync Which will sync all datastores associated with ExchangeSync@local. After this first time, you can do an incremental (much faster) sync with And that, was that.