2:50 p.m.
Hello!
Let's think about our configuration handling and how we are going to
deal with multiple peers.
Right now, we have one set of configuration files per server. Each
server synchronizes against its own set of local data sources.
In the future, we are not necessarily syncing against an HTTP server.
"Peer" is a better word, because it covers clients and servers.
Multiple peers want to synchronize against the same local data. That's
where the current configuration system becomes awkward. It forces us to
duplicate certain configuration settings (local data sources, logging).
When looking for local database dumps for restore, it would be necessary
to check multiple configurations.
Therefore I suggest that we change our configuration layout. Instead of
a "server configuration" directory, we start with a "host
configuration" (better names welcome!). A host configuration has global
settings pertaining to all syncs done with that configuration (logging).
It has a set of local source configs (backend and database selection).
Then it has a set of peers using this particular host configuration.
Each peer has its own peer-specific settings (credentials, URL and
transport) and source-specific settings (URI, sync mode). It might even
have more than one way of contacting it (Bluetooth, USB), although how
to handle this is a bit uncertain right now. My intention is to deal
with this inside the per-server config file instead of creating more
config files.
Because there hardly ever is a need to have more than one host
configuration, we could create one named "default" and use that unless
specified otherwise.
The layout would look like this:
.config/syncevolution/
default/
config.ini host config
.synthesis shared (!) Synthesis binfiles
sources/
addressbook|calendar|.../
config.ini local source config
peers/
scheduleworld/
config.ini peer configuration
sources/
addressbook/
config.ini peer-specific source config
.other.ini per peer and per source storage for backend
For each config.ini there's also an .internal.ini which is not exposed
via command line or D-Bus API. The exact location of each property needs
to be defined.
The advantage of this layout is that when a SyncML server contacts us
via OBEX, we can select the "default" host config, check whether the
server is known already, otherwise create a new peer config using the
"default" set of local data sources. In the current scheme it is unclear
where the host configuration is supposed to come from.
In the new scheme the Synthesis binfiles are shared between peers, as
intended by Synthesis. This is not necessary for us at the moment, but
some of the binfile features (simplified change tracking via a single
"updated" flag in the local database) might be useful for other
backends.
The drawback of this revised config layout is the increased complexity.
Instead of two sets of properties as in the current config (sync
properties and source properties) users of the command line and the
D-Bus API would have to deal with four sets, even in the simple use case
of a single peer.
Here's an idea how this could be avoided: when selecting a configuration
for a server, create a "view" on the more complex configuration layout
which looks exactly like it did so far. The "sync properties" include
the ones that are identical for all peers and the peer specific ones.
Likewise for each source. Setting the properties puts the properties
into the right config.ini automatically.
The D-Bus API as discussed so far and the command line interface don't
have to be changed. Configs are selected via the peer name, which is
typically unique. If not, then appending a @<host config name> makes it
unique and can be used to create new host configs.
The empty string as peer name selects only the host configuration, which
is useful for local operations (restore, status) or accessing all sync
reports.
In that scheme it may be a bit unexpected that, for example, updating
the logdir setting of the "scheduleworld" server affects the config of
the "funambol" server. When using the SetConfig(update=false) D-Bus API
to remove a config, clients must not remove the host's source
properties, because they might still be needed by other peers. I think
this is acceptable, considering the advantages of the otherwise simpler
API.
When asking for a peer template, that template should already be
populated with existing host properties, so that it can be written back
without unintentionally overwriting those.
Speaking of templates, the list of available templates might also
contain paired Bluetooth devices which are known to support SyncML. Then
(almost) the same UI as for adding an HTTP server could also be used for
devices.
Another drawback of changing the configuration layout is the question of
migrating old configs and changing back and forth between new and old
SyncEvolution releases. Ideally migration should be automatic and still
allow going back to an older SyncEvolution seamlessly.
With two tricks this might be feasible:
1. move existing properties into the new layout and *duplicate* the
host properties inside the per-server config.ini
2. recreate the traditional layout with symlinks
So we would end up with:
.config/syncevolution/scheduleworld/
.synthesis/ -> ~/.config/default/.synthesis
config.ini -> .../peers/scheduleworld/config.ini
sources/ -> .../peers/scheduleworld/sources/
config.ini files are updated by creating a new file and then renaming it
to config.ini. This replaces the config.ini symlink. Also, if a host's
source property is changed using an old SyncEvolution, the change won't
be noticed by a new SyncEvolution because although the changed
config.ini is read, the newer SyncEvolution doesn't take the host
properties from there. I think we don't have to support this.
The automatic migration also doesn't work if the user has multiple
server configs because we cannot "join" different .synthesis binfile
dirs. In such a situation the user has to force a migration and be
prepared for slow syncs. Without such a forced migration the new
SyncEvolution should continue using the old layout. This is easily
implemented by using the same config node twice.
Finally, the instructions for synchronizing multiple ScheduleWorld
calendars have to be updated. When setting up the second scheduleworld
config, the name of the "calendar" source is already in use and
something else must be used to avoid overwriting the "evolutionsource"
property in the existing source.
How does this sound so far?
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
11:30 p.m.
Therefore I suggest that we change our configuration layout. Instead
of
a "server configuration" directory, we start with a "host
configuration" (better names welcome!).
I'd like to use 'global
configuration' or something else since it may
not only be 'host's settings'.
Finally, the instructions for synchronizing multiple ScheduleWorld
calendars have to be updated. When setting up the second scheduleworld config, the name of
the "calendar" source is already in use and something
else must be used to avoid overwriting the "evolutionsource"
property in the existing source.
What does 'the second scheduleword config'
mean?
I don't quite understand how to handle multiple configs for one syncML server in new
scheme.
If similar to previous, two configs are in 2 different directories, so why existing the
above issue you raise?
Cheers,
Yongsheng
-----Original Message-----
From: syncevolution-bounces(a)syncevolution.org
[mailto:syncevolution-bounces@syncevolution.org] On Behalf Of Patrick Ohly
Sent: Sunday, September 13, 2009 3:50 AM
To: SyncEvolution
Subject: [SyncEvolution] configuration + multiple peers
Hello!
Let's think about our configuration handling and how we are going to
deal with multiple peers.
Right now, we have one set of configuration files per server. Each
server synchronizes against its own set of local data sources.
In the future, we are not necessarily syncing against an HTTP server.
"Peer" is a better word, because it covers clients and servers.
Multiple peers want to synchronize against the same local data. That's
where the current configuration system becomes awkward. It forces us to
duplicate certain configuration settings (local data sources, logging).
When looking for local database dumps for restore, it would be necessary
to check multiple configurations.
Therefore I suggest that we change our configuration layout. Instead of
a "server configuration" directory, we start with a "host
configuration" (better names welcome!). A host configuration has global
settings pertaining to all syncs done with that configuration (logging).
It has a set of local source configs (backend and database selection).
Then it has a set of peers using this particular host configuration.
Each peer has its own peer-specific settings (credentials, URL and
transport) and source-specific settings (URI, sync mode). It might even
have more than one way of contacting it (Bluetooth, USB), although how
to handle this is a bit uncertain right now. My intention is to deal
with this inside the per-server config file instead of creating more
config files.
Because there hardly ever is a need to have more than one host
configuration, we could create one named "default" and use that unless
specified otherwise.
The layout would look like this:
.config/syncevolution/
default/
config.ini host config
.synthesis shared (!) Synthesis binfiles
sources/
addressbook|calendar|.../
config.ini local source config
peers/
scheduleworld/
config.ini peer configuration
sources/
addressbook/
config.ini peer-specific source config
.other.ini per peer and per source storage for backend
For each config.ini there's also an .internal.ini which is not exposed
via command line or D-Bus API. The exact location of each property needs
to be defined.
The advantage of this layout is that when a SyncML server contacts us
via OBEX, we can select the "default" host config, check whether the
server is known already, otherwise create a new peer config using the
"default" set of local data sources. In the current scheme it is unclear
where the host configuration is supposed to come from.
In the new scheme the Synthesis binfiles are shared between peers, as
intended by Synthesis. This is not necessary for us at the moment, but
some of the binfile features (simplified change tracking via a single
"updated" flag in the local database) might be useful for other
backends.
The drawback of this revised config layout is the increased complexity.
Instead of two sets of properties as in the current config (sync
properties and source properties) users of the command line and the
D-Bus API would have to deal with four sets, even in the simple use case
of a single peer.
Here's an idea how this could be avoided: when selecting a configuration
for a server, create a "view" on the more complex configuration layout
which looks exactly like it did so far. The "sync properties" include
the ones that are identical for all peers and the peer specific ones.
Likewise for each source. Setting the properties puts the properties
into the right config.ini automatically.
The D-Bus API as discussed so far and the command line interface don't
have to be changed. Configs are selected via the peer name, which is
typically unique. If not, then appending a @<host config name> makes it
unique and can be used to create new host configs.
The empty string as peer name selects only the host configuration, which
is useful for local operations (restore, status) or accessing all sync
reports.
In that scheme it may be a bit unexpected that, for example, updating
the logdir setting of the "scheduleworld" server affects the config of
the "funambol" server. When using the SetConfig(update=false) D-Bus API
to remove a config, clients must not remove the host's source
properties, because they might still be needed by other peers. I think
this is acceptable, considering the advantages of the otherwise simpler
API.
When asking for a peer template, that template should already be
populated with existing host properties, so that it can be written back
without unintentionally overwriting those.
Speaking of templates, the list of available templates might also
contain paired Bluetooth devices which are known to support SyncML. Then
(almost) the same UI as for adding an HTTP server could also be used for
devices.
Another drawback of changing the configuration layout is the question of
migrating old configs and changing back and forth between new and old
SyncEvolution releases. Ideally migration should be automatic and still
allow going back to an older SyncEvolution seamlessly.
With two tricks this might be feasible:
1. move existing properties into the new layout and *duplicate* the
host properties inside the per-server config.ini
2. recreate the traditional layout with symlinks
So we would end up with:
.config/syncevolution/scheduleworld/
.synthesis/ -> ~/.config/default/.synthesis
config.ini -> .../peers/scheduleworld/config.ini
sources/ -> .../peers/scheduleworld/sources/
config.ini files are updated by creating a new file and then renaming it
to config.ini. This replaces the config.ini symlink. Also, if a host's
source property is changed using an old SyncEvolution, the change won't
be noticed by a new SyncEvolution because although the changed
config.ini is read, the newer SyncEvolution doesn't take the host
properties from there. I think we don't have to support this.
The automatic migration also doesn't work if the user has multiple
server configs because we cannot "join" different .synthesis binfile
dirs. In such a situation the user has to force a migration and be
prepared for slow syncs. Without such a forced migration the new
SyncEvolution should continue using the old layout. This is easily
implemented by using the same config node twice.
Finally, the instructions for synchronizing multiple ScheduleWorld
calendars have to be updated. When setting up the second scheduleworld
config, the name of the "calendar" source is already in use and
something else must be used to avoid overwriting the "evolutionsource"
property in the existing source.
How does this sound so far?
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
_______________________________________________
SyncEvolution mailing list
SyncEvolution(a)syncevolution.org
http://lists.syncevolution.org/listinfo/syncevolution
6:47 a.m.
On Mon, 2009-09-14 at 05:30 +0100, Zhu, Yongsheng wrote:
> Therefore I suggest that we change our configuration layout.
Instead of
> a "server configuration" directory, we start with a "host
> configuration" (better names welcome!).
I'd like to use 'global configuration' or something else since it may
not only be 'host's settings'.
Suppose we add even "more global" settings:
.config/syncevolution/
config.ini
We need config.ini for the "active server", eh, "active peer" setting
that is to be shared between sync-ui and Genesis.
I think "global configuration" is more suitable for these settings. In
that case we still need a better name for
the .config/syncevolution/default set of config files.
How about "source set configuration"? What all of the config files
inside the "default" subdirectory have in common is that they are
related to the same set of local data sources.
> Finally, the instructions for synchronizing multiple
ScheduleWorld calendars have to be updated. When setting up the second scheduleworld
config, the name of the "calendar" source is already in use and something
> else must be used to avoid overwriting the "evolutionsource"
> property in the existing source.
What does 'the second scheduleword config' mean?
The ScheduleWorld Wiki says (quoting from memory):
* create normal "scheduleworld" config
* create second config with "syncevolution --configure
--sync-property
syncURL=http://sync.scheduleworld.com/funambol/ds?cal=<id>
--source-property evolutionSource=<home calendar> --template
scheduleworld_home calendar
In the old scheme, this creates two server configs with different
syncURL and two different "calendar/config.ini" files, accessing
different local data.
In the new scheme, there is only one
"calendar/config.ini:evolutionSource", so the second invocation of
syncevolution changes the local data accessed by the normal
"scheduleworld" config, which is both unexpected and (in this special
case) undesirable.
I don't quite understand how to handle multiple configs for one
syncML server in new scheme.
The second configuration must use a different source name. Then the
normal "scheduleworld" config uses the default "calendar", and the
second "scheduleworld_home" the other.
Currently we don't have a good method of cloning source settings when
using the command line. Should we introduce a "--configure <server>
<source name>=<other source name>" syntax for this purpose?
If similar to previous, two configs are in 2 different directories,
so why existing the above issue you raise?
I hope it is more clear now.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
8:46 p.m.
How about "source set configuration"? What all of the
config files
inside the "default" subdirectory have in common is that they are
related to the same set of local data sources.
I see. Ok for me.
In the new scheme, there is only one
"calendar/config.ini:evolutionSource", so the second invocation of
syncevolution changes the local data accessed by the normal
"scheduleworld" config, which is both unexpected and (in this special
case) undesirable.
So we combine these 2 calendar settings into one server config, what's the main
concern
to make this different from the old scheme?
For the scenario that there are many accounts for one server, the new scheme
remains the same as the old scheme to create one server config for each account, right?
Cheers,
Yongsheng
-----Original Message-----
From: Ohly, Patrick
Sent: Monday, September 14, 2009 7:47 PM
To: Zhu, Yongsheng
Cc: SyncEvolution
Subject: RE: [SyncEvolution] configuration + multiple peers
On Mon, 2009-09-14 at 05:30 +0100, Zhu, Yongsheng wrote:
> Therefore I suggest that we change our configuration layout.
Instead of
> a "server configuration" directory, we start with a "host
> configuration" (better names welcome!).
I'd like to use 'global configuration' or something else since it may
not only be 'host's settings'.
Suppose we add even "more global" settings:
.config/syncevolution/
config.ini
We need config.ini for the "active server", eh, "active peer"
setting
that is to be shared between sync-ui and Genesis.
I think "global configuration" is more suitable for these settings. In
that case we still need a better name for
the .config/syncevolution/default set of config files.
How about "source set configuration"? What all of the config files
inside the "default" subdirectory have in common is that they are
related to the same set of local data sources.
> Finally, the instructions for synchronizing multiple
ScheduleWorld calendars have to be updated. When setting up the second scheduleworld
config, the name of the "calendar" source is already in use and something
> else must be used to avoid overwriting the "evolutionsource"
> property in the existing source.
What does 'the second scheduleword config' mean?
The ScheduleWorld Wiki says (quoting from memory):
* create normal "scheduleworld" config
* create second config with "syncevolution --configure
--sync-property
syncURL=http://sync.scheduleworld.com/funambol/ds?cal=<id>
--source-property evolutionSource=<home calendar> --template
scheduleworld_home calendar
In the old scheme, this creates two server configs with different
syncURL and two different "calendar/config.ini" files, accessing
different local data.
In the new scheme, there is only one
"calendar/config.ini:evolutionSource", so the second invocation of
syncevolution changes the local data accessed by the normal
"scheduleworld" config, which is both unexpected and (in this special
case) undesirable.
I don't quite understand how to handle multiple configs for one
syncML server in new scheme.
The second configuration must use a different source name. Then the
normal "scheduleworld" config uses the default "calendar", and the
second "scheduleworld_home" the other.
Currently we don't have a good method of cloning source settings when
using the command line. Should we introduce a "--configure <server>
<source name>=<other source name>" syntax for this purpose?
If similar to previous, two configs are in 2 different directories,
so why existing the above issue you raise?
I hope it is more clear now.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
1:49 a.m.
On Tue, 2009-09-15 at 02:46 +0100, Zhu, Yongsheng wrote:
> In the new scheme, there is only one
> "calendar/config.ini:evolutionSource", so the second invocation of
> syncevolution changes the local data accessed by the normal
> "scheduleworld" config, which is both unexpected and (in this special
> case) undesirable.
So we combine these 2 calendar settings into one server config, what's the main
concern
to make this different from the old scheme?
The disadvantages of the old scheme are:
* Finding all old sessions involving a certain local data source
is harder.
* Settings are duplicated.
* When we as a SyncML client are contacted by a server for the
first time, it is unclear which local source and logging
configuration we are supposed to use.
I'm wondering - do I overestimate the disadvantages of the old scheme?
Should we stick to it and work around its limitations?
For the scenario that there are many accounts for one server, the new
scheme
remains the same as the old scheme to create one server config for each account, right?
Yes. But this is unlikely, isn't it?
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
10:28 a.m.
Hello!
In #5188 ("Outgoing obex connection for SyncEvolution") Congwu and I
touched on some configuration related aspects. In particular we talked
about how to specify multiple ways of contacting a peer. We came to the
following conclusion:
1. add more peer settings like syncURL, using different names for
each possible transport
2. have one property which chooses the transport which is to be
used; by default it should be unset, which means that
SyncEvolution must pick a transport automatically (mostly based
on availability)
There's one problem with that: how does the GUI create such a
configuration correctly when it only knows that "Bluetooth device with
MAC address xyz is available for sync"? There might be a peer
configuration for this device already, using USB as transport. But
because the only common element between the two peers (the SyncML Device
ID) isn't available for Bluetooth at this point, the existing peer
configuration cannot be identified automatically.
Do we rely on the user to add the Bluetooth transport to the right
existing configuration? If he gets that wrong, he will end up with two
different local sets of meta data (source/addressbook/.other.ini), which
will break syncs when switching back and forth between the two
transports.
I think we should design our new configuration layout so that it is
tolerant of this. This means that the location of the .other.ini files
can only be determined once the peer's Device ID is known (internal
backend API change).
In #5188 I mentioned this idea (I said .internal.ini there, but I really
meant .other.ini, the node which is used for change tracking):
.config/syncevolution/default/.devices/<device ID>/
.other.ini
sources/addressbook/
.other.ini
We don't need .other.ini for the device. Placing the
per-source .other.ini in a place which is different from what we do now
is problematic for the backwards compatibility mode, besides making the
simple case (single-transport peers) more complex.
Therefore I suggest to keep the .other.ini in the peer/*/sources/*
directories, with one minor twist: when we learn about the Device ID of
a peer, we check which of the currently configured peers has the same ID
and .other.ini files. If we find one, we use that sources directory for
the .other.ini.
This creates another special case for removing a peer configuration: if
another peer has the same Device ID and sources, we must move
the .other.ini files to that peer, because they are still needed.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
10:47 a.m.
Hello!
I have summarized this discussion in a separate page:
http://syncevolution.org/development/configuration-handling
The goal is to capture the final result of our discussion (please
check!) before working on the implementation (which I will do next) and
document the outcome.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
4116
days inactive
4155
days old
syncevolution@synevolution.org
6 comments
2 participants
participants (2)
-
Patrick Ohly -
Zhu, Yongsheng