4:59 p.m.
http://bugzilla.moblin.org/show_bug.cgi?id=4611
Summary: join/dejoin different sources
Classification: Moblin Projects
Product: SyncEvolution
Version: upstream
Platform: Generic
OS/Version: Other
Status: ASSIGNED
Severity: normal
Priority: P2
Component: * Feature Request
AssignedTo: syncevolution(a)lists.intel.com
ReportedBy: patrick.ohly(a)intel.com
CC: syncevolution(a)lists.intel.com
It is a very common complaint that SyncML in most installations only supports
one calendar, address book, task list, memo list per device. This is not a
limitation of the protocol, more like the lowest common denominator that is
typically supported.
Our own sync UI for Moblin follows the same model, although SyncEvolution
itself could handle more than one instance of each data category if the peer
did (evolutionSource config option).
For those cases where the peer doesn't, it would be nice if SyncEvolution could
join data from multiple databases if the peer supports only one database, then
when receiving updates back correctly sorted them into the the right database.
Here's a user giving more background information:
http://article.gmane.org/gmane.comp.mobile.funambol.user/11522
ScheduleWorld has such a feature for calendars, based on the CATEGORY property.
We could adapt a similar system and perhaps even make it so that n:n calendar
synchronization within the same sync session becomes possible.
Another use case for such a join/dejoin are devices which exchange events and
tasks via one single URI. For our direct-sync work we need to support that.
The Synthesis engine has the concept of a <superstore>: it automatically merges
and splits based on some configurable property. Currently this is used to put
VEVENT into one <datastore>, VTODO into another. Perhaps this can be configured
to use CATEGORY, too.
For joining items, make sure that a unique CATEGORY is set per database. For
receiving items or updates back, check for the presence of those categories and
store accordingly, with one database as default for new items which don't have
any of these categories.
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
8:47 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
pohly <patrick.ohly(a)intel.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Blocks| |3182
AssignedTo|syncevolution(a)lists.intel.c |congwu.chen(a)intel.com
|om |
Target Milestone|future |1.0
--- Comment #1 from pohly <patrick.ohly(a)intel.com> 2009-10-29 00:47:48 ---
Besides joining/dejoining different data sources of the same content type based
on some item property like CATEGORY, a much more urgent (and simpler) use case
is accessing events and tasks under one URI.
Apparently ovi.com (bug #3182) expects its clients to support this and thus
only offers one URI for both kinds of items. It's debatable whether the server
shouldn't better support separate URIs (because there are clients which only
support that).
Our own server mode definitely has to support a joined event/task URI.
I'm not creating two issues for this because the implementation of the two use
cases clearly is related. For tracking purposes: joined events/tasks are P2 for
1.0, the rest can wait.
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
6:01 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #2 from Chen Congwu <congwu.chen(a)intel.com> 2009-11-19 21:01:21 PST
---
(In reply to comment #1)
And to sync with Nokia phones, we also need this feature.
FileSyncSource can handle both sync types but EvolutionSyncSource not.
We need to configure and join Evolution events and todos.
My suggestion on the configuration file change:
Add a "super calendar" = "super evolution calendar" special backend,
which will
map to (calendar+todo), the format used is vcard2.1 and ical2.0 by default, but
user can change it:
super calendar:vcard3.0+ical2.0
Besides joining/dejoining different data sources of the same content
type based
on some item property like CATEGORY, a much more urgent (and simpler) use case
is accessing events and tasks under one URI.
Apparently ovi.com (bug #3182) expects its clients to support this and thus
only offers one URI for both kinds of items. It's debatable whether the server
shouldn't better support separate URIs (because there are clients which only
support that).
Our own server mode definitely has to support a joined event/task URI.
I'm not creating two issues for this because the implementation of the two use
cases clearly is related. For tracking purposes: joined events/tasks are P2 for
1.0, the rest can wait.
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
9:15 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #3 from pohly <patrick.ohly(a)intel.com> 2009-11-20 00:15:22 PST ---
(In reply to comment #2)
And to sync with Nokia phones, we also need this feature.
FileSyncSource can handle both sync types but EvolutionSyncSource not.
We need to configure and join Evolution events and todos.
My suggestion on the configuration file change:
Add a "super calendar" = "super evolution calendar" special backend,
The solution of this problem should be independent of the Evolution backends.
The general task is: "combine two (or more) real data sources into one virtual
one".
I'm not sure whether we really need to write a backend. At least for the "tasks
+ events" use case, the Synthesis engine already has a solution. We just need
to activate it somehow.
which will
map to (calendar+todo), the format used is vcard2.1 and ical2.0 by default,
Why vcard2.1?
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
9:30 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #4 from Chen Congwu <congwu.chen(a)intel.com> 2009-11-20 00:30:47 PST
---
(In reply to comment #3)
(In reply to comment #2)
The solution of this problem should be independent of the Evolution backends.
The general task is: "combine two (or more) real data sources into one virtual
one".
Now comes this solution:
The configuration of type properties is defined as:
backend1+backend2+...backendN:format1+format2_...formatM (0=<M<=N)
The information we lost is: the datastore name is the same as backend name in
such joined case which I think is OK.
Another dirty point is the "forced" property in format, the original single
m_forced property is no longer working.
We will maintain a list of all joined datastores, which typically like:
superdatastore1: backend1, backend2,..backendN
This is used later to generate the "superdatastore" configuration.
I'm not sure whether we really need to write a backend. At least
for the "tasks
+ events" use case, the Synthesis engine already has a solution. We just need
to activate it somehow.
Now comes the specific part, the generation of
"superdatastore" element is
different case by case (The filtering criteria). At this time I will always
generate a "super calendar" datastore for the calender +event case to work.
> which will
> map to (calendar+todo), the format used is vcard2.1 and ical2.0 by default,
Why vcard2.1?
Because this is the situation in non-joined case, I just don't want to change.
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
10:42 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #5 from pohly <patrick.ohly(a)intel.com> 2009-11-20 01:42:16 PST ---
(In reply to comment #4)
(In reply to comment #3)
> (In reply to comment #2)
> The solution of this problem should be independent of the Evolution backends.
> The general task is: "combine two (or more) real data sources into one virtual
> one".
Now comes this solution:
The configuration of type properties is defined as:
backend1+backend2+...backendN:format1+format2_...formatM (0=<M<=N)
I assume that with "backend" you mean something like "Evolution
Contacts". When
that backend gets invoked, how does it know which local database it is supposed
to access?
I suggest to solve this problem by referencing other *source* configurations
instead of *backends*. Then these other source configurations are defined
normally. A sync where a supersource is active is treated like a sync where
each of the involved normal sources is active (database dumps, reporting, ...).
The configuration could look like this:
[calendar+todo]
type = virtual:text/calendar
evolutionsource = calendar,todo
It's a bit unfortunate that I followed Evolution's example with the
"calendar"
name. Other products use "events", "tasks" for what we call
"calendar" and
"todo" and "calendar" for what we have to call
"calendar+todo" (just a
proposal, other ideas are welcome).
Note that only data sources with compatible data formats can be merged, as far
as I understand it. Even if the Synthesis engine supports it, I don't think we
have use cases which require it, so we can impose this limitation ourselves and
thus keep the data format selection via [calendar+todo] working as usual.
The information we lost is: the datastore name is the same as backend
name in
such joined case which I think is OK.
Another dirty point is the "forced" property in format, the original single
m_forced property is no longer working.
I think these problems don't occur with the approach above.
Now comes the specific part, the generation of
"superdatastore" element is
different case by case (The filtering criteria). At this time I will always
generate a "super calendar" datastore for the calender +event case to work.
Agreed. Later we can add additional configuration parameters which control the
advanced use cases.
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
11:08 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #6 from Chen Congwu <congwu.chen(a)intel.com> 2009-11-20 02:08:16 PST
---
(In reply to comment #5)
(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > The solution of this problem should be independent of the Evolution backends.
> > The general task is: "combine two (or more) real data sources into one
virtual
> > one".
> Now comes this solution:
> The configuration of type properties is defined as:
> backend1+backend2+...backendN:format1+format2_...formatM (0=<M<=N)
I assume that with "backend" you mean something like "Evolution
Contacts". When
that backend gets invoked, how does it know which local database it is supposed
to access?
That's the part I missed.
I suggest to solve this problem by referencing other *source*
configurations
instead of *backends*. Then these other source configurations are defined
normally. A sync where a supersource is active is treated like a sync where
each of the involved normal sources is active (database dumps, reporting, ...).
The configuration could look like this:
[calendar+todo]
type = virtual:text/calendar
evolutionsource = calendar,todo
Do you mean add another dummy source folder
reference other sources to be
joined ? This simplifies my problem but adding duplicated configuration(in this
case calendar, todo, and calendar+todo are mostly duplicated). This is not very
nice and may confuse user. For example is it meaningful to fire a sync with a
Nokia phone asking two-way sync for calendar and slow-sync for todo? Mostly
this does not work.
> The information we lost is: the datastore name is the same as
backend name in
> such joined case which I think is OK.
> Another dirty point is the "forced" property in format, the original
single
> m_forced property is no longer working.
I think these problems don't occur with the approach above.
I agree with you
and actually I already changed my mind during testing.
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
11:23 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #7 from pohly <patrick.ohly(a)intel.com> 2009-11-20 02:23:18 PST ---
(In reply to comment #6)
(In reply to comment #5)
> I suggest to solve this problem by referencing other *source* configurations
> instead of *backends*. Then these other source configurations are defined
> normally. A sync where a supersource is active is treated like a sync where
> each of the involved normal sources is active (database dumps, reporting, ...).
>
> The configuration could look like this:
> [calendar+todo]
> type = virtual:text/calendar
> evolutionsource = calendar,todo
Do you mean add another dummy source folder reference other sources to be
joined ?
Yes.
This simplifies my problem but adding duplicated configuration(in
this
case calendar, todo, and calendar+todo are mostly duplicated). This is not very
nice and may confuse user. For example is it meaningful to fire a sync with a
Nokia phone asking two-way sync for calendar and slow-sync for todo? Mostly
this does not work.
[calendar+todo] is indeed mutually exclusive with both [calendar] and [todo].
This is something that users need to know. Depending on how we implement this
"virtual" backend, this semantic can be checked by SyncContext.
In the sync-UI, we can hide the complexity for the default case.
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
8:21 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #8 from Chen Congwu <congwu.chen(a)intel.com> 2009-11-22 23:21:22 PST
---
(In reply to comment #7)
(In reply to comment #6)
> (In reply to comment #5)
> > I suggest to solve this problem by referencing other *source* configurations
> > instead of *backends*. Then these other source configurations are defined
> > normally. A sync where a supersource is active is treated like a sync where
> > each of the involved normal sources is active (database dumps, reporting,
...).
Ok, I have pushed the fix to phone branch including several SAN generation
fixes to work with Nokia phone.
I ignore other properties of the virtual syncsource except "type" and
"evolution syncsource" to avoid inconsistency.
The format in the "type" field for a virtual syncsource must be presented,
because this may otherwise leads to inconsistency for different backends
providing different preferred formats.
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
12:15 p.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #9 from pohly <patrick.ohly(a)intel.com> 2009-11-23 03:15:31 PST ---
Commit message:
At this time, the configuration is fixed to work with "calendar+todo" case.
This means "only virtual sync sources which combine one calendar and one todo
are supported, in this order. In other words, a valid configuration is
evolutionsource=calendar,todo"
Is that correct?
Why was it necessary to change the ISEVENT check in the XML config? A bug in
the original example or in our copy? Please include such information in the
commit message. What I tend to do is to go through a diff while writing the
commit message; that way I notice everything which might require a comment.
+ "This field is also used to identify a virtual
\n"
+ "backend which is used to join several sub
backends \n"
+ "together, Nokia phone typlically needs
todo+calendar \n"
+ "virtual backends"
Mind the extra white space at line ends. I don't care about those much in
source code, but would like to keep the strings clean. I also suggest a
different text:
=>
-----------------------------
\n
A special "virtual" backend combines several other data sources and presents
them as one set of items to the peer. For example, Nokia phones typically
exchange tasks and events as part of one set of calendar items.\n
\n
Right now such a virtual backend is limited to combining one calendar source
with events and one task source. They have to be specified in the
"evolutionsource" property, typically as "evolutionsource =
calendar,todo".\n
\n
In all cases the format of this configuration is
<backend>[:format]
Different sources combined in one virtual source must have a common
representation. As with other backends, the preferred format can be influenced
via the "format" attribute.
-------------------------------
virtual:text/x-vcalendar - a virtual backend uses vCalendar 1.0 format\n"
=>
virtual:text/x-vcalendar - a virtual backend using vCalendar 1.0 format\n"
-------------------------------
Regarding the new "evolutionsource" parsing: are spaces around a comma allowed?
What if a user has chosen a sync source name which includes a comma? We need
some kind of escaping for this.
std::vector<VirtualSyncSource*> m_virtualDS;
Wouldn't it be better to store shared_ptrs in the vector? The code which uses
plain pointers is legacy code; the Funambol library expected an array of such
pointers.
sourceList.m_virtualDS.push_back (new VirtualSyncSource (params));
would leak the VirtualSyncSource instance if push_back throws an out-of-memory
exception.
+ }
+ else {
The normal formatting is "} else {".
You are not checking whether the virtual sync source is enabled. This has to be
added, because a virtual sync source configured for a SyncML client peer will
show up as inactive source for a SyncML server peer.
SE_THROW ("superdatastore now only supports events+tasks case");
=> the user has no idea what a "superdatastore" is. The error message must
refer to the concepts introduced for the "virtual data source" and include the
source name => vSource->throwError()
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
1:13 p.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #10 from Chen Congwu <congwu.chen(a)intel.com> 2009-11-23 04:13:21 PST
---
(In reply to comment #9)
Commit message:
At this time, the configuration is fixed to work with "calendar+todo" case.
This means "only virtual sync sources which combine one calendar
and one todo
are supported, in this order. In other words, a valid configuration is
evolutionsource=calendar,todo"
Is that correct?
Correct.
Why was it necessary to change the ISEVENT check in the XML config? A
bug in
the original example or in our copy? Please include such information in the
commit message.
That's not a bug. I need to add a field 'KIND' for
calendar+todo dispatch, this
filed also covers the original 'ISEVENT field'; therefore I changed it to use
'KIND' instead of 'ISEVENT'.
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
3:33 p.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #11 from pohly <patrick.ohly(a)intel.com> 2009-11-23 06:33:37 PST ---
(In reply to comment #10)
> Why was it necessary to change the ISEVENT check in the XML
config? A bug in
> the original example or in our copy? Please include such information in the
> commit message.
That's not a bug. I need to add a field 'KIND' for calendar+todo dispatch,
this
filed also covers the original 'ISEVENT field'; therefore I changed it to use
'KIND' instead of 'ISEVENT'.
The upstream libsynthesis syncserv_sample_config.xml still looks wrong to me:
it uses <dispatchfilter>F.KIND:=EVENT</dispatchfilter> but only defines
ISEVENT <subprofile name="VTODO" nummandatory="1"
showifselectedonly="yes"
field="ISEVENT" value="0">
Your solution solves this and should be reported to Synthesis. For the server
config we don't have a process in place to do this automatically, so an email
is due. Can you notify them?
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
4:20 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #12 from Chen Congwu <congwu.chen(a)intel.com> 2009-11-24 19:19:59 PST
---
(In reply to comment #9)
Commit message:
Mind the extra white space at line ends. I don't care about those
much in
source code, but would like to keep the strings clean. I also suggest a
different text:
Updated.
Regarding the new "evolutionsource" parsing: are spaces
around a comma allowed?
What if a user has chosen a sync source name which includes a comma? We need
some kind of escaping for this.
Right, I am adding "boost::trim" to strip the spaces and using "\" as
the
escape character to handle comma in source name problem.
Note I am not using SafeConfigNode because it is designed to be used for
internal properties which will is read and written only by SyncEvolution.
The "evolutionsource" is a public property which maybe set by user explicitly.
Therefore I am just adding escape capable decoding and update the documentation
accordingly.
Pushing to phone branch now.. Please have a look.
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
10:10 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
--- Comment #13 from pohly <patrick.ohly(a)intel.com> 2009-12-18 01:10:34 PST ---
(In reply to comment #12)
Pushing to phone branch now..
Merged. I think we have resolved the part we need for SyncEvolution 1.0. For
tracking purposes I am going to split this issue into different parts.
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
10:25 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
pohly <patrick.ohly(a)intel.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Depends on| |8752
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching someone on the CC list of the bug.
10:26 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
pohly <patrick.ohly(a)intel.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Blocks|3182 |
AssignedTo|congwu.chen(a)intel.com |syncevolution(a)lists.intel.c
| |om
Target Milestone|1.0 |future
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
10:30 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
pohly <patrick.ohly(a)intel.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Depends on| |8753
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
10:58 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
ZhangJingke <jingke.zhang(a)intel.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|ASSIGNED |RESOLVED
CC| |jingke.zhang(a)intel.com
Resolution| |MIGRATED
--- Comment #14 from ZhangJingke <jingke.zhang(a)intel.com> 2010-04-13 01:58:12 PST
---
This is moved to http://bugzilla.meego.com/show_bug.cgi?id=685
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
5:32 a.m.
New subject: [Bug 4611] join/dejoin different sources
http://bugzilla.moblin.org/show_bug.cgi?id=4611
Bug 4611 depends on bug 8753, which changed state.
Bug 8753 Summary: join/dejoin different sources: multiple calendars or address books
http://bugzilla.moblin.org/show_bug.cgi?id=8753
What |Old Value |New Value
----------------------------------------------------------------------------
Status|ASSIGNED |RESOLVED
Resolution| |MIGRATED
--
Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
4020
days inactive
4287
days old
syncevolution-issues@syncevolution.org
18 comments
1 participants
participants (1)
-
bugzilla＠moblin.org