3:12 p.m.
I have just noticed that events are being corrupted when syncing from
Exchange.
The problem is that when an event changes, EAS can send a "change"
notification, with only the fields that have changed, not the whole new
event. activesyncd then creates a partial (and invalid!) VEVENT with
only the changed fields.
Syncevolution then replaces the event with just the information from the
partial VEVENT, losing all the rest of the information.
To make it clearer, here is an extract from the EAS protocol message
sent when one attendee's status has to be updated for an existing event:
<Change>
<ServerId>1:2170</ServerId>
<ApplicationData>
<Attendees xmlns="Calendar:">
<Attendee>
<Attendee_Email>aaaa(a)example.com</Attendee_Email>
<Attendee_Name>aaaa(a)example.com</Attendee_Name>
<Attendee_Status>4</Attendee_Status>
<Attendee_Type>1</Attendee_Type>
</Attendee>
</Attendees>
</ApplicationData>
</Change>
activesyncd generates:
1:2170
BEGIN:VCALENDAR
PRODID:-//Meego//ActiveSyncD 1.0//EN
VERSION:2.0
METHOD:PUBLISH
BEGIN:VEVENT
ATTENDEE;CN=aaaa(a)example.com;PARTSTAT=DECLINED;ROLE=REQ-PARTICIPANT:
aaaa(a)example.com
END:VEVENT
END:VCALENDAR
And syncevolution ends up replacing the whole event with:
BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Synthesis AG//NONSGML SyncML Engine V3.4.0.47//EN
BEGIN:VEVENT
LAST-MODIFIED:20150710T132346Z
DTSTAMP:20150710T132346Z
UID:syuid320541.212303337826681
SUMMARY:unnamed
ATTENDEE;CN="aaaa(a)example.com";PARTSTAT=DECLINED;ROLE=REQ-
PARTICIPANT:mailto:aaaa@example.com
END:VEVENT
END:VCALENDAR
The question is, what do I do about this? Is there any way for
syncevolution to process updates and merge them with the existing
object? If not, does activesyncd have to go and refetch the entire
updated object from Exchange?
Graham
11:53 a.m.
On Fri, 2015-07-10 at 15:12 +0100, Graham Cobb wrote:
I have just noticed that events are being corrupted when syncing
from
Exchange.
The problem is that when an event changes, EAS can send a "change"
notification, with only the fields that have changed, not the whole new
event. activesyncd then creates a partial (and invalid!) VEVENT with
only the changed fields.
I agree, that cannot work in the current architecture. But I am less
sure why it occurs or how it was meant to be handled.
The question is, what do I do about this? Is there any way for
syncevolution to process updates and merge them with the existing
object?
Nothing that would be easy to use. The Synthesis engine has support for
merging fields into an existing item, but that is configured based on
capabilities of the peer and not per item.
If we were to configure the ActiveSync side of the sync such that the
engine is told that it supports no property at all, then the engine
would always preserve local properties unless explicitly overwritten.
However, that then breaks removing properties which *were* removed on
the ActiveSync side.
What we would need is a "this is an incremental update" flag per item.
Even with such a flag, any discrepancy in granularity between the
ActiveSync protocol, iCalendar and the internal field list would still
be a source of potential problems. Example: ActiveSync elements foo and
bar map to iCalendar FOOBAR, which then maps to foo and bar again in the
field list (or even something different again).
If the server changes foo and does not send bar, what would the
activesyncd server put into FOOBAR?
If not, does activesyncd have to go and refetch the entire
updated object from Exchange?
I think that would be the more realistic solution.
I also wonder whether the server can be told to avoid incremental
updates in the first place.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
11:06 p.m.
On 13/07/15 11:53, Patrick Ohly wrote:
> If not, does activesyncd have to go and refetch the entire
> updated object from Exchange?
I think that would be the more realistic solution.
Thanks for the info. I had assumed that merge/change handling was not
an option but thought I would check!
I also wonder whether the server can be told to avoid incremental
updates in the first place.
I will do more investigation. I am not sure whether the behaviour is
new (e.g. a new version of Exchange): I have recently changed the way I
am using syncevolution across my different devices and only actually
noticed the problem because the corrupted events fail to sync with
Owncloud causing the sync operation to fail with an error (note: the EAS
sync where the corruption occurs does not fail or produce any
user-visible errors at all!).
I will look into whether I can find out under what circumstances the
Change is being sent and whether I can avoid it. It only occurs with
very a small number of events (currently 2 out of over 2000 in my
calendar) but I am not sure if it is to do with the event or the type of
change, etc.
11:10 p.m.
On 16/07/15 23:06, Graham Cobb wrote:
On 13/07/15 11:53, Patrick Ohly wrote:
> I also wonder whether the server can be told to avoid incremental
> updates in the first place.
I will look into whether I can find out under what circumstances the
Change is being sent and whether I can avoid it. It only occurs with
very a small number of events (currently 2 out of over 2000 in my
calendar) but I am not sure if it is to do with the event or the type of
change, etc.
It seems that there is no way to ask for the full event to be provided
by the server. It always sends the <Change> message (unless you are
doing a slow sync and reading everything). In most cases, that is not a
problem: the documentation says that the object will be provided pretty
much in full (fields not specified in the <Change> should be deleted),
EXCEPT for a short list of special cases (for example, for emails,
sending a change to the "Read" flag does not mean everything else should
be deleted). Although the <Attendee> is NOT listed in the documentation
as one of those special cases, it seems it really is: if the only change
to the event is the attendee list, the whole event is NOT resent -- the
client is supposed to just note the update to the attendee and not
change anything else!
To make matter worse, the EAS protocol has no way (at least in the older
versions) to retrieve a particular event! The ONLY way to get a
particular event using the EAS protocol is to use the slow-sync approach
to download ALL events!
So, my current thought is to add (yet another) hack: if a calendar event
<Change> is received that does not include a <StartTime> then the whole
update is ignored. This means that (in this case) the attendee change
is ignored, but that is better than corrupting the whole event and
losing all the information about it.
The alternatives seem to be: never do syncs -- always download
everything so we always get the full content of every event (very
expensive, just to resolve an unusual corner case); or require a later
protocol version (which seems to allow fetching specific items) and add
more logic to re-fetch anything which appears to be an incremental
update. Note: the latter is only based on my reading of the spec -- I
have not tested that it really works.
10:53 a.m.
On Sun, 2015-07-19 at 23:10 +0100, Graham Cobb wrote:
On 16/07/15 23:06, Graham Cobb wrote:
> On 13/07/15 11:53, Patrick Ohly wrote:
>> I also wonder whether the server can be told to avoid incremental
>> updates in the first place.
>
> I will look into whether I can find out under what circumstances the
> Change is being sent and whether I can avoid it. It only occurs with
> very a small number of events (currently 2 out of over 2000 in my
> calendar) but I am not sure if it is to do with the event or the type of
> change, etc.
It seems that there is no way to ask for the full event to be provided
by the server. It always sends the <Change> message (unless you are
doing a slow sync and reading everything). In most cases, that is not a
problem: the documentation says that the object will be provided pretty
much in full (fields not specified in the <Change> should be deleted),
EXCEPT for a short list of special cases (for example, for emails,
sending a change to the "Read" flag does not mean everything else should
be deleted). Although the <Attendee> is NOT listed in the documentation
as one of those special cases, it seems it really is: if the only change
to the event is the attendee list, the whole event is NOT resent -- the
client is supposed to just note the update to the attendee and not
change anything else!
Thanks for digging into this. You are aware that you are turning into
the activesyncd maintainer? ;-}
So, my current thought is to add (yet another) hack: if a calendar
event
<Change> is received that does not include a <StartTime> then the whole
update is ignored. This means that (in this case) the attendee change
is ignored, but that is better than corrupting the whole event and
losing all the information about it.
I agree, this looks like the least evil option.
Sorry for the late response; I was on vacation.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
2485
days inactive
2509
days old
syncevolution@synevolution.org
4 comments
2 participants
participants (2)
-
Graham Cobb -
Patrick Ohly