11:44 p.m.
Hi Patrick,
at least I could do one half of the pending work yesternight...
On Aug 29, 2011, at 17:00 , Patrick Ohly wrote:
[...]
I've written such a test and found some problems:
1. compile issue
2. TPluginApiDS::apiAddItem() treated DB_DataMerged as a failure
and didn't return the local ID to
TCustomImplDS::implProcessItem(), which caused a segfault later
on (local ID NULL)
Patches attached.
Thanks.
But now I still see an unexpected Replace command.
I looked up what's happening in the code and the replace is not unexpected to me:
Here's the processing
of the client's Add in a SyncEvolution server where that Add maps to an
item that was new for this client:
[2011-08-29 16:43:57.474] adding "meeting on site, big meeting room"
[2011-08-29 16:43:57.479] InsertItemAsKey res=207
[2011-08-29 16:43:57.480] Database adapter indicates that added item was merged with
pre-existing data (status 207)
Status 207 is meant to indicate that the backend has added the item, but in the process
merged the add with some additional data (from an external source, or another item in the
sync set). So...
...the Server looks if the resulting item was already scheduled for sending to the client
before (in its unmerged state):
[2011-08-29 16:43:57.480] TStdLogicDS::getConflictingItemByLocalID,
found RemoteID='1234567890!@#$%^&*()<>@dummy-rid',
LocalID='1234567890!@#$%^&*()<>@dummy-rid', syncop=add
apparently yes... (so it seems to be an add/add conflict)
[2011-08-29 16:43:57.480] Preventing
localID='1234567890!@#$%^&*()<>@dummy-rid' to be sent to client
[2011-08-29 16:43:57.480] Item with
localID='1234567890!@#$%^&*()<>@dummy-rid' will NOT be sent to client
(slowsync match / duplicate prevention)
...sending this item is to be prevented, as we're not interested in sending the
unmerged state. What we want to update the client with is the merged state, so next the
server re-fetches the item from the backend, to get the merged state:
[2011-08-29 16:43:57.480] Created command 'Status'
(outgoing)
[2011-08-29 16:43:57.484] ReadItemAsKey aID=(1234567890!@#$%^&*()<>@dummy-rid,)
res=0
[2011-08-29 16:43:57.484] 'ScriptExecute' - Start executing Script,
name=afterreadscript [--][++] [->end] [->enclosing]
[...]
[2011-08-29 16:43:57.489] Forced conflict with corresponding item from server DB
So now we have the merged item (it is called forceConflict() for historical reasons, where
fetching a particular item from the backend after loading the sync set was needed for
creating a server response for an item that would not needed one).
[2011-08-29 16:43:57.489] Deleted command 'Status' (outgoing
MsgID=0, CmdID=0)
[2011-08-29 16:43:57.489] ModifyMap called: aEntryType=normal,
aLocalID='1234567890!@#$%^&*()<>@dummy-rid,
aRemoteid='1234567890!@#$%^&*()<>@dummy-rid', aMapflags=0x0, aDelete=0
[2011-08-29 16:43:57.490] - found no matching entry for localID
'1234567890!@#$%^&*()<>@dummy-rid' - creating new one, added=true
[2011-08-29 16:43:57.490] - Operation add performed (regular), Remote
ID=1234567890!@#$%^&*()<>@dummy-rid Local
ID=1234567890!@#$%^&*()<>@dummy-rid, status=201
[2011-08-29 16:43:57.490] Deleted command 'Status' (outgoing MsgID=0, CmdID=0)
Now the (new) client ID is mapped to the merge result's server ID.
[...]
Later the server generates a Replace:
–[2011-08-29 16:43:57.494] 'Item_Generate' - generating SyncML item,
SyncOp=wants-replace, RemoteID=1234567890!@#$%^&*()<>@dummy-rid [--][++]
[->end] [->enclosing]
+–[2011-08-29 16:43:57.494] 'ScriptExecute' - Start executing Script,
name=outgoingscript [--][++] [->end] [->enclosing]
[...]
+–[2011-08-29 16:43:57.505] 'issue' - issuing command, Cmd=Replace [--][++]
[->end] [->enclosing]
[2011-08-29 16:43:57.505] Command 'Replace': is 2-th counted cmd, cmdsize(+tags
needed to end msg)=500, available=38558 (maxfree=38558, freeaftersend=38058,
notUsableBufferBytes()=0)
[2011-08-29 16:43:57.505] Item
remoteID='1234567890!@#$%^&*()<>@dummy-rid', localID='',
datasize=422
[2011-08-29 16:43:57.506] Replace: issued as (outgoing MsgID=2, CmdID=5), now queueing
for status
[2011-08-29 16:43:57.506] Outgoing Message size is now 946 bytes
–[2011-08-29 16:43:57.506] End of 'issue' [->top] [->enclosing]
+–[2011-08-29 16:43:57.506] 'DSStateChange' - Datastore changes state,
datastore=calendar, oldstate=server_sync_gen_started, newstate=sync_gen_done [--][++]
[->end] [->enclosing]
–[2011-08-29 16:43:57.506] End of 'DSStateChange' [->top] [->enclosing]
[2011-08-29 16:43:57.506] engGenerateSyncCommands ended, state='sync_gen_done',
sync generation done
Shouldn't this Item_Generate be suppressed in this case?
No, that's the server updating the client with the result of the merge.
Why would you expect this to be suppressed? On the contrary, this replace is explicitly
forced by the 207 status handling code.
Maybe we can sort out why you were expecting something else, before I implement the
"other half"
The other half:
>> So what's missing right now is a simple way to actually
do the merge
>> in the backend, without re-implementing half of libsynthesis. The
>> (hopefully not so) longterm solution would be libvxxx. As a short term
>> workaround, which is not exactly super elegant but not dangerous
>> either, we could do the following:
>>
>> * we define another special return code for AddItem, say 419.
>> The backend can return it when it detects that the added data SHOULD
>> be merged with what the backend already has, but can't do that
>> itself.
>>
>> * the engine would do the same things as (details see commit msg
>> in the patch below) for 207 but additionally it would:
>> * merge the incoming data with the data fetched from the backend
>> with the localID returned from AddItem()
>> * call UpdateItem() with the merge result in the backend
>>
>> Let me know what you think...
>
> That sounds like it would solve the problem.
Is this something that you intend to work on or shall I give it a try
myself?
I am confident I can do that tomorrow.
Best Regards,
Lukas
Lukas Zeller, plan44.ch
luz(a)plan44.ch - www.plan44.ch