Re: [SyncEvolution] syncevolution and a radicale caldev server
by Patrick Ohly
On Fri, 2012-02-17 at 15:14 +0100, Gregor Horvath wrote:
> Am Fri, 17 Feb 2012 11:12:52 +0100
> schrieb Patrick Ohly <patrick.ohly(a)intel.com>:
>
> > On Tue, 2012-02-14 at 15:57 +0100, Patrick Ohly wrote:
> > > But as Radicale is an example where multiple databases are possible
> > > (in
> > > contrast to Google), it's worth spelling out explicitly how such a
> > > config can be created:
> >
> > [...]
> >
> > William, did this work?
> >
> > Perhaps you (or someone else) has the time to turn the instructions
> > into a proper HOWTO? I created a place holder Wiki page for it here:
> > http://syncevolution.org/wiki/synchronizing-radicale
> >
> >
William sent me some debug logs, but then ran out of time. So I've
installed Radicale myself and found an issue that'll break
SyncEvolution: items reported by the server include double slashes in
the path.
<multistatus xmlns="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
<response>
<href>/public_user/calendar/calendar_1//1234567890(a)dummy.ics</href>
^^
SyncEvolution will then use /1234567890(a)dummy.ics as path in future
requests, which Radicale rejects with either 404 or 401 errors.
I'll work around that in SyncEvolution. I'm not sure whether there is a
solution with 1.2.2.
Have you considered Calypso (http://keithp.com/blogs/calypso/)? I don't
know yet whether it works with SyncEvolution; if there is interest, then
I will try it.
> I am trying to do the same thing (with one calender to start with)
> N900 <-> radicale but it did not work, because it seems the Maemo
> calendar is not found:
>
> http://pastebin.com/huMXDsEx
>
> What I am doing wrong?
You have additional spaces after your \ character. The slash must be at
the end of each line.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
5 years, 2 months
[SyncEvolution] documentation update, enhanced command line
by Patrick Ohly
Hello!
The "Getting the concepts clear" mail thread and a private email
exchange with Todd identified several shortcomings in the documentation
and/or implementation. This is my attempt to address those. A diff and
the full modified README.rst are attached.
Some of these changes will require changes in the implementation.
In short, what I am trying to achieve is:
* Better explain what local sync is and how it involves two sync
configs. "originating config" gets introduces instead of just
"sync config".
* Better explain the relationship between contexts, sync configs,
and source configs ("a sync config can use the source configs in
the same context").
* An entire section on config properties in the terminology
section.
* Less focus on conflict resolution, as suggested by Graham.
* Remove the hard-coded "target-config" name. It still needs to be
there as fallback for existing configs or users continuing to
use the current instructions.
* Fix examples that became invalid when fixing the password
storage/lookup mechanism for GNOME keyring in 1.4. I noticed
that the username=email-address part should be handled without
additional properties.
* Fix WebDAV with DNS auto-discovery. Few servers support it, so
this hasn't been noticed before.
* Fix the implicit command line magic where it does some
consistency checks when creating new configs. Not exactly sure
yet how.
I ran out of stream after updating the terminology section, the "command
line conventions", "Synchronization beyond SyncML" and "CalDAV and
CardDAV". It's possible that the other sections also contain slightly
incorrect usage of the terminology or are simply out-dated.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
5 years, 8 months
[SyncEvolution] SyncML <-> CardDAV: UID handling
by Patrick Ohly
Hello!
Who is using SyncEvolution as gateway between a SyncML client and a
CardDAV server? Which client?
I am currently updating the CardDAV backend and would like to remove
some UID handling hacks that did not work in all cases anyway.
The problem is that these hacks might be needed for SyncML clients which
claim to support UID while in fact they rewrite the UID and then send
some other UID back during an update. This then becomes a problem on the
CardDAV side because some CardDAV servers (for example, Apple Calendar
server) do not allow changing the UID during an update.
I don't know whether there are such broken SyncML clients, though, and
therefore I intend to go ahead with the change.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
5 years, 9 months
[SyncEvolution] merging of winning and loosing items
by Patrick Ohly
Hello!
I'm currently writing tests for handling of conflicts. This wasn't done
earlier because SyncEvolution's test suite primarily targeted syncing
with SyncML servers, and behavior of those was out of control anyway.
With local syncing and/or SyncEvolution as HTTP server that's different.
My main concern at the moment are update/update conflicts where both
sides updated the same common item before syncing. libsynthesis
determines the winning item based on age (REV in vCard, LAST-MODIFIED in
iCalendar). That determines which side's single-value properties are
preserved when merging them isn't possible. I already noticed that I
need tests for both cases, because a particular SyncEvolution bug
(https://bugs.freedesktop.org/show_bug.cgi?id=77065) only shows up in
one of the two cases.
How does this apply to arrays? Consider the following scenario: the
client adds a new TEL, the server a new EMAIL (in this order). What I
observe is that the server's item wins, preserving the server's EMAIL,
while the client's TEL array gets overwritten with the server's less
complete TEL array. The new TEL from the client thus gets lost.
That is with the following profile:
<!-- telephone numbers -->
<field name="TEL" array="yes" type="telephone" compare="conflict"/>
<field name="TEL_FLAGS" array="yes" type="integer" compare="conflict"/> <!-- offset 0 -->
<field name="TEL_LABEL" array="yes" type="string" compare="conflict"/> <!-- offset 1 -->
<field name="TEL_ID" array="yes" type="integer" compare="conflict"/> <!-- offset 2 -->
<field name="TEL_SLOT" array="yes" type="integer" compare="never"/> <!-- offset 3 -->
<!-- emails -->
<field name="EMAIL" array="yes" type="multiline" compare="conflict"/>
<field name="EMAIL_FLAGS" array="yes" type="integer" compare="conflict"/> <!-- offset 0 -->
<field name="EMAIL_LABEL" array="yes" type="string" compare="conflict"/> <!-- offset 1 -->
<field name="EMAIL_ID" array="yes" type="integer" compare="conflict"/> <!-- offset 2 -->
<field name="EMAIL_SLOT" array="yes" type="integer" compare="never"/> <!-- offset 3 -->
I suspect that in this case the engine really has no choice other than
treating the entire TEL array as one value. Even if it knew how to merge
the two TEL arrays (which isn't that simple when one also takes
reordering and intentional removal of entries into account), would it
know that the consecutive arrays (TEL and TEL_* resp. EMAIL and EMAIL_*)
are related?
I don't think so. That's something that is specified in the profile, but
not in the field list.
So a merge script will be needed if special handling of TEL and EMAIL is
desired, right?
The problem that I was alluding to earlier is that merging
winning: TEL = [ '1', '2' ] and
loosing: TEL = [ '1', '2', '3' ]
has no unambiguous solution. Was '3' added on the loosing side or was it
removed on the winning one?
If in doubt, probably the best approach is to pick the one which avoids
data loss, i.e. preserve '3'.
Now, consider reordering:
winning: TEL = [ '1', '2' ] and
loosing: TEL = [ '2', '1' ].
Were the entries merely reordered or were the values changed on the
loosing side? In the former case, nothing needs to be done and no
information gets lost. In the later case, the changes made on the client
get lost. I don't see a way around that (*).
Copying extra values from the loosing side can work with reordering by
looking at each value and not just at the end of the array. It's not
perfect, though, when values were edited.
Any comments so far?
(*) A three-way merge where previous copies of client and server items
are available to compute a diff would work, but that increases storage
requirements for rare situations and would require significant changes
to the engine.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
5 years, 10 months
[SyncEvolution] Gnome Online Accounts
by Emiliano Heyns
Hey list,
I'm trying to figure out how to get Google OAuth2 credentials in the
hands of SyncEvolution on a headless Ubuntu server. I considered
installing gnome-online-accounts but that pulled in a metric crapload of
dependencies I'd rather not install. Copying the GOA databases from my
desktop to the server didn't work either. Is there any other way I can
hand a token to SyncEvolution? I know I can generate a token on the
server -- google-cl does it too.
Any ideas?
Emile
5 years, 10 months
Re: [SyncEvolution] Google CalDAV sync
by Patrick Ohly
[Replying to the list with Todd's permission]
On Mon, 2014-04-14 at 17:55 -0700, Todd Wilson wrote:
> Patrick,
>
>
> This is my first question about the Google side of my attempted sync
> set-up, so I'm starting a new thread. I'm using
>
>
> http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backend...
>
>
>
> as a resource. Here are the steps I've taken so far:
>
>
> 1. As a precaution, I started up GOA, deleted my Google Account, and
> re-added it, authenticating successfully inside the GOA application.
Which version of GOA, which distro, and what permissions where you asked
to grant?
It is important that "Google Calendar" and (for Google Contacts) "Google
Contacts via CardDAV" appear there. Yes, Google distinguishes between
Google Contacts via their own API and CardDAV.
> 2. I went online to my Google Account and verified, under Recent
> Activity, that I've signed in (Linux). Under Account Permissions, it
> says (among many other things, of course):
> * Ubuntu: Has full access to your Google Account (Auth date:
> aaaa/2014)
> * Ubuntu: Has access to Google Drive, Google Talk, Picasa Web
> Albums, basic account info (Auth date: yyyy/2013)
> * GNOME: Has access to Gmail, Google Calendar, Google Contacts,
> Google Docs, Google Drive, basic account info (Auth date:
> xxxx/2011)
> I'm not sure why these old connections are there, or why the most
> recent connection wasn't dated today because of what I did in Step 1.
Not sure either.
> 3. I checked with seahorse and found an entry "GOA google credentials
> for identity account_1322264161" (Gnome Online Accounts password). I
> also found "Network passwords" for "xxxxx(a)zimbra.yyyyyy/dav/twilson"
> and "xxxxx(a)zimbra.yyyyyy/dav/twilson/Calendar". Are those the entries
> created by syncevolution in my Zimbra sync? Do I need both?
Probably yes. If you look at the details of these network passwords,
you'll see that SyncEvolution includes the full path as "server" key, so
the two entries really are different. Whether they are still in use is a
different question.
Here's what it has stored for me for Google CalDAV:
server: www.google.com/calendar/dav/%u/user/?SyncEvolution=Google
user: patrick.ohly(a)googlemail.com
The "server" and "user" values are used to find the stored "password"
value. I've not found good instructions for choosing keys. Make them too
specific and you end up with more entries than necessary and the user
has to fix them multiple times when changing the password on the server.
Make them too unspecific and different passwords will conflict with each
other. For example, an HTTP server might require different passwords for
different paths, which is why SyncEvolution includes the path in the
key.
> 4. Using the account information I got from Step 3, I entered the
> command
> $ SYNCEVOLUTION_DEBUG=1 syncevolution --daemon=no --print-databases
> backend=caldav username=account_xxxxxxx
^^^^^^^^^^^^^^^^^^^^^^^^^^^
You need to tell SyncEvolution that you want to use GOA. This has to be
username=goa:account_xxxxxx.
> 5. In seahorse again, I can look at the password for
> account_1322264161 and it looks like
> {'access_token', <'...'>, 'access_token_secret':<'...'>}
> with no resemblance to my password, so I can't check to see if it's
> stored correctly, but I assume it is since there was no error where I
> created the account and Google says I'm logged in.
This is indeed normal. One of the advantages of OAuth is that the actual
password for the login doesn't need to be stored. The password allows
the application to do anything it wants with your Google account. The
OAuth tokens restrict access to those services that were requested and
granted.
> 6. I ran the command from Step 4 again with loglevel=4. I won't flood
> this email with the output (unless you want to see it), but I note
> that the 403 was accompanied by a response body that included
> <error>
> <domain>usageLimits</domain>
> <code>dailyLimitExceededUnreg</code>
> <internalReason>Daily Limit for Unauthenticated Use
> Exceeded. Continued use requires signup.</internalReason>
>
> <extendedHelp>https://code.google.com/apis/console</extendedHelp>
> </error>
> and the 401 was accompanied by a response body that included
> <error>
> <domain>GData</domain>
> <code>authError</code>
> <location type="header">Authorization</location>
> <internalReason>HTTP Basic Authentication is not supported
> for this API</internalReason>
> </error>
This confirms that SyncEvolution was attempting normal HTTP Basic Auth
with a blank password, which would have been wrong, even if it had been
supported for this particular path (which it isn't).
As reported recently on the list, the old CalDAV URL still works with
Basic Auth, but that will be turned off eventually.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
5 years, 11 months
[SyncEvolution] mapping between individual X- extensions and grouped extensions
by Patrick Ohly
Hello!
I am trying to figure out how I can translate between Evolution/SyncML
vCard format and CardDAV (in particular, Google Contacts).
Google Contacts seems to follow the format as used by Apple, which is
not surprising considering that Apple iOS and OS X are probably the main
clients using CardDAV.
The key difference is that CardDAV uses IMPP (rfc 4770) for all kinds of
chat protocols, whereas Evolution and several SyncML implementations use
one kind of X- extension per protocol (X-AIM, X-JABBER, ...).
Evolution:
X-JABBER;TYPE=HOME;X-EVOLUTION-UI-SLOT=2:JABBER DOE
Google CardDAV:
item5.IMPP;X-SERVICE-TYPE=GoogleTalk:xmpp:google%20talk
item5.X-ABLabel:Other
Note that X-SERVICE-TYPE=GoogleTalk is the provider of the service.
"xmpp:" is part of the URL value and defines the chat protocol used by
the service; the rest of the value uses URL encoding of special
characters (a space in this case). The label is just that, a label
chosen by the user.
Evolution doesn't have support for custom labels, so there is nothing
exactly corresponding to X-ABLabel yet. TYPE comes close.
Same for some dates (X-EVOLUTION-ANNIVERSARY vs. X-ABDate):
Evolution:
X-EVOLUTION-ANNIVERSARY:2006-01-09
Google CardDAV:
item3.X-ABDATE:1971-01-01
item3.X-ABLabel:Anniversary
item4.X-ABDATE:2000-02-01
item4.X-ABLabel:custom-label
The exception is BDAY, which is used by both sides.
Finally, the same difference exists for related persons:
Evolution:
X-EVOLUTION-SPOUSE:Joan Doe
Google CardDAV:
item16.X-ABRELATEDNAMES:spouse
item16.X-ABLabel:Spouse
At the moment, SyncEvolution uses separate fields for AIM (AIM_HANDLE,
an array) and spouse (a single value). I'm leaning towards changing that
into a more generic field list where there is one array for IMPP, one
array for DATE, and one array for related NAMES. I think that can be
mapped fairly directly to the vCard format used by Google CardDAV. For
example:
<field name="IMPP" array="yes" type="string" compare="conflict"/> <!-- includes protocol: xmpp:google%20talk -->
<field name="IMPP_SERVICE" array="yes" type="string" compare="conflict"/> <!-- GoogleTalk -->
<field name="IMPP_LABEL" array="yes" type="string" compare="conflict"/> <!-- Other -->
<field name="GROUP-TAG-IMPP" array="yes" type="string" compare="conflict"/> <!-- group tag connecting IMPP_LABEL = X-ABLabel with IMPP and IMPP_SERVICE -->
The bigger problem will be on the Evolution side. I don't see how I can
teach libsynthesis that a IMPP entry whose protocol (encoded as part of
the value!) is xmpp maps to X-JABBER.
Should I keep the traditional JABBER_HANDLE array and move entries back
and forth between it and the IMPP array? This could be done with
incoming/outgoing resp. afterread/beforewrite scripts.
Then the traditional profile will only use the JABBER_HANDLE, as before,
whereas the new profile only uses IMPP.
For SyncEvolution<->SyncEvolution syncing the new profile should be used
because it will be more complete. Perhaps I can achieve that by offering
multiple datatypes and then letting the sync engines negotiate the most
suitable one.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
5 years, 11 months
[SyncEvolution] srcdir + Maemo
by Patrick Ohly
Hello Ove!
In the following commit you changed the rule for syncevo-webdav-lookup:
commit ff97574d7026a008efa79bdc8a46c7fc6688d218
Author: Ove Kåven <ovek(a)arcticnet.no>
Date: Sat Aug 11 12:11:30 2012 +0200
Fixed broken rule for creating syncevo-webdav-lookup.
src/backends/webdav/syncevo-webdav-lookup:
$(srcdir)/src/backends/webdav/syncevo-webdav-lookup.sh
$(AM_V_GEN)rm -f $@ ; \
- ln -s $< $@
+ cd src/backends/webdav && ln -s $(notdir $<) $(notdir $@)
I noticed recently that this broke out-of-tree compilation, causing
syncevolution.org binaries to not have the script anymore.
Do you remember why the rule was broken for you? It works for me. Could
it be that srcdir was empty?
Guido fixed something like that for builddir:
commit 85fca619139a27016503fe8f312b9fcc870dc563
Author: Guido Günther <agx(a)sigxcpu.org>
Date: Sat May 18 19:14:39 2013 +0200
build: use top_builddir instead of builddir
when building syncevo-local-sync. Maemo's old automake doesn't now
about builddir.
This only fixes the one occurence relevant to Maemo.
diff --git a/src/src.am b/src/src.am
index 8da50e2..5505b07 100644
--- a/src/src.am
+++ b/src/src.am
@@ -183,7 +183,7 @@ src_syncevo_local_sync_LDADD = $(gdbus_build_dir)/libgdbussyncevo.la $(CORE_LDAD
src_syncevo_local_sync_CPPFLAGS = -DHAVE_CONFIG_H -I$(gdbus_dir) $(src_cppflags)
src_syncevo_local_sync_CXXFLAGS = $(PCRECPP_CFLAGS) $(SYNCEVOLUTION_CXXFLAGS) $(CORE_CXXFLAGS) $(GLIB_CFLAGS) $(DBUS_CFLAGS) $(LIBSOUP_CFLAGS) $(SYNCEVO_WFLAGS)
src_syncevo_local_sync_LDFLAGS = $(PCRECPP_LIBS) $(CORE_LD_FLAGS) $(LIBSOUP_LIBS)
-src_syncevo_local_sync_DEPENDENCIES = $(builddir)/$(gdbus_build_dir)/libgdbussyncevo.la $(EXTRA_LTLIBRARIES) $(CORE_DEP) $(SYNTHESIS_DEP)
+src_syncevo_local_sync_DEPENDENCIES = $(top_builddir)/$(gdbus_build_dir)/libgdbussyncevo.la $(EXTRA_LTLIBRARIES) $(CORE_DEP) $(SYNTHESIS_DEP)
# Do the linking here, as with all SyncEvolution executables.
I'm now wondering why Guido only had to replace $(builddir) in one
place. $(builddir) and $(srcdir) still get used in several places, some
of them relevant for Maemo (?). A global search/replace seems in order
to fix the issue everywhere. With the non-recursive build system used by
SyncEvolution top_builddir == buildir and top_srcdir == srcdir anyway,
the non-top version is just a bit shorter.
Attached a patch for the syncevo-webdav-lookup problem. Does this look
right?
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
5 years, 11 months
