[SyncEvolution] Google Calendar sync
by Patrick Ohly
Hello!
As those who have tried the "google" config template know, only contact
synchronization over SyncML works, because that is all that Google
supports. Those who want to synchronize their Google calendar have to
use third-party bridges to that data (ScheduleWorld, Goosync).
http://bugzilla.meego.com/show_bug.cgi?id=319
A "native" solution would be to access the Google calendar via CalDAV. I
gave that a try, using the Evolution Data Server CalDAV/Google backend
as implementation of the protocol. In theory, this should have worked
out-of-the-box with SyncEvolution, although setting it up would be
fairly complex:
* set up syncevo-http-server to access Google calendar
* set up a client config to access the normal local calendar and
syncevo-http-server
* run command line with "--daemon=no", because that is currently
the only way how we can have SyncML client and server on the
same host
In practice I ran into several problems which makes this approach
unusable right now. For those who are interested in details, below is a
description of the problems.
My conclusion right now is:
1. We need a better solution for backend A<->backend B syncs (I
sometimes call that "local sync").
2. We should better write a CalDAV backend instead of relying on
EDS. Not only will that avoid the issues below (which could be
fixed), such a backend would also be useful for users not
interested in running EDS and perform better. I'm assuming here
that CalDAV has functionality that would helps us (like
efficient listing of all items) which we can't use when going
through EDS because it doesn't have corresponding APIs.
Any CalDAV experts or volunteers around? The second task could be
started already with the current code base. If no-one volunteers, then
it'll end up on the backlog of feature requests.
-------------------------------------------
Here's how I set up testing:
* create two test calendars in Google
* create the same calendars in Evolution, using Google_ical20_1
and Google_ical20_2 as names (naming expected by client-test,
setting the evolutionsource property to the right caldav:// URL
should work without creating the calendars in Evolution)
* run CLIENT_TEST_EVOLUTION_PASSWORD=<password>
CLIENT_TEST_EVOLUTION_PREFIX=Google_ ./client-test
Client::Source::ical20
First problem: our test data uses a UID with special characters, because
that happens to break "dumb" XML encoders/decoders and we want to find
such issues. It also breaks the EDS backend. It can create such an
event, but then cannot access it because the UID becomes of a URL used
by WebDAV, and these special characters are not escaped there (seen in
evolution-data-server console output).
As a workaround, I simplified the test data to not use such a
complicated UID when CLIENT_TEST_SIMPLE_UID=1 is set.
Still no luck. client-test can create an item, but then retrieving it
fails. The root cause for that is the backend's implementation of
e_cal_create_object(). It should return the ID assigned to the new
object, but for the WebDAV EDS backend it returns an empty string
instead of the correct ID. Iterating over all objects later shows that
the ID is something like http://www.google.com/calendar/feeds/<unique ID
of calendar>%
40group.calendar.google.com/private/full/jjs14jjurojsoi7koo3op1vq3g
Even if e_cal_create_object() worked, we wouldn't trust it in
SyncEvolution, because it was broken in the past. From the code:
// creating new objects works for normal events and detached occurrences alike
if(e_cal_create_object(m_calendar, subcomp, (char **)&uid, &gerror)) {
// Evolution workaround: don't rely on uid being set if we already had
// one. In Evolution 2.12.1 it was set to garbage. The recurrence ID
// shouldn't have changed either.
ItemID newid(!id.m_uid.empty() ? id.m_uid : uid, id.m_rid);
newluid = newid.getLUID();
modTime = getItemModTime(newid);
m_allLUIDs.insert(newluid);
}
In other words, when a VEVENT has a UID, we use that as local ID. That's
the best we can do, because the "garbage" really is a random pointer as
far as I remember, so we cannot use it at all.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 2 months
[SyncEvolution] How to select the evolution target calendar
by Matthias Urlichs
Hello,
I have one small problem: I cannot select which local Evolution calendar
SyncEvolution actually talks to.
Apparently it uses the first entry in
the /apps/evolution/calendar/sources gconf entry that it can find. Since
the order of these entries isn't changeable and I need to separate
"work" from "home" calendar entries (with separate mobiles to sync to
…), I'd like some way to change that.
Either this is not possible yet, or I'm blind and didn't find it in the
documentation …?
Otherwise, thanks for the excellent tool; it works with our Oracle
calendar like a charm (using the values in the Oracle branch).
-- Matthias
12 years, 3 months
[SyncEvolution] single source for documentation (MBC #690)
by Patrick Ohly
Hello!
Everyone, I'm updating the documentation. Feedback please - I'm not
doing this for myself ;-)
Mike, I have a question about Drupal - see last paragraph.
Quoting http://bugs.meego.com/show_bug.cgi?id=690:
Keeping separate chunks of documentation in sync is an ongoing effort. Perhaps
we can simplify that task by keeping a single set of source files and generate
all other documentation from it automatically.
The documentation that would be nice to have is:
* README: plain text, the file that is currently maintained in git
* man pages for syncevolution, synccompare, sync-ui: doesn't exist at the moment
* the "Usage" page on syncevolution.org for the command line
As a first step (BMO#4633) the docs which have already diverged need to be
merged again. Then we need a single source format and corresponding make rules
which will generate the rest.
Today I look into this issue a bit more seriously. Originally, I
suggested Perl POD as that shared format, primarily because it is old
and typically installed on build systems.
But I never really liked the format. reStructuredText seems so much
nicer, so I did some more experiments with it and rst2man (in
python-docutils >= 0.6 (?)).
I was pleasantly surprised how easy it was to copy text from the README
into a README.rst. Very little changes were necessary to the markup. For
the most part, README.rst can still be read like a normal text, one of
the big strengths of rST. Question: should I strip the suffix when
preparing a release (install as README), replace with .txt (install as
README.txt) or leave it as it is?
The advantage of .rst is that Emacs and other tools (GNOME) recognize
the format. The disadvantage is that it might not always be recognized,
thus forcing to select a plain text reader.
The only part where markup is non-intuitive are some extra :: colons to
get the following text marked as a literal block and quoting a hyphen
where it was mistaken for something else:
\--migrate
In older SyncEvolution releases a different layout of configuration files
was used. ...
I find good enough to simply replace README with README.rst, without
doing any modifications to it.
The other change that I made is to reorganize the content so that it is
more like a reference man page. First, this minimizes the overlap with
instructions found elsewhere on syncevolution.org. Second, it was an
incentive to document some aspects (environment variables!) which were
not documented before.
Attached is a copy. What do you think about these changes? Any spelling
mistakes, unclear statements, etc.?
The conversion to man format works fine. Because rst2man is not always
installed, the man page would be optional. I'm attaching the man page
(syncevolution.1) and a rendered output (without escape codes, use "man
-l syncevolution.1" in a terminal if you want bold and underlined text).
Conversion to HTML also works and the result can be copied almost
verbatim into Drupal with "Full HTML" as format:
http://syncevolution.org/wiki/main-page
The goal is to replace the "Usage" page
(http://syncevolution.org/documentation/syncevolution-usage) with this
rendered man page.
"Copied almost verbatim" because that format differs from HTML in one
aspect: line breaks inside a paragraph are turned into a <br/>. I find
that inconvenient, because it also forces me to remove line breaks from
my announcement emails when I copy/paste them into Drupal. To me, it
also seems to deviate from normal Markdown syntax which normally uses
extra whitespace at the end of a line to insert <br/>
(http://daringfireball.net/projects/markdown/syntax#p).
Mike, is this a Drupal-specific extension? Can it be turned off?
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 3 months
[SyncEvolution] Missing timezone information
by Frederik Elwert
Hi,
I am currently experiencing difficulties using syncevolution 1.0beta2
and Evolution. When getting items from Scheduleworld, timezone
information is obviously lost. I am syncing two computers, one using SE
0.9.2, one using SE 1.0beta2. If I didn’t overlook something, it works
ok from 1.0beta2 → Scheduleworld → 0.9.2. But when getting items back
0.9.2 → Scheduleworld → 1.0beta2, calendar events are added using a Z
timezone.
In both cases, SE is from the syncevolution.org debian repository,
running Ubuntu 9.10/10.04 (same with both versions).
If I should provide more information, please tell me what you need and
how I should provide it.
Cheers,
Frederik
12 years, 4 months
[SyncEvolution] syncevo-http-server on a certain interface
by Tino Keitel
Hi,
I set up a syncevolution server (git from yesterday) on a Debian Lenny
server and it works great.
The syncevo-http-server can't use HTTP. To prevent my data from being
sent unencrypted, I set up an apache with mod_proxy to "convert" the
syncevolution HTTP server to HTTPS [1], which works fine.
However, the synceto-http-server listens on all interfaces, which still
allows unencrypted access. As a quick fix, I added a "localhost" to
the reactor.listenTCP() call. I think that for a real fix, the
hostname could be extraced from the URL argument and used for the
reactor.listenTCP(), if the user provides an option for "listen to the
interface with the host only".
Regards,
Tino
[1] http://www.onlinehowto.net/Tutorials/Apache-Web-Server/Strip-SSL-with-Apa...
12 years, 4 months
[SyncEvolution] A synchronization server named Memotoo
by Ahmed Guellil
In the compatibility section on syncevolution's website, you wrote :
> Of the servers listed here, SyncEvolution is regularly tested with
> ScheduleWorld, Funambol, Google and Synthesis server, so, the
> information about known problems with them is also more detailed. Not
> listing problems with other servers does not mean they do not exist...
> If you have tested with a server not listed here or would like to add
> further information, then your updates to this page, sent to email
> <mailto:patrick.ohly@gmx.de>, are highly appreciated.
so I would like to talk about the Memotoo <https://www.memotoo.com/> server.
I didn't see any problems concerning the todos, memos, or addressbook
(just a few modifications during sync for the contacts phone numbers
(none->home/work)).
A few days ago, I tried to synchronize my calendar that way, under
Debian Lenny :
syncml native client syncevolution (0.9.2+1.0beta2a-2)
N79 <-----------------> Memotoo <-------------------> Evolution (2.22.3.1-1)
two-way two-way
It seems that there is a problem about detached recurrences. I read your
article on this subject, and to be fair, I'm not an expert, but I run
some tests on my workstation and here is what I found :
- a detached recurrence on the N79 is well synchronized with Memotoo.
There is no duplicate.
- a detached recurrence on the Memotoo is well synchronized with the
N79. There is no duplicate.
- a detached recurrence on Memotoo is duplicated on Evolution but not on
Memotoo, even after resync.
- a detached recurrence on Evolution is duplicated on Memotoo but not on
Evolution, even after resync.
I think it's a conflict problem but I don't know how to resolve it. From
what I've heard, it seems that the events modifications are not time
stamped, so syncevolution has no means to discover which one is to save
and which one should be deleted, so it creates duplicates not to loose
data. But I don't know. Actually, I almost never modify the same event
with the two clients before synchronizing. Maybe the events aren't the
same anyway, maybe it's because of syncevolution, or maybe evolution
(too old version ?).
It's a bit disturbing because I have a lot of professional meetings and
after each one I wrote systematically a description about it. I also
modify or cancel some regularly, which means I end with a lot of duplicates.
So I would be pleased if you help me (and maybe others) to solve this
problem. But I know you're a busy guy.
Thanks.
Ahmed Guellil
Nantes, France
12 years, 4 months
[SyncEvolution] Syncevolution and direct phone syncing on Debian Lenny?
by Anssi Saari
Hi,
I got Syncevolution 1.0 beta 3 built and working fine on my Fedora 12
laptop and Nokia N85, but my Debian 5.0 desktop has problems. BTW, I
couldn't use the provided rpm package for Fedora, since it puts libs
in /usr/lib and 64-bit Fedora expects them in /usr/lib64. But building
was no problem, likewise installation using stow.
As for Debian, the package from downloads.syncevolution.org/apt
installs fine, but trying to do
syncevolution --configure --template ${template} --sync-property syncURL=obex-bt://${MacAddress} MyPhone
just complains with
[ERROR] calendar: type 'calendar' not supported by any of the backends
(syncfile.so, syncxmlrpc.so, syncecal.so, syncsqlite.so,
syncmaemocal.so, syncebook.so, syncaddressbook.so)
$template was set to Nokia N85 and $MacAddress to the correct BT address.
So, I built it myself, but now it can't find the phone config files
for some reason. syncevolution -l '??' finds nothing:
Available configuration templates:
template name = template description matching score in percent (100% = exact match)
none
and syncevolution --configure --template ${template} --sync-property syncURL=obex-bt://${MacAddress} MyPhone
gives
ERROR: no configuration template for 'Nokia N85' available.
This seems really weird. I used the same procedure in both Fedora and Debian, with
./configure --prefix=/usr/local/stow/syncevolution-1.0beta3 --enable-gui
and then ran stow for syncevolution-1.0beta3 which creates symlinks.
The N85 template and others are in the same place in both,
/usr/local/share/syncevolution/templates/clients/phone/nokia/S60/N85 (via symlink) and
/usr/local/stow/syncevolution-1.0beta3/share/syncevolution/templates/clients/phone/nokia/S60/N85
So, any help? Or is some part of Debian Stable just too old?
12 years, 5 months
[SyncEvolution] Nokia N82 success, question and feature request
by Maximilian Schleiss
Hi,
congratulations on your work on SyncEvolution, it is easy to install, use
and works great.
I use Ubuntu 10.04, Evolution is 2.28.3 and SyncEvolution is from your own
package repository. The only method to sync tested here is via bluetooth,
using a notebook and the blueman project bluetooth manager.
This is not exactly a SE issue but I noticed that Evolution has no field for
"mobile (business)" and "mobile (home)" it only has "mobile", is there a way
to add these fields to Evolution?
Another question I could ask on the Evolution ML is the way to sync the
changes made in Evolution. As it is now, one has to fire up sync-ui and
click on sync, any chance to put a sync button somewhere in the interface?
There is a sync menu in Evolution but it is only meant for Palm Pilot...
Thank you in advance for the kind answer.
Best regards,
Maximilian Schleiss
12 years, 5 months
[SyncEvolution] HTTP server state - "less tested than Bluetooth"? how so?
by Jeffrey Perry
Hello everyone,
I'm testing the Syncevolution 1.0 beta 2 (I haven't had a chance to grab
beta 3 yet) using the scratchbox emulator environment, preparing to
trust data to it.
My intent is to run the sync evolution client on an N900 and
syncevolution HTTP server on my desktop.
The HTTP Server Howto on the wiki says that the HTTP server is not the
intended use case and that the Bluetooth sync direct is the more well
tested scenario.
In what way is it more well tested?
I see there are about 450 CPPUnit tests...very nice.
Are all the same tests that are run for Bluetooth sync testing
also run for the HTTP server sync scenario?
As I understand it the key differences in the two scenarios are the HTTP
server written in python (it's code looks concise and
clean) and the evolution dbus server (also used in the Bluetooth
scenario?).
Am I correct?
Anything else I should be considering in my evaluation?
--Jeff
12 years, 5 months
Re: [SyncEvolution] Syncevolution - data preservation of unique fields
by Patrick Ohly
On Mon, 2010-04-26 at 00:04 +0100, Jeff wrote:
> >
> > I can only speak for the contact backend, as the calendar backend was
> > done by Ove. With contacts in EDS, I know that "contact lists" are not
> > synchronized (http://bugs.meego.com/show_bug.cgi?id=1008). The main
> > reason is that there is no standard format commonly understood between
> > SyncML peers. When using SyncEvolution on both sides, that isn't a
> > problem, and something that works in that combination could be added.
> >
>
> I don't mind if contact lists don't sync as long as they are not lost
> on the evolution side during sync.
>
> My interest is in a syncevolution to syncevolution case...
> You say "something that works in that combination could be added"
> I'm confused by that ... what needs to be added?
For example, contact lists. They are represented as a single contact
with additional properties. These additional properties are not modeled
in the XML datatype configuration and profile. As a result some data is
lost when synchronizing such a special contact back and forth.
> > On the N900, there are like to be some Nokia extensions which are not
> > synchronzed. I don't know what the situation is with those in Maemo 5,
> > but earlier there were such extensions
> > (http://bugs.meego.com/show_bug.cgi?id=666).
> >
>
> Fields that don't exist at one side should be ignored as Synthesis does.
> Is this what happens? or does data get lost as a consequence?
They get lost when overwriting the item. That's because when receiving
an update, the following happens:
1. read existing item from data store
2. convert to field list
3. convert update to field list
4. update field list
5. create item from field list and write back
"item" for EDS is a vCard, which completely replaces the one in the data
store. So any property lost during the vCard<->field list conversion is
also eventually lost in the data store.
This can be avoided by only using one-way synchronization, which never
writes into the local data store. But that is also less useful.
> > To put you at ease a bit, the automatic database dumps made by
> > SyncEvolution do not pass the data through the Synthesis engine. They
> > take a vCard directly out of EDS and put them into files, so the
> > complete contacts can be restored again should something go wrong.
> >
>
> Please tell me a bit more about these backups and at what stage of the
> sync process they are made and where they are located...
They end up in the session directories, see "syncevolution
--print-session". They contain the name of the source ("addressbook")
and whether they were made before or after the sync ("before"/"after").
Hard links are used to minimize overall disk usage.
The backups are created after a session has started successfully (client
and server agree on credentials and on which sources are active), and
before any data is modified ("before"). The "after" dump is done when
the session ends.
"synccompare" compares the two dumps to figure out whether data has
changed.
> If I understand you correctly, you mean in the syncevolution with http
> server configuration connecting with a syncevolution client (such as the
> N900), data traveling to the client is taken straight from the EDS and
> placed directly into files on the client.
>
> So Synthesis is not part of the process?
No, Synthesis is involved. There is no special case for
Evolution<->Evolution sync.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 5 months
