http://bugs.meego.com/show_bug.cgi?id=1369 Summary: Ovi.com: iCalendar 2.0 encoding by server is wrong (QUOTED-PRINTABLE + wrong line folding) Classification: MeeGo Projects Product: SyncEvolution Version: unspecified Platform: Netbook OS/Version: IA Status: WAITING FOR UPSTREAM Severity: normal Priority: High Component: SyncEvolution AssignedTo: syncevolution-bugs(a)meego.bugs ReportedBy: jingke.zhang(a)intel.com QAContact: jingke.zhang(a)intel.com CC: syncevolution-bugs(a)meego.bugs, syncevolution-default-bugs(a)meego.bugs Estimated Hours: 0.0 This is from http://bugzilla.moblin.org/show_bug.cgi?id=10485 Description From miurahr 2010-04-16 21:22:31 PST (-) [reply] Created an attachment (id=4685) --> (http://bugs.meego.com/attachment.cgi?id=4685) [details] raw log data of sync with ovi.com I use git repository version of syncevolution in Apr.9 (commit bc2908a39f9f692babc002ca0097ff8e1e631750) When syncing with OVI.com, which is not supported but sometimes works, description become broken. It can be seen when trying to sync Asian string which consume 3byte each in UTF-8. This means only single Asian char need 9-byte in quoted-printable:utf-8 encoding. It cause easily folding/unfolding in data streams. Things make worst, OVI.com send folding data even it split between '=' and hex string! eg. SUMMARY;ENCODING=QUOTED-PRINTABLE;CHARSET=UTF-8:=E5=AF=A9=E6=9F=BB=E5=A7=94= E5=93=A1=E4=BC=9A this data causes strange strings. libsynthesis try to convert this data but it mistake unfold last '=' and next 'E5' it make bad data which include bogus space character '0x20'. in above example, libsynthesis should decode such that 0xe5 0xaf 0xa9 0xe6 0x9f 0xbb 0xe5 0xa7 0x94 0xe5 0x93 0xa1 0xe4 0xbc 0x9a but it generate 0xe5 0xaf 0xa9 0xa6 0x9f 0xbb 0xe5 0xa7 0x94 ' ' 'E' '5' 0x93 0xa1 0xe4 0xbc 0x9a Why it generate bad sequence? it try to detect 'soft break' and '=' on end of line looks soft break mark, but it does not. it remove '=' and line break then cause this bug. ------- Comment #1 From miurahr 2010-04-16 21:55:13 PST (-) [reply] ------- Created an attachment (id=4686) --> (http://bugs.meego.com/attachment.cgi?id=4686) [details] yet another log file which happen same situation You can see data corruption in a part # [2010-04-13 09:18:59.061] deleting "審査委 E5??会, Ipa" which I explained as example in bug report. ------- Comment #2 From miurahr 2010-04-17 15:05:21 PST (-) [reply] ------- Created an attachment (id=4687) --> (http://bugs.meego.com/attachment.cgi?id=4687) [details] minimal reproduction data and its log reproduction procedure 1. setup ovi.com account and syncevolution setting according to bug #3182 and following blog description. http://mobileyog.blogspot.com/2009/11/how-to-sync-contactscalendar-from.html 2. make schedule on ovi.com that summary is '一二三四五六七八九十' (its means japanese number from 1 to 10, each char need 3byte in UTF-8, so ovi.com send its as multiline data) 3. sync calendar syncevolution --sync refresh-from-server ovi calendar 4. you may get error but you can see log file that show us it mistake qp conversion. ------- Comment #3 From miurahr 2010-04-17 15:07:05 PST (-) [reply] ------- Created an attachment (id=4688) --> (http://bugs.meego.com/attachment.cgi?id=4688) [details] an incoming data ------- Comment #4 From miurahr 2010-04-17 19:32:17 PST (-) [reply] ------- Created an attachment (id=4689) --> (http://bugs.meego.com/attachment.cgi?id=4689) [details] incoming calendar data ------- Comment #5 From pohly 2010-04-18 01:00:32 PST (-) [reply] ------- Congwu, do you have time to look into this? ------- Comment #6 From pohly 2010-04-19 04:36:03 PST (-) [reply] ------- (In reply to comment #5) I'll have a look myself. ------- Comment #7 From pohly 2010-04-19 06:14:39 PST (-) [reply] ------- Here's the data sent by Ovi.com: BEGIN:VCALENDAR VERSION:2.0 BEGIN:VEVENT DTSTART:20100417T073000Z DTEND:20100417T080000Z LAST-MODIFIED:20100417T124516Z PRIORITY:0 TRANSP:OPAQUE DESCRIPTION: SUMMARY;ENCODING=QUOTED-PRINTABLE;CHARSET=UTF-8:=E4=B8=80=E4=BA=8C=E4=B8=89= E5=9B=9B=E4=BA=94=E5=85=AD=E4=B8=83=E5=85=AB=\n=E4=B9=9D=E5=8D=81 LOCATION: UID:991399640@tss END:VEVENT END:VCALENDAR The data sent by Ovi.com is invalid because it uses ENCODING=QUOTED-PRINTABLE, which is not specified for iCalendar 2.0. vCalendar 1.0 used to have it, but it has been superseded by BASE64. In theory, QUOTED-PRINTABLE might have been added later on as "Some other IANA registered iCalendar encoding type", but I don't find it. I checked via: http://www.iana.org/assignments/icalendar/icalendar.xhtml CHARSET is also not part of iCalendar 2.0. Even if Ovi.com was using vCalendar 1.0, the line break would still be wrong. The trailing = in the first line clearly is a soft line break according to RFC 2045, so the Synthesis parser correctly interprets the next line as starting with " E5". I'm also unsure about the \n in the text. I suspect that a line break has to be encoded differently with QUOTED-PRINTABLE. I was about to suggest that you work around these issues by forcing the server to use vCalendar 1.0, but I suspect that you'll run into similar issues. Nevertheless, it is worth a try. You can do that by appending the string ":text/x-vcalendar" to the type in your config.in files for calendar, todo, and "calendar+todo" - see .config/syncevolution/peers/ovi/sources/*/config.ini. Does that help? You should see BEGIN:VCALENDAR VERSION:1.0 in the data coming from Ovi.com as a result. ------- Comment #8 From miurahr 2010-04-22 18:08:30 PST (-) [reply] ------- Created an attachment (id=4696) --> (http://bugs.meego.com/attachment.cgi?id=4696) [details] log by trying vCalendar 1.0 sync Great! It works good with amended type calendar:text/x-vcalnder by 'syncevolution --sync refresh-from-server ovi calendader' I attached logs. ------- Comment #9 From miurahr 2010-04-22 18:11:26 PST (-) [reply] ------- Here is data sent by ovi.com when forcing vCalendar 1.0 BEGIN :VCALENDAR VERSION:1.0 TZ:+09 BEGIN:VEVENT DTSTART:20100417T073000Z DTEND:20100417T080000Z LAST-MODIFIED:20100417T124516Z PRIORITY:0 SUMMARY;ENCODING=QUOTED-PRINTABLE;CHARSET=UTF-8:=E4=B8=80=E4=BA=8C=E4=B8=89=E5=9 B=9B=E4=BA=94=E5=85=AD=E4=B8=83=E5=85=AB= =E4=B9=9D=E5=8D=81 END:VEVENT END:VCALENDAR ------- Comment #10 From miurahr 2010-04-22 18:13:54 PST (-) [reply] ------- Here's a command log when forcing vCalendar 1.0: $ syncevolution --sync refresh-from-server ovi calendar [INFO] memo: inactive [INFO] addressbook: inactive [INFO] todo: inactive [INFO] calendar: starting first time sync from server Local data changes to be applied remotely during synchronization: *** calendar *** Comparison was impossible. [INFO] calendar: started [INFO] adding "一二三四五六七八九十" [INFO] calendar: received 1/1 [INFO] calendar: added 1, updated 0, removed 0 [INFO] calendar: first time sync done successfully Synchronization successful. Changes applied during synchronization: +---------------|-----------------------|-----------------------|-CON-+ | | LOCAL | REMOTE | FLI | | Source | NEW | MOD | DEL | ERR | NEW | MOD | DEL | ERR | CTS | +---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+ | calendar | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | refresh-from-server, 0 KB sent by client, 0 KB received | | item(s) in database backup: 0 before sync, 1 after it | +---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+ | start Fri Apr 23 10:04:08 2010, duration 0:36min | | synchronization completed successfully | +---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+ Data modified locally during synchronization: *** calendar *** before sync | after sync removed during sync < ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- ------- Comment #11 From pohly 2010-04-23 00:05:05 PST (-) [reply] ------- (In reply to comment #9) Good news. The problem was indeed iCalendar 2.0 specific, because the Ovi encoder (incorrectly) applied iCalendar 2.0 folding rules to the QUOTED-PRINTABLE property. This is what inserted the spurious space after the = "soft line break". In vCalendar mode, the encoder knows that line breaks cannot be inserted like that and thus sends us one long line, which is okay. I'll keep the issue open for a bit longer because I want to discuss it with the Ovi team. In SyncEvolution 1.0 beta 3 our Ovi template already uses "type = calendar:text/x-vcalendar", so we don't have to change anything there. -- Configure bugmail: http://bugs.meego.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug. You are watching someone on the CC list of the bug.