http://bugzilla.moblin.org/show_bug.cgi?id=7708 pohly <patrick.ohly(a)intel.com&gt; changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |7893 -- Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching the assignee of the bug. You are watching someone on the CC list of the bug.

http://bugzilla.moblin.org/show_bug.cgi?id=7708 --- Comment #2 from pohly <patrick.ohly(a)intel.com&gt; 2009-12-04 07:48:59 PST --- (In reply to comment #1) When looking at this issue I noticed that even those syncs had their own logdir. As an example, I disabled all sources ("syncevolution --configure --source-property sync=none <server>"), then ran a sync. I got logdir with status.ini:status = 500 and a log file. The error is not recorded right now. I have added that, in preparation for use in some future intelligent merging of logdirs. It's useful in its own right, because that way the command line can print it together with the sync report at the end of the output. It's not visible in this example, but in other cases the initial ERROR might be far up at the beginning of the sync run and no longer visible: ... [ERROR] no sources active, check configuration Synchronization failed, see /home/pohly/.cache/syncevolution/scheduleworld_1-2009-12-04-16-35/syncevolution-log.html for details. Changes applied during synchronization: +---------------|-------ON CLIENT-------|-------ON SERVER-------|-CON-+ | | rejected / total | rejected / total | FLI | | Source | NEW | MOD | DEL | NEW | MOD | DEL | CTS | +---------------+-------+-------+-------+-------+-------+-------+-----+ | start Fri Dec 4 16:35:24 2009, duration 0:00min | | synchronization failed (status code 500) | +---------------+-------+-------+-------+-------+-------+-------+-----+ First ERROR encountered: no sources active, check configuration -- Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching someone on the CC list of the bug.

http://bugzilla.moblin.org/show_bug.cgi?id=7708 --- Comment #4 from pohly <patrick.ohly(a)intel.com&gt; 2010-02-08 01:49:03 PST --- (In reply to comment #0) The reusing of old files was implemented. In contrast to the original proposal, identical files are identified before writing them, based on hash values. Here's an example of the result: $ du /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-0*/ical20.* 32 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-09-51-1/ical20.current 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-09-51-1/ical20.current.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-13/ical20.after.ini 32 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-13/ical20.before 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-13/ical20.before.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-13/ical20.current 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-13/ical20.current.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-15-1/ical20.after.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-15-1/ical20.before 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-15-1/ical20.before.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-15-2/ical20.current 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-15-2/ical20.current.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-27/ical20.after 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-27/ical20.after.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-27/ical20.before 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-27/ical20.before.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-28/ical20.after.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-28/ical20.before 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-28/ical20.before.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-28/ical20.current 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-28/ical20.current.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-30-1/ical20.after 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-30-1/ical20.after.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-30-1/ical20.before 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-30-1/ical20.before.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-30/ical20.after 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-30/ical20.after.ini 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-30/ical20.before 4 /home/pohly/.cache/syncevolution/scheduleworld__1(a)client_+test_+1-2010-02-08-10-30/ical20.before.ini Note that du recognizes shared files and only reports the space required for the file data once. Dumps are not entirely free. That number "4" must be the number of blocks required for the directory information itself. Because hard links are used, the file content is preserved when deleting older dirs. If creating the hard links fails (which could be the case when previous sessions somehow end up on a different filesystem or the filesystem does not support hard links), then each file will be written anew. TODO: detecting identical files is done with a rather simple Hash() algorithm. We want something stronger or must fall back to reading old files to detect collisions. Does anyone know a SHA implementation under a compatible license? Fixed. The code is careful to pick the latest session involving a source, regardless whether that session really contains a database dump. This is important because we use this comparison to show what will be sent to the peer, so we really need the state of the data after the last sync with that peer. This is true, but I went one step further and improved synccompare so that it excludes files with the same inode from the comparison. This is faster and more reliable than including the data in the diff and detect similar content there. Yongsheng, can you review the code in the mb7708 branch? You worked with that code last and I also fixed a bug in that code. Because of the hash issue it is not ready for master yet. -- Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching someone on the CC list of the bug.

http://bugzilla.moblin.org/show_bug.cgi?id=7708 yongsheng zhu <yongsheng.zhu(a)intel.com&gt; changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |yongsheng.zhu(a)intel.com --- Comment #6 from yongsheng zhu <yongsheng.zhu(a)intel.com&gt; 2010-02-08 18:39:10 PST --- (In reply to comment #5) I've had a look, please go forward. -- Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching someone on the CC list of the bug.

http://bugzilla.moblin.org/show_bug.cgi?id=7708 --- Comment #8 from pohly <patrick.ohly(a)intel.com&gt; 2010-02-09 10:00:16 PST --- The configure check for "glib recent enough" seems to fail on Debian Etch: /work/runtests/head/syncevolution/src/syncevo/util.cpp: In function 'std::string SyncEvo::SHA_256(const std::string&)': /work/runtests/head/syncevolution/src/syncevo/util.cpp:292: error: 'G_CHECKSUM_SHA256' was not declared in this scope /work/runtests/head/syncevolution/src/syncevo/util.cpp:292: error: 'g_compute_checksum_for_data' was not declared in this scope -- Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching someone on the CC list of the bug.

http://bugzilla.moblin.org/show_bug.cgi?id=7708 --- Comment #10 from pohly <patrick.ohly(a)intel.com&gt; 2010-02-16 13:04:08 PST --- Code all in "pohly" branch. (In reply to comment #7) commit 33f6e2064c7fc7c376336e86fd9460df3cdbf14c Author: Patrick Ohly <patrick.ohly(a)intel.com&gt; Date: Tue Feb 16 17:43:41 2010 +0100 SyncSourceRevisions: cache result of listAllItems() (MB #7708) When automatic backups are enabled (the default), SyncSourceRevisions and derived classes like TrackingSyncSource called listAllItems() twice, once during backup and once while checking for changes. This patch introduces caching of the result returned by the first call. During a normal session, that will be during backup, with change detection reusing the information. If backups are off, the call will happen during change detection, as before. The advantages of this change are: - more efficient, in particular for sources where listAllItems() is slow - avoids a race condition (backup stores one set of items, changes are made, change detection works with another set) That race condition is both fairly theoretical (short time window) and had no impact on the correctness of the sync, only on the output (data comparison would not show all changes synced later). Now the race condition still exists if changes are made while a sync runs, which is the bigger problem that still needs to be solved (for EDS, see MB #3479). Restoring data also calls listAllItems(). It does not use the cached information, just in case that it is to be called more than once per instance, and because there is no benefit. commit 5f18644fbfc944db1759dbb726f2ea8b7892e0cf Author: Patrick Ohly <patrick.ohly(a)intel.com&gt; Date: Tue Feb 16 20:11:16 2010 +0100 SyncSource API: added m_isEmpty operation (MB #7708) The current solution of checking for empty sources by looking at the "before" database dump is a hack which fails if creating that backup was disabled. It also prevents delaying the dump until after the start of the session, because the information must be available before starting the session, so that the XML config can be created with slow sync check enabled if needed. This patch adds a new operation which backends must implement if they want to provide the "is empty" information for the slow sync detection. This is optional. If the information is not available, slow sync detection is always on, even when not strictly needed. commit fc059f7017f32b9184b5781a875a3ab22ca0c0c5 Author: Patrick Ohly <patrick.ohly(a)intel.com&gt; Date: Tue Feb 16 20:15:50 2010 +0100 SQLite backend: implement m_isEmpty operation (MB #7708) This patch demonstrates how the operation can be implemented by sources implementing the operations themselves. The implementation of SQLiteContactSource::isEmpty() might not be perfect, but it should get the job done. commit 9e45cb0c56e9975637214f0f03add823171b314c Author: Patrick Ohly <patrick.ohly(a)intel.com&gt; Date: Tue Feb 16 20:17:36 2010 +0100 TrackingSyncSource: added isEmpty() pure virtual method (MB #7708) This implements the new m_isEmpty operation with a pure virtual TrackingSyncSource::isEmpty() callback. It follows the philosophy that TrackingSyncSource should a) hide the underlying operation mechanism and b) prevent derived classes from compiling unless they do everything that is expected of them. All of the in-tree backends were adapted so that they provide at least a dummy implementation of the call, to keep them in a usable state. The file backend implementation is okay. The Evolution backends get the job done correctly, but not efficiently. Maemo and XMLRPC backends always return false, which is acceptable but should be changed (disables the "allow slow sync for empty databases" heuristic). With this change in place, it was possible to delay dumping of databases also when running as client: commit 4713aa435afa8e51b18780f94595e81eead6008c Author: Patrick Ohly <patrick.ohly(a)intel.com&gt; Date: Tue Feb 16 17:15:46 2010 +0100 database comparison: also delay it in clients (MB #7708) Delaying the database comparison was done initially for servers to limit it to the sources really used by the client. In a client the same delay makes sense for syncs which fail because the peer cannot be reached. It is possible to delay the dump because starting the session no longer depends on it thanks to the m_isEmpty() operation. Not done yet. -- Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching someone on the CC list of the bug.

http://bugzilla.moblin.org/show_bug.cgi?id=7708 Bug 7708 depends on bug 7707, which changed state. Bug 7707 Summary: revised configuration layout: multiple peers per config http://bugzilla.moblin.org/show_bug.cgi?id=7707 What |Old Value |New Value ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution| |FIXED -- Configure bugmail: http://bugzilla.moblin.org/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching someone on the CC list of the bug.