Re: [bugs] problem with caldav syncing

Hi Guillaume,

On Wed, 07 Dec 2016 at 11:56:08, Guillaume Laurès wrote:
> [...]
> b. calcurse-caldav --init=keep-remote
> Connecting to mail.lauresfamily.fr...
> Removing all local calcurse objects...
> Importing new object 1850-1850.
> Importing new object 1849-1849.
> Importing new object 1847-1847.
> Importing new object 1848-1848.
> Importing new object 1851-1851.
> Importing new object 1852-1852.
> Pushing new object e920e173d4b6cdfb1fbb37ba455f1e9c138e7fac to the server.
> Pushing new object 52598c0fb30ad49fc203262c0becc18a3888a6bb to the server.
> Pushing new object 236f48097a77ad409ce651897c098f4131a4f186 to the server.
> Pushing new object 4c952e994b137568cd4b7733fd3e6be3e8739a76 to the server.
> Pushing new object 84e8936eb6e7135893cb08c102449e58189867d6 to the server.
> Removing remote object 1850-1850 (/dav/meme@xxxxxxxxxxxxxxx/Calendar/b68447d9-ac52-4868-b312-3889f753631f.1479361338377.ics).
> Removing remote object 1849-1849 (/dav/meme@xxxxxxxxxxxxxxx/Calendar/73e07335-4be8-492f-9b71-68bd028c8029.1479361188672.ics).
> Removing remote object 1851-1851 (/dav/meme@xxxxxxxxxxxxxxx/Calendar/d4471577-55dc-4c1b-8025-ed7fdcf54143.ics).
> Removing remote object 1848-1848 (/dav/meme@xxxxxxxxxxxxxxx/Calendar/21e14478-0834-46ef-acfa-450290a9e1d0.1479361188298.ics).
> Removing remote object 1852-1852 (/dav/meme@xxxxxxxxxxxxxxx/Calendar/bfdc3c4b-3660-45d5-bc57-d7a08b6998a7.ics).
> Saving synchronization database to /home/meme/.calcurse/caldav/sync.db...
> 6 items imported, 0 items removed locally.
> 5 items exported, 5 items removed from the server.
> 
> Here I wonder why it is pushing objects to the server... the db is emtpy. So I only have a few appointments on the caldav server. Shouldn't calcurse only import stuff in this case ?
> Apparently the UIds were modified, and the timezone as well. But in appeareance, all is fine on both sides : appointments seem unmodified in zimbra, 

Looks like a bug or some kind of misconfiguration to me. The relevant
parts of the calcurse-caldav synchronization protocol work as follows:

* Objects are fetched from the server. New objects are imported into
  calcurse. They obtain a local ID (which is basically a hash of the
  item). Then, all pairs of local IDs and corresponding remote
  identifiers are stored in the so-called sync database.

* A list of all local IDs (of all items) is requested from calcurse.
  Items with IDs that do not appear in the sync database are pushed to
  the server. If there are local IDs in the sync database which do no
  longer appear in the list reported by calcurse, they are removed from
  the server.

In the scenario above, one of the following things might have happened:

* The local ID was not computed, passed or stored correctly when
  importing the items.

* The local ID changed between the initial import and the push/remove
  steps.

* The list of local IDs requested from calcurse in the push/remove steps
  was broken or parsed incorrectly in some way.

I am going to add some details to the debug output of calcurse-caldav in
a couple of minutes. It would be great if you could fetch current master
and try again with --debug to see what is going on.

> 
> c. calcurse-caldavcalcurse-caldav
> Connecting to mail.lauresfamily.fr...
> Loading synchronization database from /home/meme/.calcurse/caldav/sync.db...
> Traceback (most recent call last):
>   File "/home/meme/bin/calcurse-caldav", line 522, in <module>
>     local_new, local_del = pull_objects(conn, syncdb, etagdict)
>   File "/home/meme/bin/calcurse-caldav", line 306, in pull_objects
>     root = etree.fromstring(body)
>   File "/usr/lib64/python3.4/xml/etree/ElementTree.py", line 1336, in XML
>     return parser.close()
> xml.etree.ElementTree.ParseError: no element found: line 3, column 0
> 
> If I run the sync again (without --init), without changing anything on both sides, it crashes

My guess is that your server returns an empty HTTP response body instead
of an empty XML skeleton if there are no items that match our query. We
can easily avoid this by not asking for zero items, though. I will
commit a patch for this soon; thanks for reporting!

> 
> d. calcurse-caldav
> Connecting to mail.lauresfamily.fr...
> Loading synchronization database from /home/meme/.calcurse/caldav/sync.db...
> Importing new object 1874-1874.
> Pushing new object 3300f6818e4553742c3c4f08393122452be350e5 to the server.
> Removing remote object 1874-1874 (/dav/meme@xxxxxxxxxxxxxxx/Calendar/b8755306-32b6-4076-b515-fa41c8977c78.ics).
> Saving synchronization database to /home/meme/.calcurse/caldav/sync.db...
> 1 items imported, 0 items removed locally.
> 1 items exported, 1 items removed from the server.
> 
> If a add an appointment on the server, it works however. but again we see export/remove on the server
> 
> Any idea ?

Same as above, please try with enhanced debug output which I am going to
commit soon.

Regards,
Lukas

Links