Re: [bugs] problem with caldav syncing

Hi Lukas,

Thanks for your answer. Please see below.

----- Mail original -----
> De: "Lukas Fleischer" <lfleischer@xxxxxxxxxxxx>
> À: "bugs mailing list" <bugs@xxxxxxxxxxxx>
> Envoyé: Jeudi 8 Décembre 2016 09:58:28
> Objet: Re: [bugs] problem with caldav syncing
> 
> Hi Guillaume,
> 
> On Wed, 07 Dec 2016 at 11:56:08, Guillaume Laurès wrote:
> > [...]
> > b. calcurse-caldav --init=keep-remote
> > Connecting to mail.lauresfamily.fr...
> > Removing all local calcurse objects...
> > Importing new object 1850-1850.
> > Importing new object 1849-1849.
> > Importing new object 1847-1847.
> > Importing new object 1848-1848.
> > Importing new object 1851-1851.
> > Importing new object 1852-1852.
> > Pushing new object e920e173d4b6cdfb1fbb37ba455f1e9c138e7fac to the server.
> > Pushing new object 52598c0fb30ad49fc203262c0becc18a3888a6bb to the server.
> > Pushing new object 236f48097a77ad409ce651897c098f4131a4f186 to the server.
> > Pushing new object 4c952e994b137568cd4b7733fd3e6be3e8739a76 to the server.
> > Pushing new object 84e8936eb6e7135893cb08c102449e58189867d6 to the server.
> > Removing remote object 1850-1850
> > (/dav/meme@xxxxxxxxxxxxxxx/Calendar/b68447d9-ac52-4868-b312-3889f753631f.1479361338377.ics).
> > Removing remote object 1849-1849
> > (/dav/meme@xxxxxxxxxxxxxxx/Calendar/73e07335-4be8-492f-9b71-68bd028c8029.1479361188672.ics).
> > Removing remote object 1851-1851
> > (/dav/meme@xxxxxxxxxxxxxxx/Calendar/d4471577-55dc-4c1b-8025-ed7fdcf54143.ics).
> > Removing remote object 1848-1848
> > (/dav/meme@xxxxxxxxxxxxxxx/Calendar/21e14478-0834-46ef-acfa-450290a9e1d0.1479361188298.ics).
> > Removing remote object 1852-1852
> > (/dav/meme@xxxxxxxxxxxxxxx/Calendar/bfdc3c4b-3660-45d5-bc57-d7a08b6998a7.ics).
> > Saving synchronization database to /home/meme/.calcurse/caldav/sync.db...
> > 6 items imported, 0 items removed locally.
> > 5 items exported, 5 items removed from the server.
> > 
> > Here I wonder why it is pushing objects to the server... the db is emtpy.
> > So I only have a few appointments on the caldav server. Shouldn't calcurse
> > only import stuff in this case ?
> > Apparently the UIds were modified, and the timezone as well. But in
> > appeareance, all is fine on both sides : appointments seem unmodified in
> > zimbra,
> 
> Looks like a bug or some kind of misconfiguration to me. The relevant

Just in case here are the steps I follow to reproduce :
mv ~/.calcurse /tmp
calcurse
cp -a /tmp/.calcurse/caldav/ ~/.calcurse/
rm ~/.calcurse/caldav/sync.db
sed -i 's/DryRun = No/DryRun = Yes/' ~/.calcurse/caldav/config
calcurse-caldav --init=keep-remote
sed -i 's/DryRun = Yes/DryRun = No/' ~/.calcurse/caldav/config
calcurse-caldav --init=keep-remote
calcurse
calcurse-caldav
calcurse
calcurse-caldav
calcurse
...

> parts of the calcurse-caldav synchronization protocol work as follows:
> 
> * Objects are fetched from the server. New objects are imported into
>   calcurse. They obtain a local ID (which is basically a hash of the
>   item). Then, all pairs of local IDs and corresponding remote
>   identifiers are stored in the so-called sync database.

Got it.

> 
> * A list of all local IDs (of all items) is requested from calcurse.
>   Items with IDs that do not appear in the sync database are pushed to
>   the server. If there are local IDs in the sync database which do no
>   longer appear in the list reported by calcurse, they are removed from
>   the server.

Sounds clear too.

> 
> In the scenario above, one of the following things might have happened:
> 
> * The local ID was not computed, passed or stored correctly when
>   importing the items.
> 
> * The local ID changed between the initial import and the push/remove
>   steps.
> 
> * The list of local IDs requested from calcurse in the push/remove steps
>   was broken or parsed incorrectly in some way.
> 
> I am going to add some details to the debug output of calcurse-caldav in
> a couple of minutes. It would be great if you could fetch current master
> and try again with --debug to see what is going on.

I pulled a backup of the agenda to drop any side-effects of the previous tests.
I'll test a bit later today and I'll give you the debug output.

> 
> > 
> > c. calcurse-caldavcalcurse-caldav
> > Connecting to mail.lauresfamily.fr...
> > Loading synchronization database from
> > /home/meme/.calcurse/caldav/sync.db...
> > Traceback (most recent call last):
> >   File "/home/meme/bin/calcurse-caldav", line 522, in <module>
> >     local_new, local_del = pull_objects(conn, syncdb, etagdict)
> >   File "/home/meme/bin/calcurse-caldav", line 306, in pull_objects
> >     root = etree.fromstring(body)
> >   File "/usr/lib64/python3.4/xml/etree/ElementTree.py", line 1336, in XML
> >     return parser.close()
> > xml.etree.ElementTree.ParseError: no element found: line 3, column 0
> > 
> > If I run the sync again (without --init), without changing anything on both
> > sides, it crashes
> 
> My guess is that your server returns an empty HTTP response body instead
> of an empty XML skeleton if there are no items that match our query. We
> can easily avoid this by not asking for zero items, though. I will
> commit a patch for this soon; thanks for reporting!

Okay keep me tuned I'll test it as soon as I can.

Links