1
0
mirror of https://github.com/chylex/Nextcloud-Desktop.git synced 2025-09-06 05:53:13 +02:00
Commit Graph

702 Commits

Author SHA1 Message Date
Christian Kamm
0cf19123a7 VFS: Unbreak behavior for rename+hydrate #7001
Users can rename a file *and* add/remove the vfs suffix at the same time
leading to very complex sync actions. This patch doesn't add support for
them, but adds tests and makes sure these cases do not cause unintened
behavior.

The rename will be propagated, but the users's hydrate/dehydrate request
will be ignored.
2020-12-15 10:58:44 +01:00
Olivier Goffart
46bf3ed31a Upload: asynchronious operations
Implements https://github.com/owncloud/core/pull/31851
2020-12-15 10:58:44 +01:00
Christian Kamm
4bd062f5be OwnSql: Distinguish no-data from error #6677
This could fix a problem where the client incorrectly decides to delete
local data.

Previously any sqlite3_step() return value that wasn't SQLITE_ROW would
be interpreted as "there's no more data here". Thus an sqlite error at a
bad time could cause the remote discovery to fail to read an unchanged
subtree from the database. These files would then be deleted locally.

With this change sqlite errors from sqlite3_step are detected and
logged. For the particular case of SyncJournalDb::getFilesBelowPath()
the error will now be propagated and the sync run will fail instead of
performing spurious deletes.

Note that many other database functions still don't distinguish
not-found from error cases. Most of them won't have as severe effects on
affected sync runs though.
2020-12-15 10:58:43 +01:00
Christian Kamm
575935ded0 Windows: Forbid chars 0-31 in filenames #6987 2020-12-15 10:58:42 +01:00
Christian Kamm
93afc2a04b Discovery win: Fix detection of case-only renames
Previously they were detected as DELETE+NEW because if "a" is renamed to
"A" then QFile::exists("a") will still return true on Windows.
2020-12-15 10:58:42 +01:00
Christian Kamm
238ac53666 Ensure local discovery on selective sync changes
As far as I'm aware local discovery can be skipped on folders that are
selective-sync blacklisted, so a local discovery is required when an
entry is removed from the blacklist.

Also rename
avoidReadFromDbOnNextSync() -> schedulePathForRemoteDiscovery()
since the old name might also imply it's not read from db in the local
discovery - which is not the case. Use Folder::
schedulePathForLocalDiscovery() for that.
2020-12-15 10:58:41 +01:00
Christian Kamm
4c04351360 Fix warnings about signedness
Sizes are always qint64, not unsigned.
TransferIds are always uint.
2020-12-15 10:58:41 +01:00
Christian Kamm
60cb5d3b34 Tests: Fix permission propagation check
It was using the wrong path to the conflict file.
2020-12-15 10:58:41 +01:00
Christian Kamm
5089f55629 Test: Add check for permission propagation
Also covering propagation to the downloaded file when a conflict-rename
is done at the same time, which used to not work.
2020-12-15 10:58:41 +01:00
Christian Kamm
aed1f28980 Tests: Add db checks to rename/move tests
For moves it's relevant that the old db entry is removed and the new one
is created at the right location.
2020-12-15 10:58:41 +01:00
Christian Kamm
910ccaf600 Vfs: Improve sync protocol entries for actions
Creating a new virtual file and replacing a file with a virtual one now
have their own text in the protocol, not just "Downloaded".

To do this, the SyncFileItem type is kept as
ItemTypeVirtualFileDehydration for these actions. Added new code to
ensure the type isn't written to the database.

While looking at this, I've also added documentation on SyncFileItem's
_file, _renameTarget, _originalFile and destination() because some of
the semantics weren't clear.
2020-12-15 10:58:41 +01:00
Christian Kamm
f24687ec93 Vfs: Make dehydration a SYNC and not a NEW action
That change will be useful for the notifications. Previously the
dehydrated files were reported as "newly downloaded", now they're
reported as "updated".
2020-12-15 10:58:40 +01:00
Christian Kamm
8a8e93827f Vfs suffix: Fix dehydration creating the wrong db entry 2020-12-15 10:58:40 +01:00
Christian Kamm
83a818678f PinStates cleanup
- SyncJournalDB functions now behind internalPinStates() to avoid
accidental usage, when nearly everyone should go through Vfs.
- Rename Vfs::getPinState() to Vfs::pinState()
2020-12-15 10:58:39 +01:00
Christian Kamm
0eb4065197 Folder: Add remoteFolderTrailingSlash()
There were cases where the "/" exception wasn't handled correctly
and there'd be extra slashes in generated paths.
2020-12-15 10:58:39 +01:00
Christian Kamm
01f7bc7b49 SyncFileStatusTracker: Distinguish Warning and Excluded
Any folder with a (potentially deeply) contained error will have
StatusWarning. StatusExcluded marks exclusions. The difference is useful
to know for VFS.
2020-12-15 10:58:38 +01:00
Christian Kamm
7f400e3226 Pin state updates
- unspecified and inherited are different
- move enum to header in common/
- access through Vfs instead of directly in Journal
2020-12-15 10:58:38 +01:00
Christian Kamm
4d58208676 File watcher: Reduce touch ignore duration
On Linux and Windows the file watcher can't distinguish between changes
that were caused by the process itself, like during a sync operation,
and external changes. To work around that the client keeps a list of
files it has touched and blocks notifications on these files for a bit.

The duration of this block was originally and arbitrarily set at 15
seconds. During manual tests I regularly thought there was a bug when
syncs didn't trigger, when the only problem was that my changes happened
too close to a previous sync operation.

This change reduces the duration to three seconds. I imagine that this
is still enough.

Also use std::chrono while at it.
2020-12-15 10:58:36 +01:00
Olivier Goffart
6f4bf585f0 Move: Fix move detection in directory move on the other side 2020-12-15 10:58:35 +01:00
Olivier Goffart
88d02a887f Move: add comments and tests 2020-12-15 10:58:35 +01:00
Olivier Goffart
1fb4c22adf Move: add more test and fix move within moves 2020-12-15 10:58:35 +01:00
Olivier Goffart
ade4c11de3 Rename: fix renamed folder moved into renamed folder issue
Issue #6694
2020-12-15 10:58:35 +01:00
Christian Kamm
836d298141 vfs suffix: Ignore server files or synced files with the suffix #6953 2020-12-15 10:58:34 +01:00
Christian Kamm
20ef0a0290 vfs: Allow retrieving of pin state paths and flags 2020-12-15 10:58:33 +01:00
Christian Kamm
4df101ed84 vfs: Allow (de-)hydrating the full sync folder 2020-12-15 10:58:33 +01:00
Christian Kamm
aa382eda29 Db: Add wiping of pin state for subtrees 2020-12-15 10:58:30 +01:00
Christian Kamm
d956f518a8 vfs: Remove newFilesAreVirtual - use root PinState instead
This unifies how to deal with pin states.

Also enable reading a folders direct pin state vs its effective pin
state.
2020-12-15 10:58:30 +01:00
Olivier Goffart
62ec4c9330 Discovery: Handle the blacklistFiles from the server capabilities
Issue #434

Ideally one could add the blacklist to the exlucde reggexp, but this
is simpler
2020-12-15 10:58:30 +01:00
Christian Kamm
486c25cb47 vfs: Use PinState in sync algorithm #6815
New files are virtual if the file's pin state is OnlineOnly.
2020-12-15 10:58:29 +01:00
Christian Kamm
68126ac208 vfs: Introduce PinState db storage #6815
The idea is to allow folders (and later maybe files?) to be
- pinned to be available locally
- pinned to be online only
- inherit their pin from the parent

Where this pinning only controls the default for new files.
Subfolders may have a different pin state, and contained files
may be hydrated or dehydrated based on user actions.

This value is stored in a new 'flags' table. The idea is to store
data there that doesn't necessarily exist for each metadata entry.
The selective sync state could be migrated to this table.
2020-12-15 10:58:29 +01:00
Olivier Goffart
dfedb09fd8 Move test for issue #1329 from t1.pl to new test system 2020-12-15 10:58:29 +01:00
Olivier Goffart
a29320b18d Discovery: Set right direction when restoring deleted discovery because it has modified files
(Catched by a faillure of t1.pl)
2020-12-15 10:58:29 +01:00
Olivier Goffart
6da96cd026 Port the exclude test to the QTest Framework
This is just a port to QtTest, I did not change the layout of the test.
I did search and replace to replace the assert with QCOMPARE/QVERIFY
I still call setup and setup_init like before (only explicitly, now)

Also ported the preformence tests to QBENCHMAK because windows don't have
gettimeofday.

Relates #6358
2020-12-15 10:58:29 +01:00
Olivier Goffart
fad1238b33 Test: Move the test for Utility::normalizeEtag to testutility.cpp
I just moved the text and did the minimum to port it to QtTest
Did not change hte layout of it.

Relates #6358
2020-12-15 10:58:28 +01:00
Olivier Goffart
6e048a2d3f Remove check_csync_util
It's testing nothing.
Only that one can convert a csync instruction to a string.
But this is only used in debug anyway

Relates to #6358
2020-12-15 10:58:28 +01:00
Christian Kamm
69144566ce Database: Change path for new dbs to .sync_* #5904
This is to avoid issues on OSX, where the ._ prefix has special meaning.

Originally (before 2.3.2) ._ was necessary to guarantee exclusion. But
since then the .sync_ prefix is excluded as well.

This does not affect existing database files.
2020-12-15 10:58:28 +01:00
Olivier Goffart
edd806960d Propagator: Don't abort sync on error 503
Only do it when it is actually a maintenance mode

Issues #5088, #5859, https://github.com/owncloud/enterprise/issues/2637
2020-12-15 10:58:27 +01:00
Christian Kamm
305d439c41 vfs: Separate vfs availability from new-files-virtual
This helps support 2.5 settings where there are virtual files in the
tree but new files aren't created virtual.

It's also a prelude for #6815

There's currently no way of
- upgrading vfs plugins (a silent suffix->winvfs upgrade is attempted
  once only, when moving to master)
- disabling vfs capabilities outright
2020-12-15 10:58:27 +01:00
Christian Kamm
b30f79edf6 vfs: Ensure SyncOptions::_vfs is never null
- Create a VfsOff derived class
- Make it a shared pointer shared with Folder::_vfs
2020-12-15 10:58:26 +01:00
Christian Kamm
fa2450cf11 vfs: Be more careful about Vfs instance ownership 2020-12-15 10:58:25 +01:00
Dominik Schmidt
1104883fba Cleanup CMake (output) 2020-12-15 10:58:23 +01:00
Christian Kamm
fa6f3cd847 vfs: Fix plugin decision in wizards, sanitize loading 2020-12-15 10:58:23 +01:00
Christian Kamm
0f2ef42ba2 Fixes after rebase to master
- use vfs suffix in ProcessDirectoryJob
- fix include vfs.h
- fix local vio passing vfs
- fix checksum computation
- vfs mode use
- mingw lambda compile issue
2020-12-15 10:58:21 +01:00
Christian Kamm
2b20985875 winvfs: initial work
Done by ckamm and dschmidt
2020-12-15 10:58:21 +01:00
Christian Kamm
22e08cf6ad Checksums: Work on QIODevice*s
Needed for cfapi where we want to feed data through a custom device
which retrieves data from the windows api.
2020-12-15 10:58:21 +01:00
Kevin Ottens
cce3d6f6fc Excludes: switch all the path handling to QString
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:58:20 +01:00
Olivier Goffart
3c3619f99a Exclude: do everything with QString wiuthout converting to char* 2020-12-15 10:58:19 +01:00
Olivier Goffart
69de2d5180 Sync: optimize by removing setFileRecordMetadata
Inh most case we already have a record from before, so avoid doing a useless
lookup in the database.
In owncloudpropagator.cpp, directories do not have a checksum so no need
to call a function that preserves it
2020-12-15 10:58:19 +01:00
Olivier Goffart
416a0b3482 Test System: Optimisations so the benchmark is significant 2020-12-15 10:58:19 +01:00
Olivier Goffart
bb9ce8db21 Test that the sync behave well if there are errors while reading the database 2020-12-15 10:58:17 +01:00