1
0
Fork 0
Commit Graph

60 Commits

Author SHA1 Message Date
Matthieu Gallien f1d834df8e properly query sync journal DB to know when to run fix for VFS
the new method added to query the db is not working and so the fix for
vfs is executed at each sync run

the new method for bool was not really needed so let's just remove it
(and that will make the usage of SqlQuery be correct

Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
2021-11-19 14:19:04 +01:00
Felix Weilbach 9e792369b2 Refactor key-value store query code
Signed-off-by: Felix Weilbach <felix.weilbach@nextcloud.com>
2021-11-17 09:53:26 +00:00
Felix Weilbach c76a77e431 Correct virtual files placeholder files if needed
In the past not all files were converted to placeholder files when
converting an existing sync folder to a virtual files folder. Because
some files were not converted to placeholder files, the status would
be wrong on the files. This code makes sure that every file in a
virtual files folder is a placeholder file. It could be removed in the
future.

Signed-off-by: Felix Weilbach <felix.weilbach@nextcloud.com>
2021-11-17 09:53:26 +00:00
Felix Weilbach c9863dc225 Use QRecursiveMutex
Signed-off-by: Felix Weilbach <felix.weilbach@nextcloud.com>
2021-09-08 08:54:00 +00:00
Matthieu Gallien 25669938fd let clang-tidy add missing override after enabling again the check
Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
2021-08-31 08:35:56 +00:00
Hannah von Reth de2d11125b Move Prepared sql queries to seperate class to manage access 2021-08-23 09:40:26 +02:00
Hannah von Reth e7e218d8e6 Improve the error message returned by updateMetadata
Also properly handle the case the the file can't be converted to a placeholder
as it is locked
2021-07-06 14:03:53 +00:00
allexzander dc72686ab3 Allow download with mismatched checksum if a config option is set.
Signed-off-by: allexzander <blackslayer4@gmail.com>
2021-06-10 13:39:11 +03:00
Felix Weilbach 78f1943d76 Add a key value store to the sync journal
This key value store should help to keep track of important events,
that can not be store in the logs, because the logs are deleted too fast.

Signed-off-by: Felix Weilbach <felix.weilbach@nextcloud.com>
2021-05-19 09:04:43 +02:00
Christian Kamm 5acb157a7e
VfsSuffix: Wipe stale pin states #7273
Previously the pin states of deleted files stayed in the 'flags'
database and could be inadvertently reused when a new file with the same
name appeared. Now they are deleted.

To make this work right, the meaning of the 'path' column in the 'flags'
table was changed: Previously it never had the .owncloud file suffix.
Now it's the same as in metadata.path.

This takes the safe parts from #7274 for inclusion in 2.6. The more
elaborate database schema changes (why use 'path' the join the two
tables in the first place?) shall go into master.
2020-12-15 10:58:52 +01:00
Christian Kamm edad7ce7d7
Vfs: Mark sqlite temporaries excluded on db-open #7141
The previous patch ensured that the sqlite temporaries weren't deleted
and recreated for every sync run, but there was still time between
client startup and the first sync run where they would have the
"needs-sync" icon.
2020-12-15 10:58:50 +01:00
Christian Kamm 7f3f13fd97
Vfs: "free space" only shows when it has an effect #7143
To do this, introduce AllDehydrated availability and rename
SomeDehydrated to Mixed - it now guarantees there are also hydrated
items.
2020-12-15 10:58:49 +01:00
Christian Kamm dcf34316fd
Vfs: Add 'availability', a simplified, user-facing pin state #7111
The idea is that the user's question is "is this folder's data available
offline?" and not "does this folder have AlwaysLocal pin state?".
The the answers to the two questions can differ: an always-local
folder can have subitems that are not always-local and are dehydrated.

The new availability enum intends to describe the answer to the user's
actual question and can be derived from pin states. If pin states aren't
stored in the database the way of calculating availability will depend
on the vfs plugin.
2020-12-15 10:58:47 +01:00
Olivier Goffart 4346567a03
Async Poll: keep the size in the database
This was not required with 2.5 because a size of 0 was ignorted when comparing
size by the csync updater, to be compatible with very old version of the database.
But the we discovery will still think the file is changed if the database contains
a size of 0
2020-12-15 10:58:44 +01:00
Olivier Goffart 46bf3ed31a
Upload: asynchronious operations
Implements https://github.com/owncloud/core/pull/31851
2020-12-15 10:58:44 +01:00
Christian Kamm 238ac53666
Ensure local discovery on selective sync changes
As far as I'm aware local discovery can be skipped on folders that are
selective-sync blacklisted, so a local discovery is required when an
entry is removed from the blacklist.

Also rename
avoidReadFromDbOnNextSync() -> schedulePathForRemoteDiscovery()
since the old name might also imply it's not read from db in the local
discovery - which is not the case. Use Folder::
schedulePathForLocalDiscovery() for that.
2020-12-15 10:58:41 +01:00
Christian Kamm 4c04351360
Fix warnings about signedness
Sizes are always qint64, not unsigned.
TransferIds are always uint.
2020-12-15 10:58:41 +01:00
Christian Kamm 83a818678f
PinStates cleanup
- SyncJournalDB functions now behind internalPinStates() to avoid
accidental usage, when nearly everyone should go through Vfs.
- Rename Vfs::getPinState() to Vfs::pinState()
2020-12-15 10:58:39 +01:00
Christian Kamm 7f400e3226
Pin state updates
- unspecified and inherited are different
- move enum to header in common/
- access through Vfs instead of directly in Journal
2020-12-15 10:58:38 +01:00
Christian Kamm 22255e4976
vfs: Add vfs migration options to folder context menu
This allows enabling and disabling vfs.

To distinguish this operation from setting the root pin state, the
availability setting is adjusted as well to be similar to the
menu that shows in the shell extensions.
2020-12-15 10:58:33 +01:00
Christian Kamm 20ef0a0290
vfs: Allow retrieving of pin state paths and flags 2020-12-15 10:58:33 +01:00
Christian Kamm 4df101ed84
vfs: Allow (de-)hydrating the full sync folder 2020-12-15 10:58:33 +01:00
Christian Kamm aa382eda29
Db: Add wiping of pin state for subtrees 2020-12-15 10:58:30 +01:00
Christian Kamm d956f518a8
vfs: Remove newFilesAreVirtual - use root PinState instead
This unifies how to deal with pin states.

Also enable reading a folders direct pin state vs its effective pin
state.
2020-12-15 10:58:30 +01:00
Christian Kamm 68126ac208
vfs: Introduce PinState db storage #6815
The idea is to allow folders (and later maybe files?) to be
- pinned to be available locally
- pinned to be online only
- inherit their pin from the parent

Where this pinning only controls the default for new files.
Subfolders may have a different pin state, and contained files
may be hydrated or dehydrated based on user actions.

This value is stored in a new 'flags' table. The idea is to store
data there that doesn't necessarily exist for each metadata entry.
The selective sync state could be migrated to this table.
2020-12-15 10:58:29 +01:00
Olivier Goffart d6a0290058
Optimisation: Add a cache SyncJournalDb::mapChecksumType
No need to do two sql query for something that's always the same and
there are very few checksum types
2020-12-15 10:58:19 +01:00
Olivier Goffart 69de2d5180
Sync: optimize by removing setFileRecordMetadata
Inh most case we already have a record from before, so avoid doing a useless
lookup in the database.
In owncloudpropagator.cpp, directories do not have a checksum so no need
to call a function that preserves it
2020-12-15 10:58:19 +01:00
Olivier Goffart bb9ce8db21
Test that the sync behave well if there are errors while reading the database 2020-12-15 10:58:17 +01:00
Olivier Goffart 58eaf9940a
Database: Add an index on the parent path
So we can quickly query the items in a parent directory

This uses a custom slite3 function, which means that when downgrading the client,
or using another tool to add entries in the database, any insertion in the metadata
table will produce an error: "unknown function: parent_hash()"
(This will crash the client 2.5)
2020-12-15 10:58:13 +01:00
Olivier Goffart c8eff3da2d
New Discovery algorithm: Remove the sync cleanup phase
Since we do not recurse within some directories, many files are not seen.

The stale entry will cleanup by themself as the sync engine try to remove
the files that are already removed.
Should we need to actually do this cleanup, it should be dotected in the
discovery.
2020-12-15 10:58:08 +01:00
Olivier Goffart 4ee244190b
Virtual Files: Allow to download a folder recursively from the socket API
Issue: #6466
2020-12-15 10:57:54 +01:00
Kevin Ottens 0b731eb516 Revert "moved default db sync file to Qt standard path AppDataLocation"
This reverts commit fc6b895f52.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-11-10 17:46:43 +00:00
Christian Kamm 00e901f5a7
SocketAPI: Introduce conflict resolution actions #6252
For conflicts generally as well as new files in read-only directories
the context menu will now present delete and move options.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-10-22 16:40:10 +02:00
Kevin Ottens 80cc196f6f Enable bugprone-narrowing-conversions clang-tidy check
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-09-01 06:37:03 +00:00
Kevin Ottens cdf6e7d72b Enable the modernize-user-nullptr check on clang-tidy
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-08-12 19:45:25 +02:00
Stephan Beyer 3015ea5715 Use default member initializers in OCC::SyncJournalDb sub-structs
This allows to remove their explicit default constructor definitions.

Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
2020-05-21 01:32:16 +02:00
Dominique Fuchs fc6b895f52 moved default db sync file to Qt standard path AppDataLocation
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-26 17:09:24 +02:00
Dominique Fuchs ab3d0141ec initialize _modtime to prevent undefined usage
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2019-09-09 19:23:30 +02:00
Roeland Jago Douma 2dc7ff7332
Merge branch 'master' into upstream/pr/6360 2018-10-19 16:13:09 +02:00
Olivier Goffart 779969f865
OwnSql: Put the SQL queries next to their use.
This avoid haivng to modify 4 places when we want to add a query
2018-07-03 10:33:26 +02:00
Olivier Goffart 518fb90757
OwnSql: Some refactoring
Use QByteArray for the query so we don't have to convert twice.
Automatically finish the query when the DB close, so we don't
have to call reset
2018-07-03 10:33:26 +02:00
Christian Kamm 77fcff5bdf
Conflicts: Change tags to be more user friendly #6365
From "_conflict-user-yyyymmdd-hhmmss"
to   " (conflicted copy user yyyy-mm-dd hhmmss)"
2018-06-11 14:40:40 +02:00
Olivier Goffart 012c638d4e
Uploads: Don't delete unexisting chunks
Since commit 4dc49ff3, we store an entry in the upload info table even
for non chunked uploads. However, if this fails we don't want to remove
non-existant stale chunks if the upload fails.

Without this commit, we would send a DELETE command to clean non-existant
chunks in the dav/uploads/ namespace.
2018-06-06 09:52:02 +02:00
Christian Kamm 09f83f4fa9
SyncJournal: Clear etag filter before sync
Before, we only cleared the _avoidReadFromDbOnNextSyncFilter *after* a
sync which meant that we had to sync twice after selective sync setup.

Now, we clear the filter *before* a sync as well which allows the actual
next sync to write the correct etags to the db again - instead of only
the sync after that one.

Also expand on comments and rename _avoidReadFromDbOnNextSyncFilter to
_etagStorageFilter.
2018-06-05 12:00:20 +02:00
Christian Kamm ce0e2d0b88
SyncJournal: Don't use LIKE with paths
Paths can contain the wildcards % and _ and that would lead to odd
behavior.

This patch also clarifies the behavior of avoidReadFromDbOnNextSync()
which previously dependend on whether "foo/bar" or "foo/bar/" was
passed as input.

Possibly affects #6322
2018-05-15 19:44:00 +02:00
Daniel Nicoletti d93f7269b8 Properly mark encrypted files for removal 2018-04-09 13:23:52 +02:00
Daniel Nicoletti 99117078f3 Remove table for e2e and add an e2eMangledName column (#169)
If the code was not complex enough syncing two tables
already started to give UNIQUE constrains errors on
simple sync operations, this also adds initial support
remote delete of an encrypted file
2018-02-12 12:50:51 +01:00
Tomaz Canabrava d24a1e542e Merge branch 'master' into clientSideEncryptionV3 2018-01-29 14:06:12 +01:00
Tomaz Canabrava a003d216a5 [CSE] Implementation of the setE2eRelation
This call creates a new entry on the database for e2e.
2018-01-28 18:05:28 +01:00
Tomaz Canabrava 32eaaa0601 [CSE] GetMangledName implementation 2018-01-28 17:50:10 +01:00